2023-10-17T00:45:02.992 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-17T00:45:02.993 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} --verbose -- /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/orig.config.yaml 2023-10-17T00:45:03.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T00:45:03.144 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578 branch: wip-yuri4-testing-2023-10-11-0735-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: yweinste@redhat.com first_in_suite: false job_id: '7429578' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri4-testing-2023-10-11-0735-reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: ab9fe43522bc41de59746200b95f65d4fbcccffe subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: ab9fe43522bc41de59746200b95f65d4fbcccffe kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-10-11-0735-reef sha1: ab9fe43522bc41de59746200b95f65d4fbcccffe owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3743 sha1: ab9fe43522bc41de59746200b95f65d4fbcccffe sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri4-testing-2023-10-11-0735-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ab9fe43522bc41de59746200b95f65d4fbcccffe syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_14:43:00 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-17T00:45:03.144 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa; will attempt to use it 2023-10-17T00:45:03.145 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks 2023-10-17T00:45:03.146 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-17T00:45:03.148 INFO:teuthology.task.internal:Checking packages... 2023-10-17T00:45:03.172 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'ab9fe43522bc41de59746200b95f65d4fbcccffe' 2023-10-17T00:45:03.172 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-17T00:45:03.172 INFO:teuthology.packaging:ref: None 2023-10-17T00:45:03.173 INFO:teuthology.packaging:tag: None 2023-10-17T00:45:03.173 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-10-11-0735-reef 2023-10-17T00:45:03.173 INFO:teuthology.packaging:sha1: ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T00:45:03.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2023-10-11-0735-reef 2023-10-17T00:45:03.316 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-668.gab9fe435 2023-10-17T00:45:03.316 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-17T00:45:03.323 INFO:teuthology.task.internal:no buildpackages task found 2023-10-17T00:45:03.323 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-17T00:45:03.329 INFO:teuthology.task.internal:Saving configuration 2023-10-17T00:45:03.351 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-17T00:45:03.358 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-17T00:45:03.384 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-17 00:40:02.310690', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-17T00:45:03.405 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-17 00:40:02.311847', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-17T00:45:03.429 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-17 00:40:02.312726', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-17T00:45:03.429 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-17T00:45:03.435 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-17T00:45:03.436 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-17T00:45:03.436 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-17T00:45:03.436 INFO:teuthology.run_tasks:Running task console_log... 2023-10-17T00:45:03.537 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe63c9aeca0>, signals=[15]) 2023-10-17T00:45:03.538 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-17T00:45:03.544 INFO:teuthology.task.internal:Opening connections... 2023-10-17T00:45:03.545 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T00:45:03.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T00:45:03.618 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2023-10-17T00:45:03.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T00:45:03.686 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-10-17T00:45:03.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T00:45:03.754 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-17T00:45:03.761 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-10-17T00:45:03.783 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-10-17T00:45:03.783 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-10-17T00:45:03.842 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Red Hat Enterprise Linux" 2023-10-17T00:45:03.843 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8.6 (Ootpa)" 2023-10-17T00:45:03.843 INFO:teuthology.orchestra.run.smithi045.stdout:ID="rhel" 2023-10-17T00:45:03.843 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="fedora" 2023-10-17T00:45:03.845 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8.6" 2023-10-17T00:45:03.845 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-10-17T00:45:03.845 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-17T00:45:03.846 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-10-17T00:45:03.846 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-17T00:45:03.846 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.redhat.com/" 2023-10-17T00:45:03.846 INFO:teuthology.orchestra.run.smithi045.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-17T00:45:03.847 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-17T00:45:03.847 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T00:45:03.847 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-17T00:45:03.847 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-17T00:45:03.847 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-17T00:45:03.848 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-17T00:45:03.849 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-10-17T00:45:03.871 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2023-10-17T00:45:03.892 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2023-10-17T00:45:03.892 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2023-10-17T00:45:03.952 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="Red Hat Enterprise Linux" 2023-10-17T00:45:03.952 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8.6 (Ootpa)" 2023-10-17T00:45:03.952 INFO:teuthology.orchestra.run.smithi060.stdout:ID="rhel" 2023-10-17T00:45:03.953 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="fedora" 2023-10-17T00:45:03.953 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8.6" 2023-10-17T00:45:03.953 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8" 2023-10-17T00:45:03.953 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-17T00:45:03.953 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2023-10-17T00:45:03.954 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-17T00:45:03.954 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://www.redhat.com/" 2023-10-17T00:45:03.954 INFO:teuthology.orchestra.run.smithi060.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-17T00:45:03.954 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-17T00:45:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T00:45:03.955 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-17T00:45:03.955 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-17T00:45:03.955 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-17T00:45:03.955 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-17T00:45:03.956 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2023-10-17T00:45:03.979 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-10-17T00:45:03.999 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-10-17T00:45:03.999 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-10-17T00:45:04.058 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-10-17T00:45:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.6 (Ootpa)" 2023-10-17T00:45:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-10-17T00:45:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-10-17T00:45:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.6" 2023-10-17T00:45:04.059 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-10-17T00:45:04.060 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-17T00:45:04.060 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-10-17T00:45:04.060 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-17T00:45:04.060 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-10-17T00:45:04.060 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-17T00:45:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-17T00:45:04.061 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T00:45:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-17T00:45:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-17T00:45:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-17T00:45:04.062 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-17T00:45:04.062 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-10-17T00:45:04.087 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-17T00:45:04.101 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-17T00:45:04.107 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-17T00:45:04.107 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2023-10-17T00:45:04.110 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2023-10-17T00:45:04.112 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-10-17T00:45:04.131 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-17T00:45:04.138 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-17T00:45:04.138 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2023-10-17T00:45:04.170 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2023-10-17T00:45:04.173 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-10-17T00:45:04.204 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-17T00:45:04.282 INFO:teuthology.run_tasks:Running task kernel... 2023-10-17T00:45:04.298 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-17T00:45:04.299 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-17T00:45:04.299 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-17T00:45:04.300 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-17T00:45:04.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T00:45:04.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-17T00:45:04.489 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-17T00:45:04.489 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-17T00:45:04.489 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-10-17T00:45:04.510 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-17T00:45:04.511 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-17T00:45:04.512 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-17T00:45:04.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum upgrade -y linux-firmware 2023-10-17T00:45:05.708 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-10-17T00:45:05.709 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-10-17T00:45:07.133 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-10-17T00:45:08.525 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-17T00:45:12.267 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 512 kB/s | 24 kB 00:00 2023-10-17T00:45:13.064 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-10-17T00:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-10-17T00:45:13.065 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-10-17T00:45:13.184 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-17T00:45:13.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-17T00:45:13.404 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ 2023-10-17T00:45:13.405 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-17T00:45:13.407 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi045.front.sepia.ceph.com to complete... 2023-10-17T00:45:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17 00:45:26 URL:https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [383161052/383161052] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-17T00:45:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:FINISHED --2023-10-17 00:45:26-- 2023-10-17T00:45:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:Total wall clock time: 13s 2023-10-17T00:45:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:Downloaded: 1 files, 365M in 13s (28.9 MB/s) 2023-10-17T00:45:26.309 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-17T00:45:26.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-17T00:45:27.965 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying... ######################################## 2023-10-17T00:45:28.023 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing... ######################################## 2023-10-17T00:45:28.116 INFO:teuthology.orchestra.run.smithi045.stdout:Updating / installing... 2023-10-17T00:46:08.919 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-17T00:46:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-17T00:46:33.055 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-17T00:46:33.055 DEBUG:teuthology.orchestra.run.smithi045:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-17T00:46:33.112 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.112 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.113 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-17T00:46:33.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-17T00:46:33.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-17T00:46:33.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-17T00:46:33.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-17T00:46:33.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-17T00:46:33.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-17T00:46:33.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-17T00:46:33.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-17T00:46:33.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-17T00:46:33.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-17T00:46:33.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-17T00:46:33.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-17T00:46:33.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-17T00:46:33.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-17T00:46:33.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-17T00:46:33.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-17T00:46:33.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-17T00:46:33.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-17T00:46:33.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-17T00:46:33.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-17T00:46:33.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-17T00:46:33.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-17T00:46:33.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-17T00:46:33.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-17T00:46:33.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-17T00:46:33.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-17T00:46:33.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-17T00:46:33.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-17T00:46:33.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-17T00:46:33.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-17T00:46:33.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-17T00:46:33.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-17T00:46:33.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-17T00:46:33.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-17T00:46:33.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-17T00:46:33.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-17T00:46:33.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-17T00:46:33.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-17T00:46:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-17T00:46:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-17T00:46:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-17T00:46:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-17T00:46:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-17T00:46:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-17T00:46:33.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-17T00:46:33.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-17T00:46:33.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-17T00:46:33.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-17T00:46:33.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-17T00:46:33.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-17T00:46:33.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-17T00:46:33.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-17T00:46:33.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-17T00:46:33.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-17T00:46:33.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-17T00:46:33.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-17T00:46:33.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-17T00:46:33.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-17T00:46:33.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-17T00:46:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-17T00:46:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-17T00:46:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-17T00:46:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-17T00:46:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-17T00:46:33.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-17T00:46:33.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-17T00:46:33.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-17T00:46:33.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-17T00:46:33.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-17T00:46:33.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-17T00:46:33.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-17T00:46:33.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-17T00:46:33.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-17T00:46:33.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-17T00:46:33.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-17T00:46:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-17T00:46:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-17T00:46:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-17T00:46:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-17T00:46:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-17T00:46:33.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-17T00:46:33.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-17T00:46:33.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-17T00:46:33.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-17T00:46:33.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-17T00:46:33.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-17T00:46:33.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-17T00:46:33.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-17T00:46:33.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-17T00:46:33.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-17T00:46:33.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-17T00:46:33.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-17T00:46:33.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-17T00:46:33.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-17T00:46:33.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-17T00:46:33.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-17T00:46:33.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-17T00:46:33.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-17T00:46:33.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-17T00:46:33.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-17T00:46:33.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-17T00:46:33.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-17T00:46:33.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-17T00:46:33.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-17T00:46:33.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-17T00:46:33.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-17T00:46:33.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-17T00:46:33.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-17T00:46:33.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-17T00:46:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-17T00:46:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-17T00:46:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-17T00:46:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-17T00:46:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-17T00:46:33.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-17T00:46:33.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-17T00:46:33.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-17T00:46:33.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-17T00:46:33.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-17T00:46:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-17T00:46:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-17T00:46:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-17T00:46:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-17T00:46:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-17T00:46:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-17T00:46:33.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-17T00:46:33.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-17T00:46:33.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-17T00:46:33.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-17T00:46:33.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-17T00:46:33.138 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-17T00:46:33.138 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-17T00:46:33.138 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-17T00:46:33.138 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-17T00:46:33.138 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-17T00:46:33.139 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-17T00:46:33.139 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-17T00:46:33.139 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-17T00:46:33.139 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-17T00:46:33.139 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-17T00:46:33.139 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-17T00:46:33.140 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-17T00:46:33.140 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-17T00:46:33.140 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-17T00:46:33.140 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-17T00:46:33.140 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-17T00:46:33.140 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-17T00:46:33.141 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-17T00:46:33.141 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-17T00:46:33.141 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-17T00:46:33.141 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-17T00:46:33.141 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-17T00:46:33.142 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-17T00:46:33.142 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-17T00:46:33.142 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-17T00:46:33.142 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-17T00:46:33.142 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-17T00:46:33.143 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-17T00:46:33.143 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-17T00:46:33.143 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-17T00:46:33.143 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-17T00:46:33.143 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-17T00:46:33.143 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-17T00:46:33.144 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-17T00:46:33.144 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-17T00:46:33.144 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-17T00:46:33.144 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-17T00:46:33.144 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-17T00:46:33.145 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-17T00:46:33.145 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-17T00:46:33.145 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-17T00:46:33.145 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-17T00:46:33.145 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-17T00:46:33.146 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-17T00:46:33.146 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-17T00:46:33.146 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-17T00:46:33.146 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-17T00:46:33.146 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-17T00:46:33.147 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-17T00:46:33.147 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-17T00:46:33.147 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-17T00:46:33.147 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-17T00:46:33.147 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-17T00:46:33.148 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-17T00:46:33.148 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-17T00:46:33.148 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-17T00:46:33.148 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-17T00:46:33.148 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-17T00:46:33.149 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-17T00:46:33.149 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-17T00:46:33.149 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-17T00:46:33.149 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-17T00:46:33.149 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-17T00:46:33.150 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-17T00:46:33.150 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-17T00:46:33.150 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-17T00:46:33.150 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-17T00:46:33.150 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-17T00:46:33.151 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-17T00:46:33.151 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-17T00:46:33.151 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-17T00:46:33.151 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-17T00:46:33.151 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-17T00:46:33.152 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-17T00:46:33.152 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-17T00:46:33.152 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-17T00:46:33.152 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-17T00:46:33.152 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-17T00:46:33.153 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-17T00:46:33.153 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-17T00:46:33.153 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-17T00:46:33.153 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-17T00:46:33.153 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-17T00:46:33.153 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-17T00:46:33.154 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-17T00:46:33.154 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-17T00:46:33.154 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-17T00:46:33.154 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-17T00:46:33.154 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-17T00:46:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-17T00:46:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-17T00:46:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-17T00:46:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-17T00:46:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-17T00:46:33.156 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-17T00:46:33.156 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-17T00:46:33.156 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-17T00:46:33.156 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-17T00:46:33.156 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-17T00:46:33.157 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-17T00:46:33.157 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-17T00:46:33.157 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-17T00:46:33.157 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-17T00:46:33.157 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-17T00:46:33.158 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-17T00:46:33.158 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-17T00:46:33.158 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-17T00:46:33.158 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-17T00:46:33.158 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-17T00:46:33.159 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-17T00:46:33.159 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-17T00:46:33.159 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-17T00:46:33.159 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-17T00:46:33.159 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-17T00:46:33.160 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-17T00:46:33.160 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-17T00:46:33.160 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-17T00:46:33.160 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-17T00:46:33.160 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-17T00:46:33.160 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-17T00:46:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-17T00:46:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-17T00:46:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-17T00:46:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-17T00:46:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-17T00:46:33.162 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-17T00:46:33.162 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-17T00:46:33.162 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-17T00:46:33.162 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-17T00:46:33.162 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-17T00:46:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-17T00:46:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-17T00:46:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-17T00:46:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-17T00:46:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-17T00:46:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-17T00:46:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-17T00:46:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-17T00:46:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-17T00:46:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-17T00:46:33.165 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-17T00:46:33.165 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-17T00:46:33.165 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-17T00:46:33.165 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-17T00:46:33.165 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-17T00:46:33.165 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-17T00:46:33.166 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-17T00:46:33.166 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-17T00:46:33.166 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-17T00:46:33.166 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-17T00:46:33.166 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-17T00:46:33.167 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-17T00:46:33.167 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-17T00:46:33.167 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-17T00:46:33.167 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-17T00:46:33.167 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-17T00:46:33.168 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-17T00:46:33.168 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-17T00:46:33.168 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-17T00:46:33.168 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-17T00:46:33.168 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-17T00:46:33.168 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-17T00:46:33.169 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-17T00:46:33.169 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-17T00:46:33.169 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-17T00:46:33.169 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-17T00:46:33.169 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-17T00:46:33.170 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-17T00:46:33.170 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-17T00:46:33.170 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-17T00:46:33.170 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-17T00:46:33.170 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-17T00:46:33.171 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-17T00:46:33.171 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-17T00:46:33.171 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-17T00:46:33.171 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-17T00:46:33.171 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-17T00:46:33.172 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-17T00:46:33.172 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-17T00:46:33.172 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-17T00:46:33.172 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-17T00:46:33.172 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-17T00:46:33.172 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-17T00:46:33.173 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-17T00:46:33.173 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-17T00:46:33.173 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-17T00:46:33.173 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-17T00:46:33.173 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-17T00:46:33.174 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-17T00:46:33.174 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-17T00:46:33.174 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-17T00:46:33.174 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-17T00:46:33.174 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-17T00:46:33.175 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-17T00:46:33.175 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-17T00:46:33.175 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-17T00:46:33.175 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-17T00:46:33.175 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-17T00:46:33.176 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-17T00:46:33.176 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-17T00:46:33.176 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-17T00:46:33.176 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-17T00:46:33.176 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-17T00:46:33.176 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-17T00:46:33.177 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-17T00:46:33.177 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-17T00:46:33.177 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-17T00:46:33.177 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-17T00:46:33.177 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-17T00:46:33.177 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-17T00:46:33.178 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-17T00:46:33.178 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-17T00:46:33.178 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-17T00:46:33.178 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-17T00:46:33.178 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-17T00:46:33.179 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-17T00:46:33.179 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-17T00:46:33.179 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-17T00:46:33.179 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-17T00:46:33.179 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-17T00:46:33.179 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-17T00:46:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-17T00:46:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-17T00:46:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-17T00:46:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-17T00:46:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-17T00:46:33.180 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-17T00:46:33.181 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-17T00:46:33.181 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-17T00:46:33.181 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-17T00:46:33.181 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-17T00:46:33.181 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-17T00:46:33.181 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-17T00:46:33.182 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-17T00:46:33.182 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-17T00:46:33.182 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-17T00:46:33.182 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-17T00:46:33.182 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-17T00:46:33.183 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-17T00:46:33.183 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-17T00:46:33.183 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-17T00:46:33.183 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-17T00:46:33.183 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-17T00:46:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-17T00:46:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-17T00:46:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-17T00:46:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-17T00:46:33.184 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-17T00:46:33.185 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-17T00:46:33.185 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-17T00:46:33.185 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-17T00:46:33.185 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-17T00:46:33.185 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-17T00:46:33.185 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-17T00:46:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-17T00:46:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-17T00:46:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-17T00:46:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-17T00:46:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-17T00:46:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-17T00:46:33.187 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-17T00:46:33.187 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-17T00:46:33.187 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-17T00:46:33.187 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-17T00:46:33.187 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-17T00:46:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-17T00:46:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-17T00:46:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-17T00:46:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-17T00:46:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-17T00:46:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-17T00:46:33.189 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-17T00:46:33.189 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-17T00:46:33.189 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-17T00:46:33.189 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-17T00:46:33.189 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-17T00:46:33.189 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-17T00:46:33.190 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-17T00:46:33.190 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-17T00:46:33.190 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-17T00:46:33.190 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-17T00:46:33.190 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-17T00:46:33.191 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-17T00:46:33.191 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-17T00:46:33.191 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-17T00:46:33.191 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-17T00:46:33.191 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-17T00:46:33.191 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-17T00:46:33.192 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-17T00:46:33.192 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-17T00:46:33.192 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-17T00:46:33.192 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-17T00:46:33.192 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-17T00:46:33.193 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-17T00:46:33.193 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-17T00:46:33.193 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-17T00:46:33.193 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-17T00:46:33.193 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-17T00:46:33.193 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-17T00:46:33.194 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-17T00:46:33.194 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-17T00:46:33.194 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-17T00:46:33.194 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-17T00:46:33.194 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-17T00:46:33.195 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-17T00:46:33.195 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-17T00:46:33.195 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-17T00:46:33.195 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-17T00:46:33.195 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-17T00:46:33.196 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-17T00:46:33.196 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-17T00:46:33.196 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-17T00:46:33.196 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-17T00:46:33.196 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-17T00:46:33.196 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-17T00:46:33.197 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-17T00:46:33.197 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-17T00:46:33.197 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-17T00:46:33.197 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-17T00:46:33.197 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-17T00:46:33.197 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-17T00:46:33.198 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-17T00:46:33.198 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-17T00:46:33.198 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-17T00:46:33.198 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-17T00:46:33.198 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-17T00:46:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-17T00:46:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-17T00:46:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-17T00:46:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-17T00:46:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-17T00:46:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-17T00:46:33.200 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-17T00:46:33.200 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-17T00:46:33.200 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-17T00:46:33.200 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-17T00:46:33.200 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-17T00:46:33.200 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-17T00:46:33.201 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-17T00:46:33.201 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-17T00:46:33.201 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-17T00:46:33.201 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-17T00:46:33.201 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-17T00:46:33.202 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-17T00:46:33.202 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-17T00:46:33.202 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-17T00:46:33.202 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-17T00:46:33.202 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-17T00:46:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-17T00:46:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-17T00:46:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-17T00:46:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-17T00:46:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-17T00:46:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-17T00:46:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-17T00:46:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-17T00:46:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-17T00:46:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-17T00:46:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-17T00:46:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-17T00:46:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-17T00:46:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-17T00:46:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-17T00:46:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-17T00:46:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-17T00:46:33.206 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-17T00:46:33.206 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-17T00:46:33.206 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-17T00:46:33.206 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-17T00:46:33.206 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-17T00:46:33.206 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-17T00:46:33.207 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-17T00:46:33.207 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-17T00:46:33.207 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-17T00:46:33.207 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-17T00:46:33.207 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-17T00:46:33.207 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-17T00:46:33.208 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-17T00:46:33.208 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-17T00:46:33.208 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-17T00:46:33.208 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-17T00:46:33.208 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-17T00:46:33.209 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-17T00:46:33.209 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-17T00:46:33.209 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-17T00:46:33.209 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-17T00:46:33.209 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-17T00:46:33.209 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-17T00:46:33.210 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-17T00:46:33.210 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-17T00:46:33.210 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-17T00:46:33.210 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-17T00:46:33.210 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-17T00:46:33.211 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-17T00:46:33.211 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-17T00:46:33.211 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-17T00:46:33.211 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-17T00:46:33.211 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-17T00:46:33.211 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-17T00:46:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-17T00:46:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-17T00:46:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-17T00:46:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-17T00:46:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-17T00:46:33.212 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-17T00:46:33.213 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-17T00:46:33.213 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-17T00:46:33.213 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-17T00:46:33.213 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-17T00:46:33.213 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-17T00:46:33.213 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-17T00:46:33.214 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-17T00:46:33.214 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-17T00:46:33.214 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-17T00:46:33.214 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-17T00:46:33.215 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-17T00:46:33.215 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-17T00:46:33.215 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-17T00:46:33.215 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-17T00:46:33.215 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-17T00:46:33.215 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-17T00:46:33.216 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-17T00:46:33.216 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-17T00:46:33.216 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-17T00:46:33.216 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-17T00:46:33.216 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-17T00:46:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-17T00:46:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-17T00:46:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-17T00:46:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-17T00:46:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-17T00:46:33.217 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-17T00:46:33.218 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-17T00:46:33.218 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-17T00:46:33.218 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-17T00:46:33.218 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-17T00:46:33.218 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-17T00:46:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-17T00:46:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-17T00:46:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-17T00:46:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-17T00:46:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-17T00:46:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-17T00:46:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-17T00:46:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-17T00:46:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-17T00:46:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-17T00:46:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-17T00:46:33.221 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-17T00:46:33.221 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-17T00:46:33.221 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-17T00:46:33.221 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-17T00:46:33.221 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-17T00:46:33.221 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-17T00:46:33.222 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-17T00:46:33.222 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-17T00:46:33.222 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-17T00:46:33.222 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-17T00:46:33.222 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-17T00:46:33.223 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-17T00:46:33.223 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-17T00:46:33.223 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-17T00:46:33.223 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-17T00:46:33.223 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-17T00:46:33.223 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-17T00:46:33.224 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-17T00:46:33.224 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-17T00:46:33.224 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-17T00:46:33.224 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-17T00:46:33.224 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-17T00:46:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-17T00:46:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-17T00:46:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-17T00:46:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-17T00:46:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-17T00:46:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-17T00:46:33.226 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-17T00:46:33.226 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-17T00:46:33.226 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-17T00:46:33.226 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-17T00:46:33.226 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-17T00:46:33.227 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-17T00:46:33.227 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-17T00:46:33.227 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-17T00:46:33.227 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-17T00:46:33.227 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-17T00:46:33.228 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-17T00:46:33.228 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-17T00:46:33.228 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-17T00:46:33.228 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-17T00:46:33.228 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-17T00:46:33.228 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-17T00:46:33.229 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-17T00:46:33.229 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-17T00:46:33.229 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-17T00:46:33.229 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-17T00:46:33.229 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-17T00:46:33.230 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-17T00:46:33.230 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-17T00:46:33.230 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-17T00:46:33.230 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-17T00:46:33.230 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-17T00:46:33.231 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-17T00:46:33.231 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-17T00:46:33.231 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-17T00:46:33.231 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-17T00:46:33.231 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-17T00:46:33.232 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-17T00:46:33.232 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-17T00:46:33.232 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-17T00:46:33.232 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-17T00:46:33.232 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-17T00:46:33.232 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-17T00:46:33.233 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-17T00:46:33.233 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-17T00:46:33.233 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-17T00:46:33.233 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-17T00:46:33.233 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-17T00:46:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-17T00:46:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-17T00:46:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-17T00:46:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-17T00:46:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-17T00:46:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-17T00:46:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-17T00:46:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-17T00:46:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-17T00:46:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-17T00:46:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-17T00:46:33.236 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-17T00:46:33.236 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-17T00:46:33.236 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-17T00:46:33.236 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-17T00:46:33.236 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-17T00:46:33.237 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-17T00:46:33.237 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-17T00:46:33.237 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-17T00:46:33.237 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-17T00:46:33.237 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-17T00:46:33.238 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-17T00:46:33.238 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-17T00:46:33.238 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-17T00:46:33.238 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-17T00:46:33.238 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-17T00:46:33.238 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-17T00:46:33.239 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-17T00:46:33.239 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-17T00:46:33.239 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-17T00:46:33.239 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-17T00:46:33.239 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-17T00:46:33.239 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-17T00:46:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-17T00:46:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-17T00:46:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-17T00:46:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-17T00:46:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-17T00:46:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-17T00:46:33.241 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-17T00:46:33.241 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-17T00:46:33.241 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-17T00:46:33.241 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-17T00:46:33.241 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-17T00:46:33.242 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-17T00:46:33.242 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-17T00:46:33.242 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-17T00:46:33.242 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-17T00:46:33.242 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-17T00:46:33.243 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-17T00:46:33.243 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-17T00:46:33.243 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-17T00:46:33.243 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-17T00:46:33.243 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-17T00:46:33.243 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-17T00:46:33.244 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-17T00:46:33.244 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-17T00:46:33.244 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-17T00:46:33.244 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-17T00:46:33.244 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-17T00:46:33.245 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-17T00:46:33.245 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-17T00:46:33.245 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-17T00:46:33.245 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-17T00:46:33.245 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-17T00:46:33.246 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-17T00:46:33.246 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-17T00:46:33.246 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-17T00:46:33.246 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-17T00:46:33.246 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-17T00:46:33.247 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-17T00:46:33.247 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-17T00:46:33.247 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-17T00:46:33.247 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-17T00:46:33.247 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-17T00:46:33.247 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-17T00:46:33.248 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-17T00:46:33.248 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-17T00:46:33.248 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-17T00:46:33.248 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-17T00:46:33.248 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-17T00:46:33.249 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-17T00:46:33.249 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-17T00:46:33.249 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-17T00:46:33.249 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-17T00:46:33.249 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-17T00:46:33.249 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-17T00:46:33.250 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-17T00:46:33.250 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-17T00:46:33.250 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-17T00:46:33.250 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-17T00:46:33.250 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-17T00:46:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-17T00:46:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-17T00:46:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-17T00:46:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-17T00:46:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-17T00:46:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-17T00:46:33.252 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-17T00:46:33.252 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-17T00:46:33.252 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-17T00:46:33.252 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-17T00:46:33.252 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-17T00:46:33.252 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-17T00:46:33.253 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-17T00:46:33.253 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-17T00:46:33.253 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-17T00:46:33.253 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-17T00:46:33.253 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-17T00:46:33.253 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-17T00:46:33.254 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-17T00:46:33.254 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-17T00:46:33.254 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-17T00:46:33.254 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-17T00:46:33.254 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-17T00:46:33.255 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-17T00:46:33.255 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-17T00:46:33.255 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-17T00:46:33.255 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-17T00:46:33.255 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-17T00:46:33.255 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-17T00:46:33.256 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-17T00:46:33.256 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-17T00:46:33.256 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-17T00:46:33.256 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-17T00:46:33.256 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-17T00:46:33.256 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-17T00:46:33.257 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-17T00:46:33.257 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-17T00:46:33.257 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-17T00:46:33.257 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-17T00:46:33.257 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-17T00:46:33.257 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-17T00:46:33.258 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-17T00:46:33.258 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-17T00:46:33.258 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-17T00:46:33.258 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-17T00:46:33.258 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-17T00:46:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-17T00:46:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-17T00:46:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-17T00:46:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-17T00:46:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-17T00:46:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-17T00:46:33.260 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-17T00:46:33.260 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-17T00:46:33.260 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-17T00:46:33.260 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-17T00:46:33.260 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-17T00:46:33.260 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-17T00:46:33.261 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-17T00:46:33.261 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-17T00:46:33.261 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-17T00:46:33.261 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-17T00:46:33.261 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-17T00:46:33.261 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-17T00:46:33.262 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-17T00:46:33.262 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-17T00:46:33.262 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-17T00:46:33.262 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-17T00:46:33.262 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-17T00:46:33.263 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-17T00:46:33.263 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-17T00:46:33.263 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-17T00:46:33.263 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-17T00:46:33.263 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-17T00:46:33.263 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-17T00:46:33.264 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-17T00:46:33.264 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-17T00:46:33.264 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-17T00:46:33.264 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-17T00:46:33.264 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-17T00:46:33.264 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-17T00:46:33.265 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-17T00:46:33.265 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-17T00:46:33.265 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-17T00:46:33.265 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-17T00:46:33.265 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-17T00:46:33.266 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-17T00:46:33.266 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-17T00:46:33.266 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-17T00:46:33.266 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-17T00:46:33.266 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-17T00:46:33.267 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-17T00:46:33.267 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-17T00:46:33.267 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-17T00:46:33.267 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-17T00:46:33.267 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-17T00:46:33.267 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-17T00:46:33.268 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-17T00:46:33.268 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-17T00:46:33.268 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-17T00:46:33.268 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-17T00:46:33.268 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-17T00:46:33.268 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-17T00:46:33.269 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-17T00:46:33.269 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-17T00:46:33.269 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-17T00:46:33.269 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-17T00:46:33.269 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-17T00:46:33.269 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-17T00:46:33.270 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-17T00:46:33.270 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-17T00:46:33.270 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-17T00:46:33.270 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-17T00:46:33.270 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-17T00:46:33.271 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-17T00:46:33.271 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-17T00:46:33.271 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-17T00:46:33.271 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-17T00:46:33.271 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-17T00:46:33.271 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-17T00:46:33.272 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-17T00:46:33.272 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-17T00:46:33.272 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-17T00:46:33.272 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-17T00:46:33.272 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-17T00:46:33.272 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-17T00:46:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-17T00:46:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-17T00:46:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-17T00:46:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-17T00:46:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-17T00:46:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-17T00:46:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-17T00:46:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-17T00:46:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-17T00:46:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-17T00:46:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-17T00:46:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-17T00:46:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-17T00:46:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-17T00:46:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-17T00:46:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-17T00:46:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-17T00:46:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-17T00:46:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-17T00:46:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-17T00:46:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-17T00:46:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-17T00:46:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-17T00:46:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-17T00:46:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-17T00:46:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-17T00:46:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-17T00:46:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-17T00:46:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-17T00:46:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-17T00:46:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-17T00:46:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-17T00:46:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-17T00:46:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-17T00:46:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-17T00:46:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-17T00:46:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-17T00:46:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-17T00:46:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-17T00:46:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-17T00:46:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-17T00:46:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-17T00:46:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-17T00:46:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-17T00:46:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-17T00:46:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-17T00:46:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-17T00:46:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-17T00:46:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-17T00:46:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-17T00:46:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-17T00:46:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-17T00:46:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-17T00:46:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-17T00:46:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-17T00:46:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-17T00:46:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-17T00:46:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-17T00:46:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-17T00:46:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-17T00:46:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-17T00:46:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-17T00:46:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-17T00:46:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-17T00:46:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-17T00:46:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-17T00:46:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-17T00:46:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-17T00:46:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-17T00:46:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-17T00:46:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-17T00:46:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-17T00:46:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-17T00:46:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-17T00:46:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-17T00:46:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-17T00:46:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-17T00:46:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-17T00:46:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-17T00:46:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-17T00:46:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-17T00:46:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-17T00:46:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-17T00:46:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-17T00:46:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-17T00:46:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-17T00:46:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-17T00:46:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-17T00:46:33.288 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-17T00:46:33.288 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-17T00:46:33.288 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-17T00:46:33.288 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-17T00:46:33.288 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-17T00:46:33.288 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-17T00:46:33.289 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-17T00:46:33.289 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-17T00:46:33.289 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-17T00:46:33.289 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-17T00:46:33.289 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-17T00:46:33.289 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-17T00:46:33.290 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-17T00:46:33.290 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-17T00:46:33.290 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-17T00:46:33.290 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-17T00:46:33.290 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-17T00:46:33.290 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-17T00:46:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-17T00:46:33.292 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-17T00:46:33.292 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-17T00:46:33.292 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-17T00:46:33.292 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-17T00:46:33.292 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-17T00:46:33.292 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-17T00:46:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-17T00:46:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-17T00:46:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-17T00:46:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-17T00:46:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-17T00:46:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-17T00:46:33.294 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-17T00:46:33.294 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-17T00:46:33.294 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-17T00:46:33.294 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-17T00:46:33.294 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-17T00:46:33.295 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-17T00:46:33.296 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-17T00:46:33.296 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-17T00:46:33.296 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-17T00:46:33.296 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-17T00:46:33.296 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-17T00:46:33.297 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-17T00:46:33.297 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-17T00:46:33.297 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-17T00:46:33.297 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-17T00:46:33.297 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-17T00:46:33.297 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-17T00:46:33.298 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-17T00:46:33.298 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-17T00:46:33.298 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-17T00:46:33.298 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-17T00:46:33.298 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-17T00:46:33.298 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-17T00:46:33.299 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-17T00:46:33.299 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-17T00:46:33.299 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-17T00:46:33.299 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-17T00:46:33.299 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-17T00:46:33.299 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-17T00:46:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-17T00:46:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-17T00:46:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-17T00:46:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-17T00:46:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-17T00:46:33.300 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-17T00:46:33.301 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-17T00:46:33.301 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-17T00:46:33.301 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-17T00:46:33.301 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-17T00:46:33.301 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-17T00:46:33.302 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-17T00:46:33.302 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-17T00:46:33.302 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-17T00:46:33.302 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-17T00:46:33.302 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-17T00:46:33.302 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-17T00:46:33.303 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-17T00:46:33.303 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-17T00:46:33.303 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-17T00:46:33.303 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-17T00:46:33.303 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-17T00:46:33.303 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-17T00:46:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-17T00:46:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-17T00:46:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-17T00:46:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-17T00:46:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-17T00:46:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-17T00:46:33.305 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-17T00:46:33.305 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-17T00:46:33.305 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-17T00:46:33.305 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-17T00:46:33.305 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-17T00:46:33.306 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-17T00:46:33.306 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-17T00:46:33.306 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-17T00:46:33.306 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-17T00:46:33.306 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-17T00:46:33.306 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-17T00:46:33.307 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-17T00:46:33.307 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-17T00:46:33.307 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-17T00:46:33.307 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-17T00:46:33.307 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-17T00:46:33.307 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-17T00:46:33.308 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-17T00:46:33.308 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-17T00:46:33.308 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-17T00:46:33.308 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-17T00:46:33.308 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-17T00:46:33.308 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-17T00:46:33.309 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-17T00:46:33.309 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-17T00:46:33.309 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-17T00:46:33.309 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-17T00:46:33.309 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-17T00:46:33.309 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-17T00:46:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-17T00:46:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-17T00:46:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-17T00:46:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-17T00:46:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-17T00:46:33.310 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-17T00:46:33.311 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-17T00:46:33.311 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-17T00:46:33.311 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-17T00:46:33.311 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-17T00:46:33.311 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-17T00:46:33.312 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-17T00:46:33.312 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-17T00:46:33.312 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-17T00:46:33.312 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-17T00:46:33.312 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-17T00:46:33.312 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-17T00:46:33.313 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-17T00:46:33.313 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-17T00:46:33.313 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-17T00:46:33.313 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-17T00:46:33.313 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-17T00:46:33.313 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-17T00:46:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-17T00:46:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-17T00:46:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-17T00:46:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-17T00:46:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-17T00:46:33.314 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-17T00:46:33.315 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-17T00:46:33.315 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-17T00:46:33.315 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-17T00:46:33.315 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-17T00:46:33.315 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-17T00:46:33.316 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-17T00:46:33.316 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-17T00:46:33.316 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-17T00:46:33.316 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-17T00:46:33.316 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-17T00:46:33.316 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-17T00:46:33.317 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-17T00:46:33.317 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-17T00:46:33.317 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-17T00:46:33.317 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-17T00:46:33.317 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-17T00:46:33.317 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-17T00:46:33.318 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-17T00:46:33.318 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-17T00:46:33.318 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-17T00:46:33.318 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-17T00:46:33.318 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-17T00:46:33.319 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-17T00:46:33.319 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-17T00:46:33.319 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-17T00:46:33.319 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-17T00:46:33.319 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-17T00:46:33.319 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-17T00:46:33.320 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-17T00:46:33.320 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-17T00:46:33.320 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-17T00:46:33.320 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-17T00:46:33.320 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-17T00:46:33.320 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-17T00:46:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-17T00:46:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-17T00:46:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-17T00:46:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-17T00:46:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-17T00:46:33.321 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-17T00:46:33.322 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-17T00:46:33.322 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-17T00:46:33.322 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-17T00:46:33.322 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-17T00:46:33.322 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-17T00:46:33.323 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-17T00:46:33.323 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-17T00:46:33.323 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-17T00:46:33.323 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-17T00:46:33.323 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-17T00:46:33.323 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-17T00:46:33.324 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-17T00:46:33.324 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-17T00:46:33.324 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-17T00:46:33.324 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-17T00:46:33.324 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-17T00:46:33.324 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-17T00:46:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-17T00:46:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-17T00:46:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-17T00:46:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-17T00:46:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-17T00:46:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-17T00:46:33.326 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-17T00:46:33.326 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-17T00:46:33.326 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-17T00:46:33.326 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-17T00:46:33.326 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-17T00:46:33.326 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-17T00:46:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-17T00:46:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-17T00:46:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-17T00:46:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-17T00:46:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-17T00:46:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-17T00:46:33.328 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-17T00:46:33.328 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-17T00:46:33.328 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-17T00:46:33.328 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-17T00:46:33.328 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-17T00:46:33.329 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-17T00:46:33.329 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-17T00:46:33.329 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-17T00:46:33.329 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-17T00:46:33.329 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-17T00:46:33.329 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-17T00:46:33.330 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-17T00:46:33.331 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-17T00:46:33.331 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-17T00:46:33.333 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-17T00:46:33.333 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-17T00:46:33.333 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-17T00:46:33.333 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-17T00:46:33.333 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-17T00:46:33.334 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-17T00:46:33.334 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-17T00:46:33.334 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-17T00:46:33.334 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-17T00:46:33.334 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-17T00:46:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-17T00:46:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-17T00:46:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-17T00:46:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-17T00:46:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-17T00:46:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-17T00:46:33.336 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-17T00:46:33.336 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-17T00:46:33.336 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-17T00:46:33.336 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-17T00:46:33.336 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-17T00:46:33.336 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-17T00:46:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-17T00:46:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-17T00:46:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-17T00:46:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-17T00:46:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-17T00:46:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-17T00:46:33.338 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-17T00:46:33.338 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-17T00:46:33.338 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-17T00:46:33.338 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-17T00:46:33.338 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-17T00:46:33.339 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-17T00:46:33.339 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-17T00:46:33.339 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-17T00:46:33.339 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-17T00:46:33.339 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-17T00:46:33.339 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-17T00:46:33.340 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-17T00:46:33.340 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-17T00:46:33.340 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-17T00:46:33.340 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-17T00:46:33.340 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-17T00:46:33.340 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-17T00:46:33.341 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-17T00:46:33.342 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-17T00:46:33.342 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-17T00:46:33.342 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-17T00:46:33.342 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-17T00:46:33.342 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-17T00:46:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-17T00:46:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-17T00:46:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-17T00:46:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-17T00:46:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-17T00:46:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-17T00:46:33.344 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-17T00:46:33.344 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-17T00:46:33.344 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-17T00:46:33.344 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-17T00:46:33.344 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-17T00:46:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-17T00:46:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-17T00:46:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-17T00:46:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-17T00:46:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-17T00:46:33.345 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-17T00:46:33.346 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-17T00:46:33.346 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-17T00:46:33.346 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-17T00:46:33.346 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-17T00:46:33.346 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-17T00:46:33.347 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-17T00:46:33.347 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-17T00:46:33.347 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-17T00:46:33.347 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-17T00:46:33.347 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-17T00:46:33.347 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-17T00:46:33.348 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-17T00:46:33.348 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-17T00:46:33.348 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-17T00:46:33.348 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-17T00:46:33.348 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-17T00:46:33.349 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-17T00:46:33.349 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-17T00:46:33.349 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-17T00:46:33.349 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-17T00:46:33.349 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-17T00:46:33.349 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-17T00:46:33.350 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-17T00:46:33.350 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-17T00:46:33.350 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-17T00:46:33.350 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-17T00:46:33.350 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-17T00:46:33.350 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-17T00:46:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-17T00:46:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-17T00:46:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-17T00:46:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-17T00:46:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-17T00:46:33.351 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-17T00:46:33.352 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-17T00:46:33.352 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-17T00:46:33.352 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-17T00:46:33.352 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-17T00:46:33.352 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-17T00:46:33.352 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-17T00:46:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-17T00:46:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-17T00:46:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-17T00:46:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-17T00:46:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-17T00:46:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-17T00:46:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-17T00:46:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-17T00:46:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-17T00:46:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-17T00:46:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-17T00:46:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-17T00:46:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-17T00:46:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-17T00:46:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-17T00:46:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-17T00:46:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-17T00:46:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-17T00:46:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-17T00:46:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-17T00:46:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-17T00:46:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-17T00:46:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-17T00:46:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-17T00:46:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-17T00:46:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-17T00:46:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-17T00:46:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-17T00:46:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-17T00:46:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-17T00:46:33.358 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-17T00:46:33.358 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-17T00:46:33.358 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-17T00:46:33.358 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-17T00:46:33.358 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-17T00:46:33.358 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-17T00:46:33.359 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-17T00:46:33.359 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-17T00:46:33.359 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-17T00:46:33.359 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-17T00:46:33.359 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-17T00:46:33.359 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-17T00:46:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-17T00:46:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-17T00:46:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-17T00:46:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-17T00:46:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-17T00:46:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-17T00:46:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-17T00:46:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-17T00:46:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-17T00:46:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-17T00:46:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-17T00:46:33.361 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-17T00:46:33.362 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-17T00:46:33.362 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-17T00:46:33.362 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-17T00:46:33.362 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-17T00:46:33.362 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-17T00:46:33.362 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-17T00:46:33.363 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-17T00:46:33.363 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-17T00:46:33.363 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-17T00:46:33.363 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-17T00:46:33.363 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-17T00:46:33.363 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-17T00:46:33.364 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-17T00:46:33.364 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-17T00:46:33.364 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-17T00:46:33.364 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-17T00:46:33.364 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-17T00:46:33.365 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-17T00:46:33.365 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-17T00:46:33.365 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-17T00:46:33.365 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-17T00:46:33.365 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-17T00:46:33.365 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-17T00:46:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-17T00:46:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-17T00:46:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-17T00:46:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-17T00:46:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-17T00:46:33.366 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-17T00:46:33.367 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-17T00:46:33.367 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-17T00:46:33.367 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-17T00:46:33.367 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-17T00:46:33.367 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-17T00:46:33.367 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-17T00:46:33.368 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-17T00:46:33.368 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-17T00:46:33.368 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-17T00:46:33.368 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-17T00:46:33.368 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-17T00:46:33.368 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-17T00:46:33.369 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-17T00:46:33.369 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-17T00:46:33.369 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-17T00:46:33.369 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-17T00:46:33.369 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-17T00:46:33.369 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-17T00:46:33.370 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-17T00:46:33.370 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-17T00:46:33.370 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-17T00:46:33.370 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-17T00:46:33.370 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-17T00:46:33.370 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-17T00:46:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-17T00:46:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-17T00:46:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-17T00:46:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-17T00:46:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-17T00:46:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-17T00:46:33.372 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-17T00:46:33.373 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-17T00:46:33.373 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-17T00:46:33.373 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-17T00:46:33.373 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-17T00:46:33.373 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-17T00:46:33.373 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-17T00:46:33.374 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-17T00:46:33.374 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-17T00:46:33.374 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-17T00:46:33.374 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-17T00:46:33.374 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-17T00:46:33.374 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-17T00:46:33.375 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-17T00:46:33.375 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-17T00:46:33.375 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-17T00:46:33.375 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-17T00:46:33.375 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-17T00:46:33.376 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-17T00:46:33.376 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-17T00:46:33.376 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-17T00:46:33.376 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-17T00:46:33.376 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-17T00:46:33.376 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-17T00:46:33.377 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-17T00:46:33.377 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-17T00:46:33.377 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-17T00:46:33.377 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-17T00:46:33.377 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-17T00:46:33.377 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-17T00:46:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-17T00:46:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-17T00:46:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-17T00:46:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-17T00:46:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-17T00:46:33.378 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-17T00:46:33.379 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-17T00:46:33.379 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-17T00:46:33.379 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-17T00:46:33.379 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-17T00:46:33.379 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-17T00:46:33.379 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-17T00:46:33.380 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-17T00:46:33.380 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-17T00:46:33.380 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-17T00:46:33.380 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-17T00:46:33.380 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-17T00:46:33.380 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-17T00:46:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-17T00:46:33.382 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-17T00:46:33.382 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-17T00:46:33.382 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-17T00:46:33.382 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-17T00:46:33.382 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-17T00:46:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-17T00:46:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-17T00:46:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-17T00:46:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-17T00:46:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-17T00:46:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-17T00:46:33.384 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-17T00:46:33.384 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-17T00:46:33.384 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-17T00:46:33.384 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-17T00:46:33.384 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-17T00:46:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-17T00:46:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-17T00:46:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-17T00:46:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-17T00:46:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-17T00:46:33.385 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-17T00:46:33.386 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-17T00:46:33.386 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-17T00:46:33.386 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-17T00:46:33.386 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-17T00:46:33.386 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-17T00:46:33.386 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-17T00:46:33.387 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-17T00:46:33.387 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-17T00:46:33.387 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-17T00:46:33.387 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-17T00:46:33.387 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-17T00:46:33.388 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-17T00:46:33.388 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-17T00:46:33.388 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-17T00:46:33.388 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-17T00:46:33.388 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-17T00:46:33.388 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-17T00:46:33.389 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-17T00:46:33.390 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-17T00:46:33.390 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-17T00:46:33.390 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-17T00:46:33.390 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-17T00:46:33.390 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-17T00:46:33.391 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-17T00:46:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-17T00:46:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-17T00:46:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-17T00:46:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-17T00:46:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-17T00:46:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-17T00:46:33.393 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-17T00:46:33.393 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-17T00:46:33.393 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-17T00:46:33.393 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-17T00:46:33.393 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-17T00:46:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-17T00:46:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-17T00:46:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-17T00:46:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-17T00:46:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-17T00:46:33.394 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-17T00:46:33.395 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-17T00:46:33.395 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-17T00:46:33.395 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-17T00:46:33.395 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-17T00:46:33.395 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-17T00:46:33.396 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-17T00:46:33.396 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-17T00:46:33.396 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-17T00:46:33.396 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-17T00:46:33.396 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-17T00:46:33.396 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-17T00:46:33.397 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-17T00:46:33.397 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-17T00:46:33.397 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-17T00:46:33.397 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-17T00:46:33.397 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-17T00:46:33.398 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-17T00:46:33.398 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-17T00:46:33.398 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-17T00:46:33.398 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-17T00:46:33.398 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-17T00:46:33.398 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-17T00:46:33.399 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-17T00:46:33.399 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-17T00:46:33.399 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-17T00:46:33.399 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-17T00:46:33.399 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-17T00:46:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-17T00:46:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-17T00:46:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-17T00:46:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-17T00:46:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-17T00:46:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-17T00:46:33.401 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-17T00:46:33.401 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-17T00:46:33.401 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-17T00:46:33.401 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-17T00:46:33.401 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-17T00:46:33.401 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-17T00:46:33.402 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-17T00:46:33.402 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-17T00:46:33.402 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-17T00:46:33.402 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-17T00:46:33.402 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-17T00:46:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-17T00:46:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-17T00:46:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-17T00:46:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-17T00:46:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-17T00:46:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-17T00:46:33.404 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-17T00:46:33.404 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-17T00:46:33.404 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-17T00:46:33.404 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-17T00:46:33.404 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-17T00:46:33.404 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-17T00:46:33.405 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-17T00:46:33.405 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-17T00:46:33.405 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-17T00:46:33.405 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-17T00:46:33.405 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-17T00:46:33.405 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-17T00:46:33.406 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-17T00:46:33.406 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-17T00:46:33.406 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-17T00:46:33.406 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-17T00:46:33.406 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-17T00:46:33.406 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-17T00:46:33.407 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-17T00:46:33.407 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-17T00:46:33.407 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-17T00:46:33.407 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-17T00:46:33.407 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-17T00:46:33.407 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-17T00:46:33.408 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-17T00:46:33.408 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-17T00:46:33.408 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-17T00:46:33.408 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-17T00:46:33.408 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-17T00:46:33.408 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-17T00:46:33.409 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-17T00:46:33.409 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-17T00:46:33.409 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-17T00:46:33.409 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-17T00:46:33.409 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-17T00:46:33.409 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-17T00:46:33.410 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-17T00:46:33.410 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-17T00:46:33.410 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-17T00:46:33.410 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-17T00:46:33.410 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-17T00:46:33.410 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-17T00:46:33.411 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-17T00:46:33.411 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-17T00:46:33.411 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-17T00:46:33.411 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-17T00:46:33.411 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-17T00:46:33.411 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-17T00:46:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-17T00:46:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-17T00:46:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-17T00:46:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-17T00:46:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-17T00:46:33.412 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-17T00:46:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-17T00:46:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-17T00:46:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-17T00:46:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-17T00:46:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-17T00:46:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-17T00:46:33.414 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-17T00:46:33.414 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-17T00:46:33.414 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-17T00:46:33.414 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-17T00:46:33.414 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-17T00:46:33.414 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-17T00:46:33.415 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-17T00:46:33.415 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-17T00:46:33.415 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-17T00:46:33.415 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-17T00:46:33.415 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-17T00:46:33.416 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-17T00:46:33.416 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-17T00:46:33.416 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-17T00:46:33.416 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-17T00:46:33.416 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-17T00:46:33.417 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-17T00:46:33.417 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-17T00:46:33.417 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-17T00:46:33.417 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-17T00:46:33.417 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-17T00:46:33.417 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-17T00:46:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-17T00:46:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-17T00:46:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-17T00:46:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-17T00:46:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-17T00:46:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-17T00:46:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-17T00:46:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-17T00:46:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-17T00:46:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-17T00:46:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-17T00:46:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-17T00:46:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-17T00:46:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-17T00:46:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-17T00:46:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-17T00:46:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-17T00:46:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-17T00:46:33.421 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-17T00:46:33.421 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-17T00:46:33.421 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-17T00:46:33.421 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-17T00:46:33.421 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-17T00:46:33.422 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-17T00:46:33.422 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-17T00:46:33.422 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-17T00:46:33.422 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-17T00:46:33.422 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-17T00:46:33.422 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-17T00:46:33.423 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-17T00:46:33.423 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-17T00:46:33.423 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-17T00:46:33.423 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-17T00:46:33.423 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-17T00:46:33.424 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-17T00:46:33.424 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-17T00:46:33.424 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-17T00:46:33.424 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-17T00:46:33.424 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-17T00:46:33.424 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-17T00:46:33.425 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-17T00:46:33.425 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-17T00:46:33.425 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-17T00:46:33.425 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-17T00:46:33.425 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-17T00:46:33.425 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-17T00:46:33.426 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-17T00:46:33.426 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-17T00:46:33.426 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-17T00:46:33.426 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-17T00:46:33.426 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-17T00:46:33.427 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-17T00:46:33.427 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-17T00:46:33.427 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-17T00:46:33.427 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-17T00:46:33.427 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-17T00:46:33.427 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-17T00:46:33.428 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-17T00:46:33.428 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-17T00:46:33.428 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-17T00:46:33.428 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-17T00:46:33.428 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-17T00:46:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-17T00:46:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-17T00:46:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-17T00:46:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-17T00:46:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-17T00:46:33.429 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-17T00:46:33.430 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-17T00:46:33.430 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-17T00:46:33.430 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-17T00:46:33.430 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-17T00:46:33.431 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-17T00:46:33.432 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-17T00:46:33.432 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-17T00:46:33.432 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-17T00:46:33.432 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-17T00:46:33.432 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-17T00:46:33.432 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-17T00:46:33.433 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-17T00:46:33.433 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-17T00:46:33.433 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-17T00:46:33.433 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-17T00:46:33.433 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-17T00:46:33.433 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-17T00:46:33.434 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-17T00:46:33.434 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-17T00:46:33.434 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-17T00:46:33.434 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-17T00:46:33.434 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-17T00:46:33.435 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-17T00:46:33.435 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-17T00:46:33.435 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-17T00:46:33.435 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-17T00:46:33.435 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-17T00:46:33.435 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-17T00:46:33.436 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-17T00:46:33.436 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-17T00:46:33.436 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-17T00:46:33.436 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-17T00:46:33.436 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-17T00:46:33.436 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-17T00:46:33.437 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-17T00:46:33.437 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-17T00:46:33.437 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-17T00:46:33.437 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-17T00:46:33.437 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-17T00:46:33.437 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-17T00:46:33.438 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-17T00:46:33.438 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-17T00:46:33.438 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-17T00:46:33.438 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-17T00:46:33.438 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-17T00:46:33.439 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-17T00:46:33.439 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-17T00:46:33.439 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-17T00:46:33.439 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-17T00:46:33.439 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-17T00:46:33.439 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-17T00:46:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-17T00:46:33.441 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-17T00:46:33.441 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-17T00:46:33.441 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-17T00:46:33.441 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-17T00:46:33.441 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-17T00:46:33.441 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-17T00:46:33.442 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-17T00:46:33.442 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-17T00:46:33.442 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-17T00:46:33.442 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-17T00:46:33.442 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-17T00:46:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-17T00:46:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-17T00:46:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-17T00:46:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-17T00:46:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-17T00:46:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-17T00:46:33.444 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-17T00:46:33.444 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-17T00:46:33.444 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-17T00:46:33.444 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-17T00:46:33.444 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-17T00:46:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-17T00:46:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-17T00:46:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-17T00:46:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-17T00:46:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-17T00:46:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-17T00:46:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-17T00:46:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-17T00:46:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-17T00:46:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-17T00:46:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-17T00:46:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-17T00:46:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-17T00:46:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-17T00:46:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-17T00:46:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-17T00:46:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-17T00:46:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-17T00:46:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-17T00:46:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-17T00:46:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-17T00:46:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-17T00:46:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-17T00:46:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-17T00:46:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-17T00:46:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-17T00:46:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-17T00:46:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-17T00:46:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-17T00:46:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-17T00:46:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-17T00:46:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-17T00:46:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-17T00:46:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-17T00:46:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-17T00:46:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-17T00:46:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-17T00:46:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-17T00:46:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-17T00:46:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-17T00:46:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-17T00:46:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-17T00:46:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-17T00:46:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-17T00:46:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-17T00:46:33.453 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-17T00:46:33.453 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-17T00:46:33.453 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-17T00:46:33.453 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-17T00:46:33.453 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-17T00:46:33.453 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-17T00:46:33.454 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-17T00:46:33.454 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-17T00:46:33.454 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-17T00:46:33.454 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-17T00:46:33.454 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-17T00:46:33.455 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-17T00:46:33.455 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-17T00:46:33.455 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-17T00:46:33.455 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-17T00:46:33.455 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-17T00:46:33.456 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-17T00:46:33.456 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-17T00:46:33.456 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-17T00:46:33.456 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-17T00:46:33.456 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-17T00:46:33.456 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-17T00:46:33.457 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-17T00:46:33.457 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-17T00:46:33.457 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-17T00:46:33.457 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-17T00:46:33.457 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-17T00:46:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-17T00:46:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-17T00:46:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-17T00:46:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-17T00:46:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-17T00:46:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-17T00:46:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-17T00:46:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-17T00:46:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-17T00:46:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-17T00:46:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-17T00:46:33.460 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-17T00:46:33.460 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-17T00:46:33.460 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-17T00:46:33.460 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-17T00:46:33.461 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.461 DEBUG:teuthology.orchestra.run.smithi045:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-17T00:46:33.487 INFO:teuthology.orchestra.run.smithi045.stdout:postinstall scriptlet (using /bin/sh): 2023-10-17T00:46:33.488 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-17T00:46:33.488 INFO:teuthology.orchestra.run.smithi045.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-17T00:46:33.488 INFO:teuthology.orchestra.run.smithi045.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-17T00:46:33.488 INFO:teuthology.orchestra.run.smithi045.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.488 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-17T00:46:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-17T00:46:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2023-10-17T00:46:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-17T00:46:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-17T00:46:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-17T00:46:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-17T00:46:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2023-10-17T00:46:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-17T00:46:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-17T00:46:33.490 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.491 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2023-10-17T00:46:33.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2023-10-17T00:46:33.567 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2023-10-17T00:46:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2023-10-17T00:46:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.02 2023-10-17T00:46:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 123.el8 2023-10-17T00:46:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2023-10-17T00:46:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-17T00:46:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:Group : System Environment/Base 2023-10-17T00:46:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 9265450 2023-10-17T00:46:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2023-10-17T00:46:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-17T00:46:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-17T00:46:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-17T00:46:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-17T00:46:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:Relocations : (not relocatable) 2023-10-17T00:46:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : Red Hat, Inc. 2023-10-17T00:46:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : Red Hat, Inc. 2023-10-17T00:46:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-17T00:46:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2023-10-17T00:46:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2023-10-17T00:46:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T00:46:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-17T00:46:33.572 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-17T00:46:33.572 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-17T00:46:33.572 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2023-10-17T00:46:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T00:46:33.572 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2023-10-17T00:46:33.573 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-17T00:46:33.573 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 6.6.0-rc5-g47672119149b 2023-10-17T00:46:33.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-17T00:46:38.132 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2023-10-17T00:46:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:done 2023-10-17T00:46:40.165 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2023-10-17T00:46:40.190 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.B4QSrmC8K8 2023-10-17T00:46:40.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.B4QSrmC8K8 2023-10-17T00:46:40.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.B4QSrmC8K8 2023-10-17T00:46:40.419 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.B4QSrmC8K8 is 6KB 2023-10-17T00:46:40.467 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.B4QSrmC8K8 2023-10-17T00:46:40.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2023-10-17T00:46:40.549 INFO:teuthology.orchestra.run.smithi045.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-17T00:46:40.549 INFO:teuthology.orchestra.run.smithi045.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-17T00:46:40.550 INFO:teuthology.orchestra.run.smithi045.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b.conf 2023-10-17T00:46:40.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b 2023-10-17T00:46:40.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2023-10-17T00:47:10.694 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-17T00:47:10.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-17T00:47:10.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T00:47:29.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-10-17T00:47:38.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-17T00:47:38.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T00:48:38.151 DEBUG:teuthology.orchestra.remote:timed out 2023-10-17T00:48:50.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-17T00:48:50.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T00:48:50.515 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-10-17T00:48:50.944 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-17T00:48:50.944 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-17T00:48:50.944 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-10-17T00:48:50.988 INFO:teuthology.orchestra.run.smithi045.stdout:6.6.0-rc5-g47672119149b 2023-10-17T00:48:50.988 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-17T00:48:50.988 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-17T00:48:50.989 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-17T00:48:51.990 DEBUG:teuthology.parallel:result is None 2023-10-17T00:48:51.990 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-17T00:48:51.997 INFO:teuthology.task.internal:Creating test directory... 2023-10-17T00:48:51.998 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-17T00:48:52.003 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-17T00:48:52.006 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-17T00:48:52.046 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-17T00:48:52.054 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-17T00:48:52.064 INFO:teuthology.task.internal:Creating archive directory... 2023-10-17T00:48:52.065 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-17T00:48:52.070 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-17T00:48:52.073 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-17T00:48:52.130 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-17T00:48:52.139 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-17T00:48:52.140 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-17T00:48:52.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T00:48:52.170 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-17T00:48:52.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T00:48:52.189 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-17T00:48:52.194 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-17T00:48:52.213 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-17T00:48:52.248 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-17T00:48:52.269 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-17T00:48:52.284 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-17T00:48:52.290 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-17T00:48:52.330 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-17T00:48:52.369 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-17T00:48:52.373 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-17T00:48:52.384 INFO:teuthology.task.internal:Configuring sudo... 2023-10-17T00:48:52.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-17T00:48:52.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-17T00:48:52.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-17T00:48:52.456 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-17T00:48:52.466 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-17T00:48:52.466 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-17T00:48:52.470 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-17T00:48:52.473 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-17T00:48:52.509 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-17T00:48:52.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-17T00:48:52.687 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-17T00:48:52.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-17T00:48:52.829 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T00:48:52.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-17T00:48:52.899 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-17T00:48:52.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-17T00:48:53.002 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-17T00:48:53.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-17T00:48:53.138 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T00:48:53.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-17T00:48:53.211 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-17T00:48:53.258 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-17T00:48:53.315 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-17T00:48:53.399 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-17T00:48:53.446 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T00:48:53.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-17T00:48:53.515 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2023-10-17T00:48:53.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2023-10-17T00:48:53.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-10-17T00:48:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-17T00:48:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-17T00:48:53.639 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-17T00:48:54.088 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-17T00:48:54.097 INFO:teuthology.task.internal:Starting timer... 2023-10-17T00:48:54.097 INFO:teuthology.run_tasks:Running task pcp... 2023-10-17T00:48:54.118 INFO:teuthology.run_tasks:Running task selinux... 2023-10-17T00:48:54.127 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-17T00:48:54.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2023-10-17T00:48:54.249 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2023-10-17T00:48:54.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2023-10-17T00:48:54.313 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2023-10-17T00:48:54.316 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-10-17T00:48:54.376 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-10-17T00:48:54.377 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-17T00:48:54.378 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2023-10-17T00:48:54.436 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2023-10-17T00:48:54.437 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2023-10-17T00:48:54.467 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2023-10-17T00:48:54.467 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-10-17T00:48:54.486 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-10-17T00:48:54.486 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-10-17T00:48:54.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-17T00:48:54.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T00:48:54.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-17T00:48:54.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T00:48:54.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-17T00:48:54.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T00:48:54.619 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-17T00:48:54.619 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2023-10-17T00:48:54.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2023-10-17T00:48:54.718 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-10-17T00:48:54.745 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-17T00:48:54.755 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-17T00:48:54.761 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-17T00:48:54.877 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-17T00:48:54.893 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-17T00:48:54.894 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi060.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-17T01:01:27.096 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-10-17T01:01:27.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-17T01:01:27.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T01:01:27.189 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-10-17T01:01:27.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-17T01:01:27.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-10-17T01:01:27.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T01:01:27.380 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-10-17T01:01:27.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-10-17T01:01:27.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-17T01:01:27.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-17T01:01:27.521 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-10-17T01:01:27.588 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-17T01:01:27.589 INFO:teuthology.run_tasks:Running task clock... 2023-10-17T01:01:27.600 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-17T01:01:27.600 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-17T01:01:27.601 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-17T01:01:27.606 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-17T01:01:27.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-17T01:01:27.609 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-17T01:01:27.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-17T01:01:27.644 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-17T01:01:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-17T01:01:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-17T01:01:27.678 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-17T01:01:27.679 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-17T01:01:27.716 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-17T01:01:27.729 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2023-10-17T01:01:27.742 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2023-10-17T01:01:27.755 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-10-17T01:01:27.758 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-17T01:01:27.767 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-10-17T01:01:27.773 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-17T01:01:27.781 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-17T01:01:27.795 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-17T01:01:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2023-10-17T01:01:27.840 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2023-10-17T01:01:27.842 INFO:teuthology.orchestra.run.smithi060.stderr:bash: ntpq: command not found 2023-10-17T01:01:27.845 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-17T01:01:27.845 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-10-17T01:01:27.845 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.846 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.846 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.846 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.870 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-10-17T01:01:27.873 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-17T01:01:27.873 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-17T01:01:27.873 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:27.875 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-17T01:01:27.911 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-17T01:01:28.018 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-10-17T01:01:28.025 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-17T01:01:28.025 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-10-17T01:01:28.025 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:28.026 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:28.026 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:28.026 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:01:28.027 INFO:teuthology.run_tasks:Running task pexec... 2023-10-17T01:01:28.037 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-17T01:01:28.037 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:01:28.038 DEBUG:teuthology.orchestra.run.smithi045:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-17T01:01:28.038 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi060.front.sepia.ceph.com 2023-10-17T01:01:28.038 DEBUG:teuthology.orchestra.run.smithi060:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-17T01:01:28.039 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi181.front.sepia.ceph.com 2023-10-17T01:01:28.039 DEBUG:teuthology.orchestra.run.smithi181:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-17T01:01:28.411 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:28.432 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:28.557 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:29.234 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:12 ago on Tue 17 Oct 2023 12:56:16 AM UTC. 2023-10-17T01:01:29.292 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:04:46 ago on Tue 17 Oct 2023 12:56:42 AM UTC. 2023-10-17T01:01:29.567 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Oct 2023 12:59:40 AM UTC. 2023-10-17T01:01:31.108 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-17T01:01:31.109 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-10-17T01:01:31.109 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-10-17T01:01:31.110 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-10-17T01:01:31.110 INFO:teuthology.orchestra.run.smithi181.stdout:Resetting modules: 2023-10-17T01:01:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools 2023-10-17T01:01:31.110 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:31.110 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-17T01:01:31.111 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-10-17T01:01:31.111 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:31.250 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-10-17T01:01:31.252 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-10-17T01:01:31.252 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2023-10-17T01:01:31.252 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-10-17T01:01:31.252 INFO:teuthology.orchestra.run.smithi060.stdout:Resetting modules: 2023-10-17T01:01:31.253 INFO:teuthology.orchestra.run.smithi060.stdout: container-tools 2023-10-17T01:01:31.253 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:31.253 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-10-17T01:01:31.253 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-10-17T01:01:31.254 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:31.641 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-10-17T01:01:31.642 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-10-17T01:01:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-10-17T01:01:31.643 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-10-17T01:01:31.643 INFO:teuthology.orchestra.run.smithi045.stdout:Resetting modules: 2023-10-17T01:01:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools 2023-10-17T01:01:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:31.643 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-10-17T01:01:31.644 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-10-17T01:01:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:32.081 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:15 ago on Tue 17 Oct 2023 12:56:16 AM UTC. 2023-10-17T01:01:32.249 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:04:49 ago on Tue 17 Oct 2023 12:56:42 AM UTC. 2023-10-17T01:01:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Oct 2023 12:59:40 AM UTC. 2023-10-17T01:01:34.876 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-17T01:01:35.344 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:35.353 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-10-17T01:01:35.842 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:36.180 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-10-17T01:01:36.181 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:19 ago on Tue 17 Oct 2023 12:56:16 AM UTC. 2023-10-17T01:01:36.705 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:04:54 ago on Tue 17 Oct 2023 12:56:42 AM UTC. 2023-10-17T01:01:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:37.857 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:57 ago on Tue 17 Oct 2023 12:59:40 AM UTC. 2023-10-17T01:01:38.032 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-17T01:01:38.127 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-10-17T01:01:38.127 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-10-17T01:01:38.128 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-10-17T01:01:38.128 INFO:teuthology.orchestra.run.smithi181.stdout:Installing group/module packages: 2023-10-17T01:01:38.128 INFO:teuthology.orchestra.run.smithi181.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-17T01:01:38.128 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-17T01:01:38.128 INFO:teuthology.orchestra.run.smithi181.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-17T01:01:38.129 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-17T01:01:38.129 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-17T01:01:38.129 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-17T01:01:38.129 INFO:teuthology.orchestra.run.smithi181.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-17T01:01:38.130 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-10-17T01:01:38.130 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-17T01:01:38.130 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-17T01:01:38.130 INFO:teuthology.orchestra.run.smithi181.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-17T01:01:38.130 INFO:teuthology.orchestra.run.smithi181.stdout:Installing module profiles: 2023-10-17T01:01:38.131 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools/common 2023-10-17T01:01:38.131 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:38.131 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-10-17T01:01:38.131 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools rhel8 2023-10-17T01:01:38.131 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:38.132 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-17T01:01:38.132 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-10-17T01:01:38.132 INFO:teuthology.orchestra.run.smithi181.stdout:Install 10 Packages 2023-10-17T01:01:38.132 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:38.133 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 18 M 2023-10-17T01:01:38.133 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 65 M 2023-10-17T01:01:38.133 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-10-17T01:01:38.230 INFO:teuthology.orchestra.run.smithi181.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 938 kB/s | 94 kB 00:00 2023-10-17T01:01:38.247 INFO:teuthology.orchestra.run.smithi181.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 212 kB/s | 25 kB 00:00 2023-10-17T01:01:38.263 INFO:teuthology.orchestra.run.smithi181.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 305 kB/s | 41 kB 00:00 2023-10-17T01:01:38.305 INFO:teuthology.orchestra.run.smithi181.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-10-17T01:01:38.339 INFO:teuthology.orchestra.run.smithi181.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-10-17T01:01:38.364 INFO:teuthology.orchestra.run.smithi181.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-17T01:01:38.406 INFO:teuthology.orchestra.run.smithi181.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.1 MB/s | 213 kB 00:00 2023-10-17T01:01:38.515 INFO:teuthology.orchestra.run.smithi181.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-10-17T01:01:38.615 INFO:teuthology.orchestra.run.smithi181.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-10-17T01:01:38.676 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-10-17T01:01:38.707 INFO:teuthology.orchestra.run.smithi181.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-10-17T01:01:38.708 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:01:38.708 INFO:teuthology.orchestra.run.smithi181.stdout:Total 32 MB/s | 18 MB 00:00 2023-10-17T01:01:38.778 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================================ 2023-10-17T01:01:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-10-17T01:01:38.779 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================================ 2023-10-17T01:01:38.779 INFO:teuthology.orchestra.run.smithi060.stdout:Installing group/module packages: 2023-10-17T01:01:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-17T01:01:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-17T01:01:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-17T01:01:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-17T01:01:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-17T01:01:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-17T01:01:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-17T01:01:38.781 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-10-17T01:01:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-17T01:01:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-17T01:01:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-17T01:01:38.782 INFO:teuthology.orchestra.run.smithi060.stdout:Installing module profiles: 2023-10-17T01:01:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: container-tools/common 2023-10-17T01:01:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:38.782 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling module streams: 2023-10-17T01:01:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: container-tools rhel8 2023-10-17T01:01:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:38.783 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-10-17T01:01:38.783 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================================ 2023-10-17T01:01:38.783 INFO:teuthology.orchestra.run.smithi060.stdout:Install 10 Packages 2023-10-17T01:01:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:38.784 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 18 M 2023-10-17T01:01:38.784 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 65 M 2023-10-17T01:01:38.785 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-10-17T01:01:38.869 INFO:teuthology.orchestra.run.smithi060.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.1 MB/s | 94 kB 00:00 2023-10-17T01:01:38.894 INFO:teuthology.orchestra.run.smithi060.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 220 kB/s | 25 kB 00:00 2023-10-17T01:01:38.909 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-10-17T01:01:38.919 INFO:teuthology.orchestra.run.smithi060.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 296 kB/s | 41 kB 00:00 2023-10-17T01:01:38.927 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-10-17T01:01:38.927 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-10-17T01:01:38.944 INFO:teuthology.orchestra.run.smithi060.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-10-17T01:01:39.003 INFO:teuthology.orchestra.run.smithi060.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-10-17T01:01:39.028 INFO:teuthology.orchestra.run.smithi060.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.7 MB/s | 149 kB 00:00 2023-10-17T01:01:39.054 INFO:teuthology.orchestra.run.smithi060.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-17T01:01:39.110 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-10-17T01:01:39.114 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-10-17T01:01:39.154 INFO:teuthology.orchestra.run.smithi060.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-10-17T01:01:39.279 INFO:teuthology.orchestra.run.smithi060.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-10-17T01:01:39.379 INFO:teuthology.orchestra.run.smithi060.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 17 MB/s | 8.0 MB 00:00 2023-10-17T01:01:39.380 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:01:39.380 INFO:teuthology.orchestra.run.smithi060.stdout:Total 30 MB/s | 18 MB 00:00 2023-10-17T01:01:39.586 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-10-17T01:01:39.607 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-10-17T01:01:39.607 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-10-17T01:01:39.641 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-10-17T01:01:39.803 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-10-17T01:01:39.807 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-10-17T01:01:39.835 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-17T01:01:39.962 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-10-17T01:01:40.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-17T01:01:40.068 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================================ 2023-10-17T01:01:40.068 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-10-17T01:01:40.069 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================================ 2023-10-17T01:01:40.069 INFO:teuthology.orchestra.run.smithi045.stdout:Installing group/module packages: 2023-10-17T01:01:40.069 INFO:teuthology.orchestra.run.smithi045.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-17T01:01:40.069 INFO:teuthology.orchestra.run.smithi045.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-17T01:01:40.069 INFO:teuthology.orchestra.run.smithi045.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-17T01:01:40.069 INFO:teuthology.orchestra.run.smithi045.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-17T01:01:40.070 INFO:teuthology.orchestra.run.smithi045.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-17T01:01:40.070 INFO:teuthology.orchestra.run.smithi045.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-17T01:01:40.070 INFO:teuthology.orchestra.run.smithi045.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-17T01:01:40.070 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-10-17T01:01:40.070 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-17T01:01:40.071 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-17T01:01:40.071 INFO:teuthology.orchestra.run.smithi045.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-17T01:01:40.071 INFO:teuthology.orchestra.run.smithi045.stdout:Installing module profiles: 2023-10-17T01:01:40.071 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools/common 2023-10-17T01:01:40.072 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:40.072 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling module streams: 2023-10-17T01:01:40.072 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools rhel8 2023-10-17T01:01:40.072 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:40.072 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-10-17T01:01:40.073 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================================ 2023-10-17T01:01:40.073 INFO:teuthology.orchestra.run.smithi045.stdout:Install 10 Packages 2023-10-17T01:01:40.073 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:40.074 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 18 M 2023-10-17T01:01:40.074 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 65 M 2023-10-17T01:01:40.074 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-10-17T01:01:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 490 kB/s | 41 kB 00:00 2023-10-17T01:01:40.180 INFO:teuthology.orchestra.run.smithi045.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 229 kB/s | 25 kB 00:00 2023-10-17T01:01:40.202 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-17T01:01:40.205 INFO:teuthology.orchestra.run.smithi045.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 984 kB/s | 49 kB 00:00 2023-10-17T01:01:40.230 INFO:teuthology.orchestra.run.smithi045.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 590 kB/s | 94 kB 00:00 2023-10-17T01:01:40.264 INFO:teuthology.orchestra.run.smithi045.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-17T01:01:40.290 INFO:teuthology.orchestra.run.smithi045.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-17T01:01:40.313 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-10-17T01:01:40.323 INFO:teuthology.orchestra.run.smithi045.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-10-17T01:01:40.413 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-17T01:01:40.432 INFO:teuthology.orchestra.run.smithi045.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-10-17T01:01:40.554 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-17T01:01:40.599 INFO:teuthology.orchestra.run.smithi045.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 23 MB/s | 7.0 MB 00:00 2023-10-17T01:01:40.699 INFO:teuthology.orchestra.run.smithi045.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-10-17T01:01:40.699 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:01:40.700 INFO:teuthology.orchestra.run.smithi045.stdout:Total 29 MB/s | 18 MB 00:00 2023-10-17T01:01:40.716 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-17T01:01:40.761 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-17T01:01:40.905 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-17T01:01:41.048 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-10-17T01:01:41.072 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-10-17T01:01:41.073 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-10-17T01:01:41.109 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-17T01:01:41.472 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-17T01:01:41.530 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-17T01:01:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-10-17T01:01:41.566 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-10-17T01:01:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-10-17T01:01:42.246 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-17T01:01:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-17T01:01:42.398 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-17T01:01:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-17T01:01:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-17T01:01:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-17T01:01:42.789 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-17T01:01:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-17T01:01:43.009 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-17T01:01:43.177 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-17T01:01:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-17T01:01:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-17T01:01:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-17T01:01:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-17T01:01:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-17T01:01:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-17T01:01:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-17T01:01:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-17T01:01:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-17T01:01:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-17T01:01:43.384 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-17T01:01:43.413 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-17T01:01:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-17T01:01:43.714 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-17T01:01:43.994 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-17T01:01:43.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-17T01:01:43.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-17T01:01:43.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-17T01:01:43.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-17T01:01:43.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-17T01:01:43.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-17T01:01:43.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-17T01:01:43.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-17T01:01:43.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-17T01:01:44.230 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-17T01:01:44.462 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-17T01:01:44.462 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:05:27 ago on Tue 17 Oct 2023 12:56:16 AM UTC. 2023-10-17T01:01:45.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-17T01:01:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:05:02 ago on Tue 17 Oct 2023 12:56:42 AM UTC. 2023-10-17T01:01:45.202 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-17T01:01:45.498 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-17T01:01:45.677 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-17T01:01:45.909 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-17T01:01:46.330 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-17T01:01:46.331 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-17T01:01:46.331 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-17T01:01:46.331 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-17T01:01:46.331 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-17T01:01:46.332 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-17T01:01:46.332 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-17T01:01:46.332 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-17T01:01:46.332 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-17T01:01:46.332 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-17T01:01:47.215 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-10-17T01:01:47.417 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:47.417 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-10-17T01:01:47.417 INFO:teuthology.orchestra.run.smithi181.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:47.418 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-17T01:01:47.418 INFO:teuthology.orchestra.run.smithi181.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:47.418 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-17T01:01:47.418 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-17T01:01:47.418 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-17T01:01:47.419 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:47.419 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:47.419 INFO:teuthology.orchestra.run.smithi181.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-17T01:01:47.419 INFO:teuthology.orchestra.run.smithi181.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-17T01:01:47.419 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:01:47.420 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-17T01:01:47.616 DEBUG:teuthology.parallel:result is None 2023-10-17T01:01:47.863 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-17T01:01:47.864 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:07 ago on Tue 17 Oct 2023 12:59:40 AM UTC. 2023-10-17T01:01:47.890 INFO:teuthology.orchestra.run.smithi060.stdout:Installed products updated. 2023-10-17T01:01:48.122 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:48.123 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-10-17T01:01:48.123 INFO:teuthology.orchestra.run.smithi060.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:48.123 INFO:teuthology.orchestra.run.smithi060.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-17T01:01:48.123 INFO:teuthology.orchestra.run.smithi060.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:48.124 INFO:teuthology.orchestra.run.smithi060.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-17T01:01:48.124 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-17T01:01:48.124 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-17T01:01:48.124 INFO:teuthology.orchestra.run.smithi060.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:48.124 INFO:teuthology.orchestra.run.smithi060.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:48.125 INFO:teuthology.orchestra.run.smithi060.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-17T01:01:48.125 INFO:teuthology.orchestra.run.smithi060.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-17T01:01:48.125 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:01:48.125 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-10-17T01:01:48.291 DEBUG:teuthology.parallel:result is None 2023-10-17T01:01:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:Installed products updated. 2023-10-17T01:01:51.117 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:51.117 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-10-17T01:01:51.117 INFO:teuthology.orchestra.run.smithi045.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:51.117 INFO:teuthology.orchestra.run.smithi045.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-17T01:01:51.118 INFO:teuthology.orchestra.run.smithi045.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:51.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-17T01:01:51.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-17T01:01:51.118 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-17T01:01:51.119 INFO:teuthology.orchestra.run.smithi045.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:51.119 INFO:teuthology.orchestra.run.smithi045.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-17T01:01:51.119 INFO:teuthology.orchestra.run.smithi045.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-17T01:01:51.119 INFO:teuthology.orchestra.run.smithi045.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-17T01:01:51.119 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:01:51.120 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-10-17T01:01:51.334 DEBUG:teuthology.parallel:result is None 2023-10-17T01:01:51.335 INFO:teuthology.run_tasks:Running task install... 2023-10-17T01:01:51.344 DEBUG:teuthology.task.install:project ceph 2023-10-17T01:01:51.344 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe'}} 2023-10-17T01:01:51.345 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe'} 2023-10-17T01:01:51.345 INFO:teuthology.task.install:Using flavor: default 2023-10-17T01:01:51.351 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-17T01:01:51.351 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-17T01:01:51.352 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe', 'tag': None, 'wait_for_package': False} 2023-10-17T01:01:51.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:01:51.355 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe', 'tag': None, 'wait_for_package': False} 2023-10-17T01:01:51.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:01:51.356 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe', 'tag': None, 'wait_for_package': False} 2023-10-17T01:01:51.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:01:51.587 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/ 2023-10-17T01:01:51.587 INFO:teuthology.task.install.rpm:Package version is 18.2.0-668.gab9fe435 2023-10-17T01:01:51.607 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/ 2023-10-17T01:01:51.608 INFO:teuthology.task.install.rpm:Package version is 18.2.0-668.gab9fe435 2023-10-17T01:01:51.626 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/ 2023-10-17T01:01:51.626 INFO:teuthology.task.install.rpm:Package version is 18.2.0-668.gab9fe435 2023-10-17T01:01:51.732 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-17T01:01:51.732 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:01:51.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-17T01:01:51.735 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-17T01:01:51.735 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:01:51.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-17T01:01:51.761 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-17T01:01:51.762 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:01:51.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-17T01:01:51.763 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-17T01:01:51.764 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ab9fe43522bc41de59746200b95f65d4fbcccffe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-17T01:01:51.780 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-17T01:01:51.781 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ab9fe43522bc41de59746200b95f65d4fbcccffe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-17T01:01:51.832 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-17T01:01:51.835 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-17T01:01:51.836 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ab9fe43522bc41de59746200b95f65d4fbcccffe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-17T01:01:51.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-17T01:01:51.910 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-17T01:01:51.931 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-17T01:01:51.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-17T01:01:51.977 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-10-17T01:01:51.978 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-10-17T01:01:51.999 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2023-10-17T01:01:52.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2023-10-17T01:01:52.045 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-17T01:01:52.114 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2023-10-17T01:01:52.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2023-10-17T01:01:52.406 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:52.445 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:52.598 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:53.120 INFO:teuthology.orchestra.run.smithi181.stdout:44 files removed 2023-10-17T01:01:53.139 INFO:teuthology.orchestra.run.smithi060.stdout:44 files removed 2023-10-17T01:01:53.172 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-17T01:01:53.183 INFO:teuthology.orchestra.run.smithi045.stdout:44 files removed 2023-10-17T01:01:53.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-17T01:01:53.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-17T01:01:53.522 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:53.596 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:53.720 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-10-17T01:01:54.342 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 206 kB/s | 80 kB 00:00 2023-10-17T01:01:54.490 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 198 kB/s | 80 kB 00:00 2023-10-17T01:01:54.633 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 209 kB/s | 80 kB 00:00 2023-10-17T01:01:54.744 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-10-17T01:01:55.033 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-10-17T01:01:55.184 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-10-17T01:01:55.253 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2023-10-17T01:01:55.418 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-10-17T01:01:55.451 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 6.2 kB/s | 1.5 kB 00:00 2023-10-17T01:01:55.538 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-10-17T01:01:55.653 INFO:teuthology.orchestra.run.smithi060.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-10-17T01:01:55.694 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-10-17T01:02:03.888 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 16 MB 00:08 2023-10-17T01:02:06.286 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 16 MB 00:10 2023-10-17T01:02:08.219 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 361 kB/s | 24 kB 00:00 2023-10-17T01:02:08.321 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:12 2023-10-17T01:02:09.387 INFO:teuthology.orchestra.run.smithi045.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-10-17T01:02:09.794 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 506 kB/s | 24 kB 00:00 2023-10-17T01:02:10.671 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-10-17T01:02:12.044 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 428 kB/s | 24 kB 00:00 2023-10-17T01:02:13.046 INFO:teuthology.orchestra.run.smithi060.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-10-17T01:02:18.531 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-10-17T01:02:18.546 INFO:teuthology.orchestra.run.smithi045.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-10-17T01:02:21.839 INFO:teuthology.orchestra.run.smithi060.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-10-17T01:02:29.838 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-17T01:02:29.839 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-17T01:02:29.854 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-17T01:02:29.855 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-17T01:02:29.860 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-17T01:02:29.861 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-17T01:02:29.861 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-17T01:02:29.863 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-17T01:02:29.864 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-17T01:02:29.996 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-17T01:02:30.008 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-10-17T01:02:30.008 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-10-17T01:02:30.009 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-10-17T01:02:30.009 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-10-17T01:02:30.009 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-17T01:02:30.009 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-17T01:02:30.009 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.2.0-668.gab9fe435.el8 ceph 6.4 k 2023-10-17T01:02:30.010 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.2.0-668.gab9fe435.el8 ceph 5.4 M 2023-10-17T01:02:30.010 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 847 k 2023-10-17T01:02:30.010 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-668.gab9fe435.el8 ceph 165 k 2023-10-17T01:02:30.010 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.2.0-668.gab9fe435.el8 ceph 1.5 M 2023-10-17T01:02:30.011 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 132 k 2023-10-17T01:02:30.011 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 1.8 M 2023-10-17T01:02:30.011 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 7.4 M 2023-10-17T01:02:30.011 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 47 k 2023-10-17T01:02:30.012 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.2.0-668.gab9fe435.el8 ceph 13 M 2023-10-17T01:02:30.012 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.2.0-668.gab9fe435.el8 ceph 47 M 2023-10-17T01:02:30.012 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 261 k 2023-10-17T01:02:30.012 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 167 k 2023-10-17T01:02:30.012 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.2.0-668.gab9fe435.el8 ceph 257 k 2023-10-17T01:02:30.013 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 18 k 2023-10-17T01:02:30.013 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-17T01:02:30.013 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-17T01:02:30.013 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-17T01:02:30.014 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.2.0-668.gab9fe435.el8 ceph 30 k 2023-10-17T01:02:30.014 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 770 k 2023-10-17T01:02:30.014 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.2.0-668.gab9fe435.el8 ceph 139 k 2023-10-17T01:02:30.014 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-17T01:02:30.015 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-17T01:02:30.015 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-17T01:02:30.015 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-17T01:02:30.015 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-17T01:02:30.015 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-17T01:02:30.016 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.2.0-668.gab9fe435.el8 ceph 190 k 2023-10-17T01:02:30.016 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.2.0-668.gab9fe435.el8 ceph 370 k 2023-10-17T01:02:30.016 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 364 k 2023-10-17T01:02:30.016 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.2.0-668.gab9fe435.el8 ceph 107 k 2023-10-17T01:02:30.016 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 94 k 2023-10-17T01:02:30.017 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.4 M 2023-10-17T01:02:30.017 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 179 k 2023-10-17T01:02:30.017 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-17T01:02:30.017 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-10-17T01:02:30.017 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-17T01:02:30.018 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.7 M 2023-10-17T01:02:30.018 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.6 M 2023-10-17T01:02:30.018 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-10-17T01:02:30.018 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-17T01:02:30.019 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.2.0-668.gab9fe435.el8 ceph 22 M 2023-10-17T01:02:30.019 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 23 k 2023-10-17T01:02:30.019 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.2.0-668.gab9fe435.el8 ceph 2.3 M 2023-10-17T01:02:30.019 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 242 k 2023-10-17T01:02:30.019 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.2.0-668.gab9fe435.el8 ceph 4.6 M 2023-10-17T01:02:30.020 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 18 M 2023-10-17T01:02:30.020 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 15 k 2023-10-17T01:02:30.020 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.2.0-668.gab9fe435.el8 ceph 24 k 2023-10-17T01:02:30.020 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-17T01:02:30.021 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-17T01:02:30.021 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-17T01:02:30.021 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-17T01:02:30.021 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-17T01:02:30.021 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.2.0-668.gab9fe435.el8 ceph 141 k 2023-10-17T01:02:30.022 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-17T01:02:30.022 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-17T01:02:30.022 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-17T01:02:30.022 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-17T01:02:30.023 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-17T01:02:30.023 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-17T01:02:30.023 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 474 k 2023-10-17T01:02:30.023 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-17T01:02:30.023 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 6.5 M 2023-10-17T01:02:30.024 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-17T01:02:30.024 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-17T01:02:30.024 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-17T01:02:30.024 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-17T01:02:30.025 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-17T01:02:30.025 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-17T01:02:30.025 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-17T01:02:30.025 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-17T01:02:30.026 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-17T01:02:30.026 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-17T01:02:30.026 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-17T01:02:30.026 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-17T01:02:30.027 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-17T01:02:30.027 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-17T01:02:30.027 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-17T01:02:30.027 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-17T01:02:30.028 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 45 k 2023-10-17T01:02:30.028 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.2.0-668.gab9fe435.el8 ceph 123 k 2023-10-17T01:02:30.028 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-17T01:02:30.028 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-17T01:02:30.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-17T01:02:30.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-17T01:02:30.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-17T01:02:30.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-17T01:02:30.029 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-17T01:02:30.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-17T01:02:30.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-17T01:02:30.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-17T01:02:30.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-17T01:02:30.030 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-17T01:02:30.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-17T01:02:30.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-17T01:02:30.031 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-17T01:02:30.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-17T01:02:30.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-17T01:02:30.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-17T01:02:30.032 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-17T01:02:30.033 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-17T01:02:30.033 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-17T01:02:30.033 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-17T01:02:30.033 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-17T01:02:30.033 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-17T01:02:30.034 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-17T01:02:30.034 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-17T01:02:30.034 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-17T01:02:30.034 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-17T01:02:30.034 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-17T01:02:30.035 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-17T01:02:30.035 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-17T01:02:30.035 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-17T01:02:30.035 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-17T01:02:30.036 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-17T01:02:30.036 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-17T01:02:30.036 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-17T01:02:30.036 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-17T01:02:30.036 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-17T01:02:30.037 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-17T01:02:30.037 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-17T01:02:30.037 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-17T01:02:30.037 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-17T01:02:30.038 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-17T01:02:30.038 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-17T01:02:30.038 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-17T01:02:30.038 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-17T01:02:30.039 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-17T01:02:30.039 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-17T01:02:30.039 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-17T01:02:30.039 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-10-17T01:02:30.039 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2023-10-17T01:02:30.040 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:02:30.040 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-17T01:02:30.040 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-10-17T01:02:30.040 INFO:teuthology.orchestra.run.smithi181.stdout:Install 128 Packages 2023-10-17T01:02:30.040 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 3 Packages 2023-10-17T01:02:30.041 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:02:30.041 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 203 M 2023-10-17T01:02:30.042 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-10-17T01:02:30.365 INFO:teuthology.orchestra.run.smithi181.stdout:(1/131): ceph-18.2.0-668.gab9fe435.el8.x86_64.r 24 kB/s | 6.4 kB 00:00 2023-10-17T01:02:30.849 INFO:teuthology.orchestra.run.smithi181.stdout:(2/131): ceph-base-18.2.0-668.gab9fe435.el8.x86 7.2 MB/s | 5.4 MB 00:00 2023-10-17T01:02:30.916 INFO:teuthology.orchestra.run.smithi181.stdout:(3/131): ceph-immutable-object-cache-18.2.0-668 2.4 MB/s | 165 kB 00:00 2023-10-17T01:02:31.066 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-17T01:02:31.067 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-17T01:02:31.082 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-17T01:02:31.083 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-17T01:02:31.085 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-17T01:02:31.086 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-17T01:02:31.088 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-17T01:02:31.092 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-17T01:02:31.093 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-17T01:02:31.244 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-10-17T01:02:31.258 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-10-17T01:02:31.258 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-10-17T01:02:31.258 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-10-17T01:02:31.259 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-10-17T01:02:31.259 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-17T01:02:31.259 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-17T01:02:31.259 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:18.2.0-668.gab9fe435.el8 ceph 6.4 k 2023-10-17T01:02:31.259 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:18.2.0-668.gab9fe435.el8 ceph 5.4 M 2023-10-17T01:02:31.260 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 847 k 2023-10-17T01:02:31.260 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-668.gab9fe435.el8 ceph 165 k 2023-10-17T01:02:31.260 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:18.2.0-668.gab9fe435.el8 ceph 1.5 M 2023-10-17T01:02:31.260 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 132 k 2023-10-17T01:02:31.261 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 1.8 M 2023-10-17T01:02:31.261 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 7.4 M 2023-10-17T01:02:31.261 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 47 k 2023-10-17T01:02:31.262 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:18.2.0-668.gab9fe435.el8 ceph 13 M 2023-10-17T01:02:31.262 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:18.2.0-668.gab9fe435.el8 ceph 47 M 2023-10-17T01:02:31.263 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 261 k 2023-10-17T01:02:31.263 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 167 k 2023-10-17T01:02:31.263 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:18.2.0-668.gab9fe435.el8 ceph 257 k 2023-10-17T01:02:31.264 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 18 k 2023-10-17T01:02:31.264 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-17T01:02:31.264 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-17T01:02:31.265 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-17T01:02:31.265 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:18.2.0-668.gab9fe435.el8 ceph 30 k 2023-10-17T01:02:31.266 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 770 k 2023-10-17T01:02:31.266 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:18.2.0-668.gab9fe435.el8 ceph 139 k 2023-10-17T01:02:31.266 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-17T01:02:31.267 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-17T01:02:31.267 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-17T01:02:31.267 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-17T01:02:31.268 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-17T01:02:31.268 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-17T01:02:31.269 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:18.2.0-668.gab9fe435.el8 ceph 190 k 2023-10-17T01:02:31.269 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:18.2.0-668.gab9fe435.el8 ceph 370 k 2023-10-17T01:02:31.269 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 364 k 2023-10-17T01:02:31.270 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:18.2.0-668.gab9fe435.el8 ceph 107 k 2023-10-17T01:02:31.270 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 94 k 2023-10-17T01:02:31.270 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.4 M 2023-10-17T01:02:31.271 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 179 k 2023-10-17T01:02:31.271 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-17T01:02:31.271 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2023-10-17T01:02:31.272 INFO:teuthology.orchestra.run.smithi045.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-17T01:02:31.272 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.7 M 2023-10-17T01:02:31.272 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.6 M 2023-10-17T01:02:31.273 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-10-17T01:02:31.273 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-17T01:02:31.274 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:18.2.0-668.gab9fe435.el8 ceph 22 M 2023-10-17T01:02:31.274 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 23 k 2023-10-17T01:02:31.274 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:18.2.0-668.gab9fe435.el8 ceph 2.3 M 2023-10-17T01:02:31.275 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 242 k 2023-10-17T01:02:31.275 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:18.2.0-668.gab9fe435.el8 ceph 4.6 M 2023-10-17T01:02:31.276 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 18 M 2023-10-17T01:02:31.276 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 15 k 2023-10-17T01:02:31.277 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:18.2.0-668.gab9fe435.el8 ceph 24 k 2023-10-17T01:02:31.277 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-17T01:02:31.277 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-17T01:02:31.278 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-17T01:02:31.278 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-17T01:02:31.278 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-17T01:02:31.279 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:18.2.0-668.gab9fe435.el8 ceph 141 k 2023-10-17T01:02:31.279 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-17T01:02:31.279 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-17T01:02:31.280 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-17T01:02:31.280 INFO:teuthology.orchestra.run.smithi045.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-17T01:02:31.281 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-17T01:02:31.281 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-17T01:02:31.281 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 474 k 2023-10-17T01:02:31.282 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-17T01:02:31.282 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 6.5 M 2023-10-17T01:02:31.282 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-17T01:02:31.283 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-17T01:02:31.283 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-17T01:02:31.283 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-17T01:02:31.284 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-17T01:02:31.284 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-17T01:02:31.284 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-17T01:02:31.285 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-17T01:02:31.285 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-17T01:02:31.286 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-17T01:02:31.286 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-17T01:02:31.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-17T01:02:31.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-17T01:02:31.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-17T01:02:31.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-17T01:02:31.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-17T01:02:31.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 45 k 2023-10-17T01:02:31.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:18.2.0-668.gab9fe435.el8 ceph 123 k 2023-10-17T01:02:31.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-17T01:02:31.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-17T01:02:31.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-17T01:02:31.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-17T01:02:31.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-17T01:02:31.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-17T01:02:31.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-17T01:02:31.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-17T01:02:31.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-17T01:02:31.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-17T01:02:31.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-17T01:02:31.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-17T01:02:31.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-17T01:02:31.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-17T01:02:31.295 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-17T01:02:31.295 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-17T01:02:31.295 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-17T01:02:31.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-17T01:02:31.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-17T01:02:31.296 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-17T01:02:31.297 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-17T01:02:31.297 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-17T01:02:31.297 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-17T01:02:31.298 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-17T01:02:31.298 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-17T01:02:31.298 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-17T01:02:31.299 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-17T01:02:31.299 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-17T01:02:31.299 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-17T01:02:31.300 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-17T01:02:31.300 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-17T01:02:31.300 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-17T01:02:31.300 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-17T01:02:31.300 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-17T01:02:31.301 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-17T01:02:31.301 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-17T01:02:31.301 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-17T01:02:31.301 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-17T01:02:31.301 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-17T01:02:31.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-17T01:02:31.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-17T01:02:31.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-17T01:02:31.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-17T01:02:31.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-17T01:02:31.303 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-17T01:02:31.303 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-17T01:02:31.303 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-17T01:02:31.303 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-17T01:02:31.303 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-17T01:02:31.304 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling module streams: 2023-10-17T01:02:31.304 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql 10 2023-10-17T01:02:31.304 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:02:31.304 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-10-17T01:02:31.305 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-10-17T01:02:31.305 INFO:teuthology.orchestra.run.smithi045.stdout:Install 128 Packages 2023-10-17T01:02:31.305 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 3 Packages 2023-10-17T01:02:31.305 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:02:31.306 INFO:teuthology.orchestra.run.smithi181.stdout:(4/131): ceph-fuse-18.2.0-668.gab9fe435.el8.x86 932 kB/s | 847 kB 00:00 2023-10-17T01:02:31.307 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 203 M 2023-10-17T01:02:31.307 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-10-17T01:02:31.725 INFO:teuthology.orchestra.run.smithi181.stdout:(5/131): ceph-common-18.2.0-668.gab9fe435.el8.x 13 MB/s | 22 MB 00:01 2023-10-17T01:02:31.916 INFO:teuthology.orchestra.run.smithi181.stdout:(6/131): ceph-mds-18.2.0-668.gab9fe435.el8.x86_ 2.3 MB/s | 2.3 MB 00:01 2023-10-17T01:02:32.405 INFO:teuthology.orchestra.run.smithi045.stdout:(1/131): ceph-18.2.0-668.gab9fe435.el8.x86_64.r 8.2 kB/s | 6.4 kB 00:00 2023-10-17T01:02:32.458 INFO:teuthology.orchestra.run.smithi181.stdout:(7/131): ceph-mon-18.2.0-668.gab9fe435.el8.x86_ 6.3 MB/s | 4.6 MB 00:00 2023-10-17T01:02:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:(8/131): ceph-mgr-18.2.0-668.gab9fe435.el8.x86_ 1.0 MB/s | 1.5 MB 00:01 2023-10-17T01:02:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:(9/131): ceph-selinux-18.2.0-668.gab9fe435.el8. 224 kB/s | 24 kB 00:00 2023-10-17T01:02:33.539 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-17T01:02:33.540 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-17T01:02:33.555 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-17T01:02:33.556 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-17T01:02:33.557 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-17T01:02:33.559 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-17T01:02:33.561 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-17T01:02:33.564 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-17T01:02:33.565 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-17T01:02:33.717 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-10-17T01:02:33.730 INFO:teuthology.orchestra.run.smithi060.stdout:=========================================================================================================================== 2023-10-17T01:02:33.730 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-10-17T01:02:33.730 INFO:teuthology.orchestra.run.smithi060.stdout:=========================================================================================================================== 2023-10-17T01:02:33.731 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-10-17T01:02:33.731 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-17T01:02:33.731 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-17T01:02:33.731 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:18.2.0-668.gab9fe435.el8 ceph 6.4 k 2023-10-17T01:02:33.731 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:18.2.0-668.gab9fe435.el8 ceph 5.4 M 2023-10-17T01:02:33.732 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 847 k 2023-10-17T01:02:33.732 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-668.gab9fe435.el8 ceph 165 k 2023-10-17T01:02:33.732 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:18.2.0-668.gab9fe435.el8 ceph 1.5 M 2023-10-17T01:02:33.732 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 132 k 2023-10-17T01:02:33.732 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 1.8 M 2023-10-17T01:02:33.733 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 7.4 M 2023-10-17T01:02:33.733 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 47 k 2023-10-17T01:02:33.733 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:18.2.0-668.gab9fe435.el8 ceph 13 M 2023-10-17T01:02:33.733 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:18.2.0-668.gab9fe435.el8 ceph 47 M 2023-10-17T01:02:33.733 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 261 k 2023-10-17T01:02:33.734 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 167 k 2023-10-17T01:02:33.734 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:18.2.0-668.gab9fe435.el8 ceph 257 k 2023-10-17T01:02:33.734 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 18 k 2023-10-17T01:02:33.734 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-17T01:02:33.734 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-17T01:02:33.735 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-17T01:02:33.735 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:18.2.0-668.gab9fe435.el8 ceph 30 k 2023-10-17T01:02:33.735 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 770 k 2023-10-17T01:02:33.735 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:18.2.0-668.gab9fe435.el8 ceph 139 k 2023-10-17T01:02:33.736 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-17T01:02:33.736 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-17T01:02:33.736 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-17T01:02:33.736 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-17T01:02:33.737 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-17T01:02:33.737 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-17T01:02:33.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:18.2.0-668.gab9fe435.el8 ceph 190 k 2023-10-17T01:02:33.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:18.2.0-668.gab9fe435.el8 ceph 370 k 2023-10-17T01:02:33.737 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 364 k 2023-10-17T01:02:33.738 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:18.2.0-668.gab9fe435.el8 ceph 107 k 2023-10-17T01:02:33.738 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 94 k 2023-10-17T01:02:33.738 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.4 M 2023-10-17T01:02:33.738 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 179 k 2023-10-17T01:02:33.738 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-17T01:02:33.739 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2023-10-17T01:02:33.739 INFO:teuthology.orchestra.run.smithi060.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-17T01:02:33.739 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.7 M 2023-10-17T01:02:33.739 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 3.6 M 2023-10-17T01:02:33.739 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-10-17T01:02:33.740 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-17T01:02:33.740 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:18.2.0-668.gab9fe435.el8 ceph 22 M 2023-10-17T01:02:33.740 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 23 k 2023-10-17T01:02:33.740 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:18.2.0-668.gab9fe435.el8 ceph 2.3 M 2023-10-17T01:02:33.740 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 242 k 2023-10-17T01:02:33.741 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:18.2.0-668.gab9fe435.el8 ceph 4.6 M 2023-10-17T01:02:33.741 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:18.2.0-668.gab9fe435.el8 ceph 18 M 2023-10-17T01:02:33.741 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:18.2.0-668.gab9fe435.el8 ceph-noarch 15 k 2023-10-17T01:02:33.741 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:18.2.0-668.gab9fe435.el8 ceph 24 k 2023-10-17T01:02:33.741 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-17T01:02:33.741 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-17T01:02:33.742 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-17T01:02:33.742 INFO:teuthology.orchestra.run.smithi060.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-17T01:02:33.742 INFO:teuthology.orchestra.run.smithi060.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-17T01:02:33.742 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:18.2.0-668.gab9fe435.el8 ceph 141 k 2023-10-17T01:02:33.742 INFO:teuthology.orchestra.run.smithi060.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-17T01:02:33.743 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-17T01:02:33.743 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-17T01:02:33.743 INFO:teuthology.orchestra.run.smithi060.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-17T01:02:33.743 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-17T01:02:33.743 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-17T01:02:33.744 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 474 k 2023-10-17T01:02:33.744 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-17T01:02:33.744 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:18.2.0-668.gab9fe435.el8 ceph 6.5 M 2023-10-17T01:02:33.744 INFO:teuthology.orchestra.run.smithi060.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-17T01:02:33.744 INFO:teuthology.orchestra.run.smithi060.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-17T01:02:33.745 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-17T01:02:33.745 INFO:teuthology.orchestra.run.smithi060.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-17T01:02:33.745 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-17T01:02:33.745 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-17T01:02:33.745 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-17T01:02:33.746 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-17T01:02:33.746 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-17T01:02:33.746 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-17T01:02:33.747 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-17T01:02:33.747 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-17T01:02:33.747 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-17T01:02:33.747 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-17T01:02:33.747 INFO:teuthology.orchestra.run.smithi060.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-17T01:02:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-17T01:02:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:18.2.0-668.gab9fe435.el8 ceph 45 k 2023-10-17T01:02:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:18.2.0-668.gab9fe435.el8 ceph 123 k 2023-10-17T01:02:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-17T01:02:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-17T01:02:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-17T01:02:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-17T01:02:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-17T01:02:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-17T01:02:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-17T01:02:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-17T01:02:33.750 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-17T01:02:33.750 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-17T01:02:33.750 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-17T01:02:33.750 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-17T01:02:33.751 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-17T01:02:33.751 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-17T01:02:33.751 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-17T01:02:33.751 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-17T01:02:33.752 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-17T01:02:33.752 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-17T01:02:33.752 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-17T01:02:33.752 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-17T01:02:33.752 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-17T01:02:33.753 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-17T01:02:33.753 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-17T01:02:33.753 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-17T01:02:33.753 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-17T01:02:33.753 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-17T01:02:33.754 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-17T01:02:33.754 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-17T01:02:33.754 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-17T01:02:33.754 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-17T01:02:33.754 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-17T01:02:33.755 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-17T01:02:33.755 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-17T01:02:33.755 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-17T01:02:33.755 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-17T01:02:33.755 INFO:teuthology.orchestra.run.smithi060.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-17T01:02:33.756 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-17T01:02:33.756 INFO:teuthology.orchestra.run.smithi060.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-17T01:02:33.756 INFO:teuthology.orchestra.run.smithi060.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-17T01:02:33.756 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-17T01:02:33.757 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-17T01:02:33.757 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-17T01:02:33.757 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-17T01:02:33.757 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-17T01:02:33.758 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-17T01:02:33.758 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-17T01:02:33.758 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-17T01:02:33.758 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-17T01:02:33.758 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-17T01:02:33.759 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling module streams: 2023-10-17T01:02:33.759 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql 10 2023-10-17T01:02:33.759 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:02:33.759 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-10-17T01:02:33.759 INFO:teuthology.orchestra.run.smithi060.stdout:=========================================================================================================================== 2023-10-17T01:02:33.760 INFO:teuthology.orchestra.run.smithi060.stdout:Install 128 Packages 2023-10-17T01:02:33.760 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 3 Packages 2023-10-17T01:02:33.760 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:02:33.761 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 203 M 2023-10-17T01:02:33.761 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-10-17T01:02:34.257 INFO:teuthology.orchestra.run.smithi060.stdout:(1/131): ceph-18.2.0-668.gab9fe435.el8.x86_64.r 15 kB/s | 6.4 kB 00:00 2023-10-17T01:02:34.266 INFO:teuthology.orchestra.run.smithi045.stdout:(2/131): ceph-fuse-18.2.0-668.gab9fe435.el8.x86 455 kB/s | 847 kB 00:01 2023-10-17T01:02:34.400 INFO:teuthology.orchestra.run.smithi181.stdout:(10/131): ceph-radosgw-18.2.0-668.gab9fe435.el8 6.7 MB/s | 13 MB 00:01 2023-10-17T01:02:34.501 INFO:teuthology.orchestra.run.smithi181.stdout:(11/131): cephfs-mirror-18.2.0-668.gab9fe435.el 2.5 MB/s | 257 kB 00:00 2023-10-17T01:02:34.564 INFO:teuthology.orchestra.run.smithi045.stdout:(3/131): ceph-immutable-object-cache-18.2.0-668 556 kB/s | 165 kB 00:00 2023-10-17T01:02:34.584 INFO:teuthology.orchestra.run.smithi181.stdout:(12/131): libcephfs-devel-18.2.0-668.gab9fe435. 366 kB/s | 30 kB 00:00 2023-10-17T01:02:34.893 INFO:teuthology.orchestra.run.smithi181.stdout:(13/131): libcephfs2-18.2.0-668.gab9fe435.el8.x 2.4 MB/s | 770 kB 00:00 2023-10-17T01:02:35.018 INFO:teuthology.orchestra.run.smithi181.stdout:(14/131): libcephsqlite-18.2.0-668.gab9fe435.el 1.1 MB/s | 141 kB 00:00 2023-10-17T01:02:35.119 INFO:teuthology.orchestra.run.smithi181.stdout:(15/131): librados-devel-18.2.0-668.gab9fe435.e 1.4 MB/s | 139 kB 00:00 2023-10-17T01:02:35.568 INFO:teuthology.orchestra.run.smithi181.stdout:(16/131): libradosstriper1-18.2.0-668.gab9fe435 1.0 MB/s | 474 kB 00:00 2023-10-17T01:02:35.624 INFO:teuthology.orchestra.run.smithi060.stdout:(2/131): ceph-fuse-18.2.0-668.gab9fe435.el8.x86 620 kB/s | 847 kB 00:01 2023-10-17T01:02:35.899 INFO:teuthology.orchestra.run.smithi060.stdout:(3/131): ceph-immutable-object-cache-18.2.0-668 602 kB/s | 165 kB 00:00 2023-10-17T01:02:36.069 INFO:teuthology.orchestra.run.smithi181.stdout:(17/131): ceph-osd-18.2.0-668.gab9fe435.el8.x86 4.3 MB/s | 18 MB 00:04 2023-10-17T01:02:36.169 INFO:teuthology.orchestra.run.smithi181.stdout:(18/131): python3-ceph-argparse-18.2.0-668.gab9 448 kB/s | 45 kB 00:00 2023-10-17T01:02:36.261 INFO:teuthology.orchestra.run.smithi181.stdout:(19/131): python3-ceph-common-18.2.0-668.gab9fe 1.3 MB/s | 123 kB 00:00 2023-10-17T01:02:36.353 INFO:teuthology.orchestra.run.smithi181.stdout:(20/131): python3-cephfs-18.2.0-668.gab9fe435.e 2.0 MB/s | 190 kB 00:00 2023-10-17T01:02:36.528 INFO:teuthology.orchestra.run.smithi181.stdout:(21/131): python3-rados-18.2.0-668.gab9fe435.el 2.1 MB/s | 370 kB 00:00 2023-10-17T01:02:36.712 INFO:teuthology.orchestra.run.smithi181.stdout:(22/131): python3-rbd-18.2.0-668.gab9fe435.el8. 1.9 MB/s | 364 kB 00:00 2023-10-17T01:02:36.812 INFO:teuthology.orchestra.run.smithi181.stdout:(23/131): python3-rgw-18.2.0-668.gab9fe435.el8. 1.0 MB/s | 107 kB 00:00 2023-10-17T01:02:36.912 INFO:teuthology.orchestra.run.smithi181.stdout:(24/131): rbd-fuse-18.2.0-668.gab9fe435.el8.x86 946 kB/s | 94 kB 00:00 2023-10-17T01:02:37.171 INFO:teuthology.orchestra.run.smithi181.stdout:(25/131): librgw2-18.2.0-668.gab9fe435.el8.x86_ 4.1 MB/s | 6.5 MB 00:01 2023-10-17T01:02:37.263 INFO:teuthology.orchestra.run.smithi181.stdout:(26/131): rbd-nbd-18.2.0-668.gab9fe435.el8.x86_ 1.9 MB/s | 179 kB 00:00 2023-10-17T01:02:37.355 INFO:teuthology.orchestra.run.smithi181.stdout:(27/131): ceph-grafana-dashboards-18.2.0-668.ga 252 kB/s | 23 kB 00:00 2023-10-17T01:02:37.455 INFO:teuthology.orchestra.run.smithi181.stdout:(28/131): ceph-mgr-cephadm-18.2.0-668.gab9fe435 1.3 MB/s | 132 kB 00:00 2023-10-17T01:02:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:(4/131): ceph-base-18.2.0-668.gab9fe435.el8.x86 895 kB/s | 5.4 MB 00:06 2023-10-17T01:02:37.955 INFO:teuthology.orchestra.run.smithi181.stdout:(29/131): rbd-mirror-18.2.0-668.gab9fe435.el8.x 3.3 MB/s | 3.4 MB 00:01 2023-10-17T01:02:37.958 INFO:teuthology.orchestra.run.smithi060.stdout:(4/131): ceph-common-18.2.0-668.gab9fe435.el8.x 5.2 MB/s | 22 MB 00:04 2023-10-17T01:02:37.997 INFO:teuthology.orchestra.run.smithi181.stdout:(30/131): ceph-mgr-dashboard-18.2.0-668.gab9fe4 3.3 MB/s | 1.8 MB 00:00 2023-10-17T01:02:38.075 INFO:teuthology.orchestra.run.smithi060.stdout:(5/131): ceph-base-18.2.0-668.gab9fe435.el8.x86 1.3 MB/s | 5.4 MB 00:04 2023-10-17T01:02:38.131 INFO:teuthology.orchestra.run.smithi181.stdout:(31/131): ceph-mgr-modules-core-18.2.0-668.gab9 1.8 MB/s | 242 kB 00:00 2023-10-17T01:02:38.206 INFO:teuthology.orchestra.run.smithi181.stdout:(32/131): ceph-mgr-rook-18.2.0-668.gab9fe435.el 631 kB/s | 47 kB 00:00 2023-10-17T01:02:38.258 INFO:teuthology.orchestra.run.smithi060.stdout:(6/131): ceph-mds-18.2.0-668.gab9fe435.el8.x86_ 990 kB/s | 2.3 MB 00:02 2023-10-17T01:02:38.273 INFO:teuthology.orchestra.run.smithi181.stdout:(33/131): ceph-prometheus-alerts-18.2.0-668.gab 218 kB/s | 15 kB 00:00 2023-10-17T01:02:38.325 INFO:teuthology.orchestra.run.smithi060.stdout:(7/131): ceph-mgr-18.2.0-668.gab9fe435.el8.x86_ 4.2 MB/s | 1.5 MB 00:00 2023-10-17T01:02:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:(5/131): ceph-common-18.2.0-668.gab9fe435.el8.x 3.2 MB/s | 22 MB 00:06 2023-10-17T01:02:38.374 INFO:teuthology.orchestra.run.smithi181.stdout:(34/131): ceph-volume-18.2.0-668.gab9fe435.el8. 2.6 MB/s | 261 kB 00:00 2023-10-17T01:02:38.414 INFO:teuthology.orchestra.run.smithi045.stdout:(6/131): ceph-mds-18.2.0-668.gab9fe435.el8.x86_ 606 kB/s | 2.3 MB 00:03 2023-10-17T01:02:38.474 INFO:teuthology.orchestra.run.smithi181.stdout:(35/131): cephadm-18.2.0-668.gab9fe435.el8.noar 1.6 MB/s | 167 kB 00:00 2023-10-17T01:02:38.557 INFO:teuthology.orchestra.run.smithi181.stdout:(36/131): cephfs-top-18.2.0-668.gab9fe435.el8.n 216 kB/s | 18 kB 00:00 2023-10-17T01:02:38.690 INFO:teuthology.orchestra.run.smithi045.stdout:(7/131): ceph-mgr-18.2.0-668.gab9fe435.el8.x86_ 1.7 MB/s | 1.5 MB 00:00 2023-10-17T01:02:39.399 INFO:teuthology.orchestra.run.smithi181.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 364 kB/s | 306 kB 00:00 2023-10-17T01:02:39.465 INFO:teuthology.orchestra.run.smithi045.stdout:(8/131): ceph-mon-18.2.0-668.gab9fe435.el8.x86_ 4.1 MB/s | 4.6 MB 00:01 2023-10-17T01:02:39.583 INFO:teuthology.orchestra.run.smithi181.stdout:(38/131): ceph-mgr-diskprediction-local-18.2.0- 4.5 MB/s | 7.4 MB 00:01 2023-10-17T01:02:39.607 INFO:teuthology.orchestra.run.smithi045.stdout:(9/131): ceph-selinux-18.2.0-668.gab9fe435.el8. 171 kB/s | 24 kB 00:00 2023-10-17T01:02:39.608 INFO:teuthology.orchestra.run.smithi181.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 282 kB/s | 59 kB 00:00 2023-10-17T01:02:40.217 INFO:teuthology.orchestra.run.smithi181.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 118 kB/s | 75 kB 00:00 2023-10-17T01:02:40.300 INFO:teuthology.orchestra.run.smithi060.stdout:(8/131): ceph-radosgw-18.2.0-668.gab9fe435.el8. 6.6 MB/s | 13 MB 00:01 2023-10-17T01:02:40.334 INFO:teuthology.orchestra.run.smithi181.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2023-10-17T01:02:40.359 INFO:teuthology.orchestra.run.smithi060.stdout:(9/131): ceph-selinux-18.2.0-668.gab9fe435.el8. 419 kB/s | 24 kB 00:00 2023-10-17T01:02:40.618 INFO:teuthology.orchestra.run.smithi181.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 654 kB/s | 185 kB 00:00 2023-10-17T01:02:40.718 INFO:teuthology.orchestra.run.smithi181.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-10-17T01:02:40.843 INFO:teuthology.orchestra.run.smithi181.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-10-17T01:02:40.977 INFO:teuthology.orchestra.run.smithi181.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 348 kB/s | 476 kB 00:01 2023-10-17T01:02:41.110 INFO:teuthology.orchestra.run.smithi181.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 646 kB/s | 173 kB 00:00 2023-10-17T01:02:41.302 INFO:teuthology.orchestra.run.smithi181.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 424 kB/s | 81 kB 00:00 2023-10-17T01:02:41.402 INFO:teuthology.orchestra.run.smithi181.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-10-17T01:02:41.511 INFO:teuthology.orchestra.run.smithi181.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-10-17T01:02:41.611 INFO:teuthology.orchestra.run.smithi181.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 606 kB/s | 384 kB 00:00 2023-10-17T01:02:41.659 INFO:teuthology.orchestra.run.smithi060.stdout:(10/131): ceph-mon-18.2.0-668.gab9fe435.el8.x86 1.3 MB/s | 4.6 MB 00:03 2023-10-17T01:02:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 417 kB/s | 49 kB 00:00 2023-10-17T01:02:41.854 INFO:teuthology.orchestra.run.smithi181.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 469 kB/s | 59 kB 00:00 2023-10-17T01:02:41.917 INFO:teuthology.orchestra.run.smithi060.stdout:(11/131): cephfs-mirror-18.2.0-668.gab9fe435.el 998 kB/s | 257 kB 00:00 2023-10-17T01:02:41.962 INFO:teuthology.orchestra.run.smithi181.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 556 kB/s | 60 kB 00:00 2023-10-17T01:02:42.010 INFO:teuthology.orchestra.run.smithi060.stdout:(12/131): libcephfs-devel-18.2.0-668.gab9fe435. 332 kB/s | 30 kB 00:00 2023-10-17T01:02:42.413 INFO:teuthology.orchestra.run.smithi181.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 629 kB/s | 283 kB 00:00 2023-10-17T01:02:42.521 INFO:teuthology.orchestra.run.smithi181.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-10-17T01:02:42.638 INFO:teuthology.orchestra.run.smithi181.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 288 kB/s | 34 kB 00:00 2023-10-17T01:02:42.710 INFO:teuthology.orchestra.run.smithi060.stdout:(13/131): libcephfs2-18.2.0-668.gab9fe435.el8.x 1.1 MB/s | 770 kB 00:00 2023-10-17T01:02:42.893 INFO:teuthology.orchestra.run.smithi060.stdout:(14/131): libcephsqlite-18.2.0-668.gab9fe435.el 772 kB/s | 141 kB 00:00 2023-10-17T01:02:42.988 INFO:teuthology.orchestra.run.smithi181.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 562 kB/s | 196 kB 00:00 2023-10-17T01:02:43.072 INFO:teuthology.orchestra.run.smithi181.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 765 kB/s | 1.2 MB 00:01 2023-10-17T01:02:43.085 INFO:teuthology.orchestra.run.smithi060.stdout:(15/131): librados-devel-18.2.0-668.gab9fe435.e 726 kB/s | 139 kB 00:00 2023-10-17T01:02:43.181 INFO:teuthology.orchestra.run.smithi181.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 334 kB/s | 64 kB 00:00 2023-10-17T01:02:43.206 INFO:teuthology.orchestra.run.smithi181.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-10-17T01:02:43.281 INFO:teuthology.orchestra.run.smithi181.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2023-10-17T01:02:43.315 INFO:teuthology.orchestra.run.smithi181.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-10-17T01:02:43.390 INFO:teuthology.orchestra.run.smithi181.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-10-17T01:02:43.502 INFO:teuthology.orchestra.run.smithi060.stdout:(16/131): libradosstriper1-18.2.0-668.gab9fe435 1.1 MB/s | 474 kB 00:00 2023-10-17T01:02:43.665 INFO:teuthology.orchestra.run.smithi181.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 729 kB/s | 255 kB 00:00 2023-10-17T01:02:43.690 INFO:teuthology.orchestra.run.smithi045.stdout:(10/131): ceph-radosgw-18.2.0-668.gab9fe435.el8 2.6 MB/s | 13 MB 00:04 2023-10-17T01:02:43.741 INFO:teuthology.orchestra.run.smithi181.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 717 kB/s | 251 kB 00:00 2023-10-17T01:02:43.774 INFO:teuthology.orchestra.run.smithi181.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-10-17T01:02:43.832 INFO:teuthology.orchestra.run.smithi045.stdout:(11/131): cephfs-mirror-18.2.0-668.gab9fe435.el 1.8 MB/s | 257 kB 00:00 2023-10-17T01:02:43.883 INFO:teuthology.orchestra.run.smithi181.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 212 kB/s | 23 kB 00:00 2023-10-17T01:02:43.907 INFO:teuthology.orchestra.run.smithi045.stdout:(12/131): libcephfs-devel-18.2.0-668.gab9fe435. 409 kB/s | 30 kB 00:00 2023-10-17T01:02:43.924 INFO:teuthology.orchestra.run.smithi181.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 465 kB/s | 85 kB 00:00 2023-10-17T01:02:43.950 INFO:teuthology.orchestra.run.smithi181.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-17T01:02:44.182 INFO:teuthology.orchestra.run.smithi045.stdout:(13/131): libcephfs2-18.2.0-668.gab9fe435.el8.x 2.7 MB/s | 770 kB 00:00 2023-10-17T01:02:44.242 INFO:teuthology.orchestra.run.smithi181.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 47 MB/s | 14 MB 00:00 2023-10-17T01:02:44.266 INFO:teuthology.orchestra.run.smithi045.stdout:(14/131): libcephsqlite-18.2.0-668.gab9fe435.el 1.7 MB/s | 141 kB 00:00 2023-10-17T01:02:44.341 INFO:teuthology.orchestra.run.smithi045.stdout:(15/131): librados-devel-18.2.0-668.gab9fe435.e 1.8 MB/s | 139 kB 00:00 2023-10-17T01:02:44.426 INFO:teuthology.orchestra.run.smithi181.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 601 kB/s | 110 kB 00:00 2023-10-17T01:02:44.459 INFO:teuthology.orchestra.run.smithi181.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-17T01:02:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:(16/131): libradosstriper1-18.2.0-668.gab9fe435 3.1 MB/s | 474 kB 00:00 2023-10-17T01:02:44.501 INFO:teuthology.orchestra.run.smithi181.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-17T01:02:44.543 INFO:teuthology.orchestra.run.smithi181.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-10-17T01:02:44.568 INFO:teuthology.orchestra.run.smithi181.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.5 MB/s | 39 kB 00:00 2023-10-17T01:02:44.619 INFO:teuthology.orchestra.run.smithi181.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-10-17T01:02:44.686 INFO:teuthology.orchestra.run.smithi181.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-10-17T01:02:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-10-17T01:02:44.778 INFO:teuthology.orchestra.run.smithi181.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-17T01:02:44.820 INFO:teuthology.orchestra.run.smithi181.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 17 MB/s | 713 kB 00:00 2023-10-17T01:02:44.946 INFO:teuthology.orchestra.run.smithi181.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 38 MB/s | 4.8 MB 00:00 2023-10-17T01:02:44.979 INFO:teuthology.orchestra.run.smithi181.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 799 kB/s | 27 kB 00:00 2023-10-17T01:02:45.021 INFO:teuthology.orchestra.run.smithi181.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-17T01:02:45.055 INFO:teuthology.orchestra.run.smithi181.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-17T01:02:45.172 INFO:teuthology.orchestra.run.smithi181.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 32 MB/s | 3.7 MB 00:00 2023-10-17T01:02:45.213 INFO:teuthology.orchestra.run.smithi181.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-10-17T01:02:45.247 INFO:teuthology.orchestra.run.smithi181.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.6 MB/s | 225 kB 00:00 2023-10-17T01:02:45.289 INFO:teuthology.orchestra.run.smithi181.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-10-17T01:02:45.331 INFO:teuthology.orchestra.run.smithi181.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-10-17T01:02:45.373 INFO:teuthology.orchestra.run.smithi181.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 479 kB/s | 20 kB 00:00 2023-10-17T01:02:45.415 INFO:teuthology.orchestra.run.smithi181.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-10-17T01:02:45.457 INFO:teuthology.orchestra.run.smithi181.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 17 MB/s | 709 kB 00:00 2023-10-17T01:02:45.582 INFO:teuthology.orchestra.run.smithi181.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 39 MB/s | 4.9 MB 00:00 2023-10-17T01:02:45.641 INFO:teuthology.orchestra.run.smithi181.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-10-17T01:02:45.717 INFO:teuthology.orchestra.run.smithi045.stdout:(17/131): librgw2-18.2.0-668.gab9fe435.el8.x86_ 5.3 MB/s | 6.5 MB 00:01 2023-10-17T01:02:45.758 INFO:teuthology.orchestra.run.smithi181.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 41 MB/s | 4.7 MB 00:00 2023-10-17T01:02:45.783 INFO:teuthology.orchestra.run.smithi045.stdout:(18/131): python3-ceph-argparse-18.2.0-668.gab9 673 kB/s | 45 kB 00:00 2023-10-17T01:02:45.800 INFO:teuthology.orchestra.run.smithi181.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-17T01:02:45.841 INFO:teuthology.orchestra.run.smithi181.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-17T01:02:45.850 INFO:teuthology.orchestra.run.smithi045.stdout:(19/131): python3-ceph-common-18.2.0-668.gab9fe 1.8 MB/s | 123 kB 00:00 2023-10-17T01:02:45.951 INFO:teuthology.orchestra.run.smithi045.stdout:(20/131): python3-cephfs-18.2.0-668.gab9fe435.e 1.9 MB/s | 190 kB 00:00 2023-10-17T01:02:45.958 INFO:teuthology.orchestra.run.smithi181.stdout:(98/131): postgresql-server-10.21-2.module+el8. 43 MB/s | 5.1 MB 00:00 2023-10-17T01:02:46.009 INFO:teuthology.orchestra.run.smithi181.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-10-17T01:02:46.067 INFO:teuthology.orchestra.run.smithi181.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 26 MB/s | 1.5 MB 00:00 2023-10-17T01:02:46.076 INFO:teuthology.orchestra.run.smithi045.stdout:(21/131): python3-rados-18.2.0-668.gab9fe435.el 2.9 MB/s | 370 kB 00:00 2023-10-17T01:02:46.101 INFO:teuthology.orchestra.run.smithi181.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-10-17T01:02:46.143 INFO:teuthology.orchestra.run.smithi181.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.7 MB/s | 200 kB 00:00 2023-10-17T01:02:46.160 INFO:teuthology.orchestra.run.smithi045.stdout:(22/131): python3-rbd-18.2.0-668.gab9fe435.el8. 4.3 MB/s | 364 kB 00:00 2023-10-17T01:02:46.177 INFO:teuthology.orchestra.run.smithi181.stdout:(103/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-10-17T01:02:46.210 INFO:teuthology.orchestra.run.smithi181.stdout:(104/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-10-17T01:02:46.235 INFO:teuthology.orchestra.run.smithi045.stdout:(23/131): python3-rgw-18.2.0-668.gab9fe435.el8. 1.4 MB/s | 107 kB 00:00 2023-10-17T01:02:46.244 INFO:teuthology.orchestra.run.smithi181.stdout:(105/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.4 MB/s | 48 kB 00:00 2023-10-17T01:02:46.277 INFO:teuthology.orchestra.run.smithi181.stdout:(106/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.6 MB/s | 87 kB 00:00 2023-10-17T01:02:46.311 INFO:teuthology.orchestra.run.smithi181.stdout:(107/131): python3-requests-oauthlib-1.0.0-1.el 1.2 MB/s | 43 kB 00:00 2023-10-17T01:02:46.318 INFO:teuthology.orchestra.run.smithi045.stdout:(24/131): rbd-fuse-18.2.0-668.gab9fe435.el8.x86 1.1 MB/s | 94 kB 00:00 2023-10-17T01:02:46.353 INFO:teuthology.orchestra.run.smithi181.stdout:(108/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-10-17T01:02:46.395 INFO:teuthology.orchestra.run.smithi181.stdout:(109/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-10-17T01:02:46.437 INFO:teuthology.orchestra.run.smithi181.stdout:(110/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 13 MB/s | 560 kB 00:00 2023-10-17T01:02:46.462 INFO:teuthology.orchestra.run.smithi181.stdout:(111/131): libaio-devel-0.3.112-1.el8.x86_64.rp 754 kB/s | 19 kB 00:00 2023-10-17T01:02:46.504 INFO:teuthology.orchestra.run.smithi181.stdout:(112/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-10-17T01:02:46.546 INFO:teuthology.orchestra.run.smithi181.stdout:(113/131): python3-cryptography-3.2.1-5.el8.x86 13 MB/s | 559 kB 00:00 2023-10-17T01:02:46.605 INFO:teuthology.orchestra.run.smithi181.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 307 kB/s | 18 kB 00:00 2023-10-17T01:02:46.655 INFO:teuthology.orchestra.run.smithi181.stdout:(115/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-10-17T01:02:46.697 INFO:teuthology.orchestra.run.smithi181.stdout:(116/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-10-17T01:02:46.781 INFO:teuthology.orchestra.run.smithi181.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 28 MB/s | 2.3 MB 00:00 2023-10-17T01:02:46.814 INFO:teuthology.orchestra.run.smithi181.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-10-17T01:02:46.827 INFO:teuthology.orchestra.run.smithi045.stdout:(25/131): rbd-mirror-18.2.0-668.gab9fe435.el8.x 6.8 MB/s | 3.4 MB 00:00 2023-10-17T01:02:46.856 INFO:teuthology.orchestra.run.smithi181.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-10-17T01:02:46.885 INFO:teuthology.orchestra.run.smithi045.stdout:(26/131): rbd-nbd-18.2.0-668.gab9fe435.el8.x86_ 3.0 MB/s | 179 kB 00:00 2023-10-17T01:02:46.898 INFO:teuthology.orchestra.run.smithi181.stdout:(120/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-10-17T01:02:46.936 INFO:teuthology.orchestra.run.smithi045.stdout:(27/131): ceph-grafana-dashboards-18.2.0-668.ga 461 kB/s | 23 kB 00:00 2023-10-17T01:02:46.940 INFO:teuthology.orchestra.run.smithi181.stdout:(121/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-10-17T01:02:46.982 INFO:teuthology.orchestra.run.smithi181.stdout:(122/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.4 MB/s | 58 kB 00:00 2023-10-17T01:02:46.994 INFO:teuthology.orchestra.run.smithi060.stdout:(17/131): ceph-test-18.2.0-668.gab9fe435.el8.x8 7.0 MB/s | 47 MB 00:06 2023-10-17T01:02:46.995 INFO:teuthology.orchestra.run.smithi045.stdout:(28/131): ceph-mgr-cephadm-18.2.0-668.gab9fe435 2.2 MB/s | 132 kB 00:00 2023-10-17T01:02:47.016 INFO:teuthology.orchestra.run.smithi181.stdout:(123/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-10-17T01:02:47.044 INFO:teuthology.orchestra.run.smithi060.stdout:(18/131): python3-ceph-argparse-18.2.0-668.gab9 899 kB/s | 45 kB 00:00 2023-10-17T01:02:47.058 INFO:teuthology.orchestra.run.smithi181.stdout:(124/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 15 MB/s | 644 kB 00:00 2023-10-17T01:02:47.100 INFO:teuthology.orchestra.run.smithi181.stdout:(125/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 4.0 MB/s | 171 kB 00:00 2023-10-17T01:02:47.142 INFO:teuthology.orchestra.run.smithi181.stdout:(126/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-10-17T01:02:47.170 INFO:teuthology.orchestra.run.smithi060.stdout:(19/131): python3-ceph-common-18.2.0-668.gab9fe 989 kB/s | 123 kB 00:00 2023-10-17T01:02:47.270 INFO:teuthology.orchestra.run.smithi045.stdout:(29/131): ceph-mgr-dashboard-18.2.0-668.gab9fe4 6.4 MB/s | 1.8 MB 00:00 2023-10-17T01:02:47.278 INFO:teuthology.orchestra.run.smithi060.stdout:(20/131): python3-cephfs-18.2.0-668.gab9fe435.e 1.7 MB/s | 190 kB 00:00 2023-10-17T01:02:47.420 INFO:teuthology.orchestra.run.smithi060.stdout:(21/131): python3-rados-18.2.0-668.gab9fe435.el 2.6 MB/s | 370 kB 00:00 2023-10-17T01:02:47.521 INFO:teuthology.orchestra.run.smithi060.stdout:(22/131): python3-rbd-18.2.0-668.gab9fe435.el8. 3.6 MB/s | 364 kB 00:00 2023-10-17T01:02:47.613 INFO:teuthology.orchestra.run.smithi060.stdout:(23/131): python3-rgw-18.2.0-668.gab9fe435.el8. 1.1 MB/s | 107 kB 00:00 2023-10-17T01:02:47.675 INFO:teuthology.orchestra.run.smithi181.stdout:(127/131): thrift-0.13.0-2.el8.x86_64.rpm 468 kB/s | 1.7 MB 00:03 2023-10-17T01:02:47.705 INFO:teuthology.orchestra.run.smithi060.stdout:(24/131): rbd-fuse-18.2.0-668.gab9fe435.el8.x86 1.0 MB/s | 94 kB 00:00 2023-10-17T01:02:48.379 INFO:teuthology.orchestra.run.smithi045.stdout:(30/131): ceph-mgr-diskprediction-local-18.2.0- 6.7 MB/s | 7.4 MB 00:01 2023-10-17T01:02:48.384 INFO:teuthology.orchestra.run.smithi181.stdout:(128/131): librados2-18.2.0-668.gab9fe435.el8.x 3.0 MB/s | 3.7 MB 00:01 2023-10-17T01:02:48.442 INFO:teuthology.orchestra.run.smithi181.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-17T01:02:48.462 INFO:teuthology.orchestra.run.smithi045.stdout:(31/131): ceph-mgr-modules-core-18.2.0-668.gab9 2.8 MB/s | 242 kB 00:00 2023-10-17T01:02:48.471 INFO:teuthology.orchestra.run.smithi060.stdout:(25/131): rbd-mirror-18.2.0-668.gab9fe435.el8.x 4.5 MB/s | 3.4 MB 00:00 2023-10-17T01:02:48.521 INFO:teuthology.orchestra.run.smithi045.stdout:(32/131): ceph-mgr-rook-18.2.0-668.gab9fe435.el 816 kB/s | 47 kB 00:00 2023-10-17T01:02:48.538 INFO:teuthology.orchestra.run.smithi060.stdout:(26/131): rbd-nbd-18.2.0-668.gab9fe435.el8.x86_ 2.6 MB/s | 179 kB 00:00 2023-10-17T01:02:48.579 INFO:teuthology.orchestra.run.smithi045.stdout:(33/131): ceph-prometheus-alerts-18.2.0-668.gab 250 kB/s | 15 kB 00:00 2023-10-17T01:02:48.597 INFO:teuthology.orchestra.run.smithi060.stdout:(27/131): ceph-grafana-dashboards-18.2.0-668.ga 396 kB/s | 23 kB 00:00 2023-10-17T01:02:48.646 INFO:teuthology.orchestra.run.smithi045.stdout:(34/131): ceph-volume-18.2.0-668.gab9fe435.el8. 3.8 MB/s | 261 kB 00:00 2023-10-17T01:02:48.672 INFO:teuthology.orchestra.run.smithi060.stdout:(28/131): ceph-mgr-cephadm-18.2.0-668.gab9fe435 1.7 MB/s | 132 kB 00:00 2023-10-17T01:02:48.713 INFO:teuthology.orchestra.run.smithi045.stdout:(35/131): cephadm-18.2.0-668.gab9fe435.el8.noar 2.5 MB/s | 167 kB 00:00 2023-10-17T01:02:48.784 INFO:teuthology.orchestra.run.smithi181.stdout:(130/131): librbd1-18.2.0-668.gab9fe435.el8.x86 3.2 MB/s | 3.6 MB 00:01 2023-10-17T01:02:48.789 INFO:teuthology.orchestra.run.smithi045.stdout:(36/131): cephfs-top-18.2.0-668.gab9fe435.el8.n 240 kB/s | 18 kB 00:00 2023-10-17T01:02:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:(29/131): ceph-mgr-dashboard-18.2.0-668.gab9fe4 6.2 MB/s | 1.8 MB 00:00 2023-10-17T01:02:49.856 INFO:teuthology.orchestra.run.smithi045.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 287 kB/s | 306 kB 00:01 2023-10-17T01:02:49.917 INFO:teuthology.orchestra.run.smithi060.stdout:(30/131): ceph-mgr-diskprediction-local-18.2.0- 7.7 MB/s | 7.4 MB 00:00 2023-10-17T01:02:49.981 INFO:teuthology.orchestra.run.smithi045.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 473 kB/s | 59 kB 00:00 2023-10-17T01:02:49.990 INFO:teuthology.orchestra.run.smithi060.stdout:(31/131): ceph-mgr-modules-core-18.2.0-668.gab9 3.3 MB/s | 242 kB 00:00 2023-10-17T01:02:50.049 INFO:teuthology.orchestra.run.smithi060.stdout:(32/131): ceph-mgr-rook-18.2.0-668.gab9fe435.el 813 kB/s | 47 kB 00:00 2023-10-17T01:02:50.099 INFO:teuthology.orchestra.run.smithi060.stdout:(33/131): ceph-prometheus-alerts-18.2.0-668.gab 291 kB/s | 15 kB 00:00 2023-10-17T01:02:50.158 INFO:teuthology.orchestra.run.smithi060.stdout:(34/131): ceph-volume-18.2.0-668.gab9fe435.el8. 4.4 MB/s | 261 kB 00:00 2023-10-17T01:02:50.216 INFO:teuthology.orchestra.run.smithi060.stdout:(35/131): cephadm-18.2.0-668.gab9fe435.el8.noar 2.8 MB/s | 167 kB 00:00 2023-10-17T01:02:50.266 INFO:teuthology.orchestra.run.smithi060.stdout:(36/131): cephfs-top-18.2.0-668.gab9fe435.el8.n 360 kB/s | 18 kB 00:00 2023-10-17T01:02:50.556 INFO:teuthology.orchestra.run.smithi045.stdout:(39/131): ceph-test-18.2.0-668.gab9fe435.el8.x8 4.3 MB/s | 47 MB 00:10 2023-10-17T01:02:50.573 INFO:teuthology.orchestra.run.smithi045.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 127 kB/s | 75 kB 00:00 2023-10-17T01:02:50.834 INFO:teuthology.orchestra.run.smithi060.stdout:(37/131): librgw2-18.2.0-668.gab9fe435.el8.x86_ 913 kB/s | 6.5 MB 00:07 2023-10-17T01:02:50.925 INFO:teuthology.orchestra.run.smithi060.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 465 kB/s | 306 kB 00:00 2023-10-17T01:02:51.068 INFO:teuthology.orchestra.run.smithi060.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 549 kB/s | 75 kB 00:00 2023-10-17T01:02:51.148 INFO:teuthology.orchestra.run.smithi045.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 76 kB/s | 44 kB 00:00 2023-10-17T01:02:51.305 INFO:teuthology.orchestra.run.smithi045.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 636 kB/s | 476 kB 00:00 2023-10-17T01:02:51.415 INFO:teuthology.orchestra.run.smithi045.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 299 kB/s | 33 kB 00:00 2023-10-17T01:02:51.434 INFO:teuthology.orchestra.run.smithi060.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 98 kB/s | 59 kB 00:00 2023-10-17T01:02:51.441 INFO:teuthology.orchestra.run.smithi045.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 634 kB/s | 185 kB 00:00 2023-10-17T01:02:51.460 INFO:teuthology.orchestra.run.smithi060.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-10-17T01:02:51.524 INFO:teuthology.orchestra.run.smithi045.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-10-17T01:02:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 647 kB/s | 173 kB 00:00 2023-10-17T01:02:51.727 INFO:teuthology.orchestra.run.smithi060.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 149 kB/s | 44 kB 00:00 2023-10-17T01:02:51.892 INFO:teuthology.orchestra.run.smithi045.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 444 kB/s | 81 kB 00:00 2023-10-17T01:02:51.935 INFO:teuthology.orchestra.run.smithi060.stdout:(43/131): ceph-osd-18.2.0-668.gab9fe435.el8.x86 1.3 MB/s | 18 MB 00:13 2023-10-17T01:02:51.961 INFO:teuthology.orchestra.run.smithi060.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 141 kB/s | 33 kB 00:00 2023-10-17T01:02:51.986 INFO:teuthology.orchestra.run.smithi060.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 352 kB/s | 185 kB 00:00 2023-10-17T01:02:52.008 INFO:teuthology.orchestra.run.smithi045.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-10-17T01:02:52.054 INFO:teuthology.orchestra.run.smithi045.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 726 kB/s | 384 kB 00:00 2023-10-17T01:02:52.087 INFO:teuthology.orchestra.run.smithi060.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-10-17T01:02:52.117 INFO:teuthology.orchestra.run.smithi045.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-10-17T01:02:52.218 INFO:teuthology.orchestra.run.smithi045.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 488 kB/s | 49 kB 00:00 2023-10-17T01:02:52.245 INFO:teuthology.orchestra.run.smithi060.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-10-17T01:02:52.279 INFO:teuthology.orchestra.run.smithi060.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 423 kB/s | 81 kB 00:00 2023-10-17T01:02:52.327 INFO:teuthology.orchestra.run.smithi045.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 542 kB/s | 59 kB 00:00 2023-10-17T01:02:52.354 INFO:teuthology.orchestra.run.smithi060.stdout:(49/131): python3-certifi-2018.10.15-7.el8.noar 39 kB/s | 16 kB 00:00 2023-10-17T01:02:52.379 INFO:teuthology.orchestra.run.smithi060.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-10-17T01:02:52.396 INFO:teuthology.orchestra.run.smithi060.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-10-17T01:02:52.460 INFO:teuthology.orchestra.run.smithi045.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 453 kB/s | 60 kB 00:00 2023-10-17T01:02:52.513 INFO:teuthology.orchestra.run.smithi060.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2023-10-17T01:02:52.530 INFO:teuthology.orchestra.run.smithi060.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 440 kB/s | 59 kB 00:00 2023-10-17T01:02:52.630 INFO:teuthology.orchestra.run.smithi060.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2023-10-17T01:02:52.672 INFO:teuthology.orchestra.run.smithi060.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-10-17T01:02:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:(54/131): ceph-osd-18.2.0-668.gab9fe435.el8.x86 1.2 MB/s | 18 MB 00:14 2023-10-17T01:02:52.739 INFO:teuthology.orchestra.run.smithi060.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-10-17T01:02:52.781 INFO:teuthology.orchestra.run.smithi060.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-10-17T01:02:52.873 INFO:teuthology.orchestra.run.smithi060.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-10-17T01:02:52.898 INFO:teuthology.orchestra.run.smithi060.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 550 kB/s | 64 kB 00:00 2023-10-17T01:02:52.902 INFO:teuthology.orchestra.run.smithi045.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 640 kB/s | 283 kB 00:00 2023-10-17T01:02:52.949 INFO:teuthology.orchestra.run.smithi060.stdout:(60/131): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-10-17T01:02:52.974 INFO:teuthology.orchestra.run.smithi060.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-10-17T01:02:53.001 INFO:teuthology.orchestra.run.smithi060.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 237 kB/s | 24 kB 00:00 2023-10-17T01:02:53.011 INFO:teuthology.orchestra.run.smithi045.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 313 kB/s | 34 kB 00:00 2023-10-17T01:02:53.058 INFO:teuthology.orchestra.run.smithi060.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 268 kB/s | 29 kB 00:00 2023-10-17T01:02:53.083 INFO:teuthology.orchestra.run.smithi060.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 252 kB/s | 27 kB 00:00 2023-10-17T01:02:53.134 INFO:teuthology.orchestra.run.smithi060.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-10-17T01:02:53.195 INFO:teuthology.orchestra.run.smithi045.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 32 kB/s | 16 kB 00:00 2023-10-17T01:02:53.200 INFO:teuthology.orchestra.run.smithi060.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-10-17T01:02:53.226 INFO:teuthology.orchestra.run.smithi060.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 430 kB/s | 61 kB 00:00 2023-10-17T01:02:53.260 INFO:teuthology.orchestra.run.smithi060.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 680 kB/s | 85 kB 00:00 2023-10-17T01:02:53.293 INFO:teuthology.orchestra.run.smithi060.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2023-10-17T01:02:53.295 INFO:teuthology.orchestra.run.smithi045.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 693 kB/s | 196 kB 00:00 2023-10-17T01:02:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 195 kB/s | 23 kB 00:00 2023-10-17T01:02:53.377 INFO:teuthology.orchestra.run.smithi060.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 562 kB/s | 47 kB 00:00 2023-10-17T01:02:53.402 INFO:teuthology.orchestra.run.smithi060.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-10-17T01:02:53.404 INFO:teuthology.orchestra.run.smithi045.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 173 kB/s | 19 kB 00:00 2023-10-17T01:02:53.427 INFO:teuthology.orchestra.run.smithi060.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-10-17T01:02:53.453 INFO:teuthology.orchestra.run.smithi060.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-17T01:02:53.454 INFO:teuthology.orchestra.run.smithi045.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 248 kB/s | 64 kB 00:00 2023-10-17T01:02:53.478 INFO:teuthology.orchestra.run.smithi060.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 951 kB/s | 48 kB 00:00 2023-10-17T01:02:53.504 INFO:teuthology.orchestra.run.smithi060.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-17T01:02:53.512 INFO:teuthology.orchestra.run.smithi045.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 221 kB/s | 24 kB 00:00 2023-10-17T01:02:53.521 INFO:teuthology.orchestra.run.smithi060.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-10-17T01:02:53.546 INFO:teuthology.orchestra.run.smithi060.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.5 MB/s | 109 kB 00:00 2023-10-17T01:02:53.588 INFO:teuthology.orchestra.run.smithi060.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2023-10-17T01:02:53.588 INFO:teuthology.orchestra.run.smithi045.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 219 kB/s | 29 kB 00:00 2023-10-17T01:02:53.613 INFO:teuthology.orchestra.run.smithi060.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-10-17T01:02:53.630 INFO:teuthology.orchestra.run.smithi045.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-10-17T01:02:53.663 INFO:teuthology.orchestra.run.smithi060.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.7 MB/s | 560 kB 00:00 2023-10-17T01:02:53.680 INFO:teuthology.orchestra.run.smithi045.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 734 kB/s | 1.2 MB 00:01 2023-10-17T01:02:53.689 INFO:teuthology.orchestra.run.smithi060.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 186 kB/s | 19 kB 00:00 2023-10-17T01:02:53.714 INFO:teuthology.orchestra.run.smithi060.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-10-17T01:02:53.739 INFO:teuthology.orchestra.run.smithi060.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 356 kB/s | 18 kB 00:00 2023-10-17T01:02:53.773 INFO:teuthology.orchestra.run.smithi060.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2023-10-17T01:02:53.807 INFO:teuthology.orchestra.run.smithi060.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-10-17T01:02:53.822 INFO:teuthology.orchestra.run.smithi045.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 432 kB/s | 61 kB 00:00 2023-10-17T01:02:53.832 INFO:teuthology.orchestra.run.smithi060.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-10-17T01:02:53.857 INFO:teuthology.orchestra.run.smithi060.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-10-17T01:02:53.882 INFO:teuthology.orchestra.run.smithi060.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-17T01:02:53.908 INFO:teuthology.orchestra.run.smithi060.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-10-17T01:02:53.933 INFO:teuthology.orchestra.run.smithi060.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-17T01:02:53.958 INFO:teuthology.orchestra.run.smithi060.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-17T01:02:53.983 INFO:teuthology.orchestra.run.smithi060.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-10-17T01:02:54.022 INFO:teuthology.orchestra.run.smithi045.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 588 kB/s | 255 kB 00:00 2023-10-17T01:02:54.025 INFO:teuthology.orchestra.run.smithi060.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-10-17T01:02:54.047 INFO:teuthology.orchestra.run.smithi045.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 380 kB/s | 85 kB 00:00 2023-10-17T01:02:54.051 INFO:teuthology.orchestra.run.smithi060.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-10-17T01:02:54.073 INFO:teuthology.orchestra.run.smithi045.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 567 kB/s | 251 kB 00:00 2023-10-17T01:02:54.076 INFO:teuthology.orchestra.run.smithi060.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-17T01:02:54.089 INFO:teuthology.orchestra.run.smithi045.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.2 MB/s | 70 kB 00:00 2023-10-17T01:02:54.143 INFO:teuthology.orchestra.run.smithi060.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2023-10-17T01:02:54.168 INFO:teuthology.orchestra.run.smithi060.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-10-17T01:02:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-17T01:02:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 153 kB/s | 23 kB 00:00 2023-10-17T01:02:54.227 INFO:teuthology.orchestra.run.smithi060.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-17T01:02:54.261 INFO:teuthology.orchestra.run.smithi060.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-17T01:02:54.265 INFO:teuthology.orchestra.run.smithi045.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-10-17T01:02:54.286 INFO:teuthology.orchestra.run.smithi060.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 661 kB/s | 39 kB 00:00 2023-10-17T01:02:54.332 INFO:teuthology.orchestra.run.smithi045.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 813 kB/s | 54 kB 00:00 2023-10-17T01:02:54.345 INFO:teuthology.orchestra.run.smithi060.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.0 MB/s | 688 kB 00:00 2023-10-17T01:02:54.382 INFO:teuthology.orchestra.run.smithi045.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-10-17T01:02:54.387 INFO:teuthology.orchestra.run.smithi060.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.2 MB/s | 538 kB 00:00 2023-10-17T01:02:54.412 INFO:teuthology.orchestra.run.smithi060.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-10-17T01:02:54.438 INFO:teuthology.orchestra.run.smithi060.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-17T01:02:54.441 INFO:teuthology.orchestra.run.smithi045.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-10-17T01:02:54.483 INFO:teuthology.orchestra.run.smithi045.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 937 kB/s | 39 kB 00:00 2023-10-17T01:02:54.488 INFO:teuthology.orchestra.run.smithi060.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-10-17T01:02:54.530 INFO:teuthology.orchestra.run.smithi060.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-10-17T01:02:54.541 INFO:teuthology.orchestra.run.smithi045.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-10-17T01:02:54.580 INFO:teuthology.orchestra.run.smithi060.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-10-17T01:02:54.600 INFO:teuthology.orchestra.run.smithi045.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 9.0 MB/s | 538 kB 00:00 2023-10-17T01:02:54.614 INFO:teuthology.orchestra.run.smithi060.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-10-17T01:02:54.642 INFO:teuthology.orchestra.run.smithi045.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-10-17T01:02:54.706 INFO:teuthology.orchestra.run.smithi060.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-10-17T01:02:54.759 INFO:teuthology.orchestra.run.smithi045.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-10-17T01:02:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-10-17T01:02:54.807 INFO:teuthology.orchestra.run.smithi060.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-10-17T01:02:54.874 INFO:teuthology.orchestra.run.smithi060.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-10-17T01:02:54.899 INFO:teuthology.orchestra.run.smithi060.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-10-17T01:02:54.918 INFO:teuthology.orchestra.run.smithi045.stdout:(80/131): python3-scipy-1.0.0-21.module+el8.5.0 17 MB/s | 14 MB 00:00 2023-10-17T01:02:54.924 INFO:teuthology.orchestra.run.smithi060.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.4 MB/s | 126 kB 00:00 2023-10-17T01:02:54.949 INFO:teuthology.orchestra.run.smithi060.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 396 kB/s | 20 kB 00:00 2023-10-17T01:02:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 3.5 MB/s | 713 kB 00:00 2023-10-17T01:02:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-17T01:02:55.010 INFO:teuthology.orchestra.run.smithi045.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 533 kB/s | 27 kB 00:00 2023-10-17T01:02:55.016 INFO:teuthology.orchestra.run.smithi060.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-17T01:02:55.068 INFO:teuthology.orchestra.run.smithi045.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-10-17T01:02:55.135 INFO:teuthology.orchestra.run.smithi045.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 662 kB/s | 44 kB 00:00 2023-10-17T01:02:55.167 INFO:teuthology.orchestra.run.smithi060.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 12 MB/s | 14 MB 00:01 2023-10-17T01:02:55.192 INFO:teuthology.orchestra.run.smithi060.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-10-17T01:02:55.228 INFO:teuthology.orchestra.run.smithi045.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-10-17T01:02:55.234 INFO:teuthology.orchestra.run.smithi060.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-17T01:02:55.284 INFO:teuthology.orchestra.run.smithi060.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-10-17T01:02:55.286 INFO:teuthology.orchestra.run.smithi045.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-10-17T01:02:55.336 INFO:teuthology.orchestra.run.smithi045.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-10-17T01:02:55.376 INFO:teuthology.orchestra.run.smithi060.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-10-17T01:02:55.403 INFO:teuthology.orchestra.run.smithi045.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-10-17T01:02:55.428 INFO:teuthology.orchestra.run.smithi045.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-10-17T01:02:55.482 INFO:teuthology.orchestra.run.smithi045.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.6 MB/s | 126 kB 00:00 2023-10-17T01:02:55.485 INFO:teuthology.orchestra.run.smithi060.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-10-17T01:02:55.504 INFO:teuthology.orchestra.run.smithi045.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 266 kB/s | 20 kB 00:00 2023-10-17T01:02:55.527 INFO:teuthology.orchestra.run.smithi060.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-10-17T01:02:55.529 INFO:teuthology.orchestra.run.smithi045.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.2 MB/s | 103 kB 00:00 2023-10-17T01:02:55.571 INFO:teuthology.orchestra.run.smithi045.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-17T01:02:55.585 INFO:teuthology.orchestra.run.smithi060.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-10-17T01:02:55.622 INFO:teuthology.orchestra.run.smithi045.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-10-17T01:02:55.669 INFO:teuthology.orchestra.run.smithi060.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-10-17T01:02:55.719 INFO:teuthology.orchestra.run.smithi060.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-17T01:02:55.756 INFO:teuthology.orchestra.run.smithi045.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-10-17T01:02:55.797 INFO:teuthology.orchestra.run.smithi045.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-17T01:02:55.906 INFO:teuthology.orchestra.run.smithi045.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-10-17T01:02:55.931 INFO:teuthology.orchestra.run.smithi045.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-10-17T01:02:55.990 INFO:teuthology.orchestra.run.smithi045.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-10-17T01:02:56.073 INFO:teuthology.orchestra.run.smithi045.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-10-17T01:02:56.157 INFO:teuthology.orchestra.run.smithi045.stdout:(101/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-10-17T01:02:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 432 kB/s | 47 kB 00:00 2023-10-17T01:02:56.207 INFO:teuthology.orchestra.run.smithi045.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-10-17T01:02:56.258 INFO:teuthology.orchestra.run.smithi045.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 566 kB/s | 43 kB 00:00 2023-10-17T01:02:56.283 INFO:teuthology.orchestra.run.smithi045.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2023-10-17T01:02:56.325 INFO:teuthology.orchestra.run.smithi045.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 719 kB/s | 48 kB 00:00 2023-10-17T01:02:56.350 INFO:teuthology.orchestra.run.smithi045.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-10-17T01:02:56.375 INFO:teuthology.orchestra.run.smithi045.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-10-17T01:02:56.401 INFO:teuthology.orchestra.run.smithi045.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-17T01:02:56.426 INFO:teuthology.orchestra.run.smithi045.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-10-17T01:02:56.476 INFO:teuthology.orchestra.run.smithi045.stdout:(111/131): thrift-0.13.0-2.el8.x86_64.rpm 731 kB/s | 1.7 MB 00:02 2023-10-17T01:02:56.502 INFO:teuthology.orchestra.run.smithi045.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 248 kB/s | 19 kB 00:00 2023-10-17T01:02:56.544 INFO:teuthology.orchestra.run.smithi045.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-10-17T01:02:56.577 INFO:teuthology.orchestra.run.smithi045.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.5 MB/s | 155 kB 00:00 2023-10-17T01:02:56.628 INFO:teuthology.orchestra.run.smithi045.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 214 kB/s | 18 kB 00:00 2023-10-17T01:02:56.670 INFO:teuthology.orchestra.run.smithi045.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.4 MB/s | 559 kB 00:00 2023-10-17T01:02:56.712 INFO:teuthology.orchestra.run.smithi045.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-10-17T01:02:56.737 INFO:teuthology.orchestra.run.smithi045.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-10-17T01:02:56.744 INFO:teuthology.orchestra.run.smithi060.stdout:(130/131): librados2-18.2.0-668.gab9fe435.el8.x 3.0 MB/s | 3.7 MB 00:01 2023-10-17T01:02:56.762 INFO:teuthology.orchestra.run.smithi045.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-10-17T01:02:56.788 INFO:teuthology.orchestra.run.smithi045.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-17T01:02:56.813 INFO:teuthology.orchestra.run.smithi045.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-17T01:02:56.838 INFO:teuthology.orchestra.run.smithi045.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-17T01:02:56.863 INFO:teuthology.orchestra.run.smithi045.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-17T01:02:56.889 INFO:teuthology.orchestra.run.smithi045.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 765 kB/s | 39 kB 00:00 2023-10-17T01:02:56.931 INFO:teuthology.orchestra.run.smithi045.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-10-17T01:02:56.956 INFO:teuthology.orchestra.run.smithi045.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-10-17T01:02:56.973 INFO:teuthology.orchestra.run.smithi045.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-10-17T01:02:57.048 INFO:teuthology.orchestra.run.smithi045.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.2 MB/s | 2.3 MB 00:00 2023-10-17T01:02:57.115 INFO:teuthology.orchestra.run.smithi045.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-10-17T01:02:57.234 INFO:teuthology.orchestra.run.smithi181.stdout:(131/131): ceph-test-18.2.0-668.gab9fe435.el8.x 1.9 MB/s | 47 MB 00:24 2023-10-17T01:02:57.238 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:02:57.239 INFO:teuthology.orchestra.run.smithi181.stdout:Total 7.5 MB/s | 203 MB 00:27 2023-10-17T01:02:57.295 INFO:teuthology.orchestra.run.smithi060.stdout:(131/131): librbd1-18.2.0-668.gab9fe435.el8.x86 2.1 MB/s | 3.6 MB 00:01 2023-10-17T01:02:57.299 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:02:57.300 INFO:teuthology.orchestra.run.smithi060.stdout:Total 8.6 MB/s | 203 MB 00:23 2023-10-17T01:02:57.332 INFO:teuthology.orchestra.run.smithi045.stdout:(130/131): librbd1-18.2.0-668.gab9fe435.el8.x86 9.9 MB/s | 3.6 MB 00:00 2023-10-17T01:02:58.086 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-10-17T01:02:58.157 INFO:teuthology.orchestra.run.smithi045.stdout:(131/131): librados2-18.2.0-668.gab9fe435.el8.x 3.1 MB/s | 3.7 MB 00:01 2023-10-17T01:02:58.163 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:02:58.163 INFO:teuthology.orchestra.run.smithi045.stdout:Total 7.6 MB/s | 203 MB 00:26 2023-10-17T01:02:58.185 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-10-17T01:02:58.222 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-10-17T01:02:58.222 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-10-17T01:02:58.319 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-10-17T01:02:58.319 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-10-17T01:02:59.305 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-10-17T01:02:59.309 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-10-17T01:02:59.489 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-10-17T01:02:59.494 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-10-17T01:02:59.892 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-10-17T01:03:00.040 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-10-17T01:03:00.041 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-10-17T01:03:01.617 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-10-17T01:03:01.622 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-10-17T01:03:04.222 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-10-17T01:03:04.379 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-17T01:03:04.532 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-17T01:03:04.738 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-17T01:03:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-10-17T01:03:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-17T01:03:04.931 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-17T01:03:05.045 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-17T01:03:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-17T01:03:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-17T01:03:05.281 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-17T01:03:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-17T01:03:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-17T01:03:05.688 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-17T01:03:05.906 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:03:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-17T01:03:06.222 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:18.2.0-668.gab9fe435.el8.x86_64 5/134 2023-10-17T01:03:06.288 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:03:06.456 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:03:06.583 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:03:06.646 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:18.2.0-668.gab9fe435.el8.x86_64 5/134 2023-10-17T01:03:06.765 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:03:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:03:06.824 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-10-17T01:03:06.887 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:03:06.908 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:03:07.039 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-17T01:03:07.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:03:07.080 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 9/134 2023-10-17T01:03:07.162 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:03:07.197 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-17T01:03:07.292 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-17T01:03:07.311 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:03:07.410 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-17T01:03:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-17T01:03:07.509 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-17T01:03:07.574 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-17T01:03:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-17T01:03:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-17T01:03:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-17T01:03:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-17T01:03:07.949 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-17T01:03:08.160 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-17T01:03:08.298 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 14/134 2023-10-17T01:03:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-17T01:03:08.385 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 14/134 2023-10-17T01:03:08.463 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-17T01:03:08.510 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-17T01:03:08.547 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 15/134 2023-10-17T01:03:08.618 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 15/134 2023-10-17T01:03:08.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 14/134 2023-10-17T01:03:08.682 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:03:08.711 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 14/134 2023-10-17T01:03:08.789 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-17T01:03:08.847 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 15/134 2023-10-17T01:03:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-17T01:03:08.999 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 15/134 2023-10-17T01:03:09.089 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-17T01:03:09.182 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:03:09.201 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-17T01:03:09.348 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:03:09.362 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-17T01:03:09.502 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:03:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-17T01:03:09.638 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-17T01:03:09.699 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:03:09.701 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-17T01:03:09.767 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-17T01:03:09.804 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:03:09.897 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-17T01:03:09.906 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-17T01:03:10.030 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 9/134 2023-10-17T01:03:10.284 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-17T01:03:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-17T01:03:10.513 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-17T01:03:10.539 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-17T01:03:10.689 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-17T01:03:10.730 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-17T01:03:10.913 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-17T01:03:11.162 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-17T01:03:11.298 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 14/134 2023-10-17T01:03:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-17T01:03:11.432 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-17T01:03:11.436 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 14/134 2023-10-17T01:03:11.488 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-17T01:03:11.581 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 15/134 2023-10-17T01:03:11.710 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 15/134 2023-10-17T01:03:11.737 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-17T01:03:11.865 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-17T01:03:11.948 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-17T01:03:12.028 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-17T01:03:12.159 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-17T01:03:12.226 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-17T01:03:12.227 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-17T01:03:12.378 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-17T01:03:12.462 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-17T01:03:12.577 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-17T01:03:12.658 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-17T01:03:12.815 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-17T01:03:12.869 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-17T01:03:12.945 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x 26/134 2023-10-17T01:03:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-17T01:03:13.020 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x 26/134 2023-10-17T01:03:13.057 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8 27/134 2023-10-17T01:03:13.123 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-17T01:03:13.133 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8 27/134 2023-10-17T01:03:13.341 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 28/134 2023-10-17T01:03:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-17T01:03:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-17T01:03:13.975 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 28/134 2023-10-17T01:03:14.031 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-17T01:03:14.294 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-17T01:03:14.607 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-17T01:03:14.873 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-17T01:03:14.943 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-17T01:03:15.071 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-17T01:03:15.214 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-17T01:03:15.466 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-17T01:03:15.495 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-17T01:03:15.738 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-17T01:03:15.925 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-17T01:03:16.188 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-17T01:03:16.437 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-17T01:03:16.530 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-17T01:03:16.722 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x 26/134 2023-10-17T01:03:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-17T01:03:16.829 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-17T01:03:16.886 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8 27/134 2023-10-17T01:03:17.086 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-17T01:03:17.131 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-17T01:03:17.287 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-17T01:03:17.350 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-17T01:03:17.485 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-17T01:03:17.541 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-17T01:03:17.672 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-17T01:03:17.743 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 36/134 2023-10-17T01:03:17.917 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 37/134 2023-10-17T01:03:17.937 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 28/134 2023-10-17T01:03:18.030 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 36/134 2023-10-17T01:03:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-17T01:03:18.273 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-17T01:03:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-17T01:03:18.287 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 37/134 2023-10-17T01:03:18.429 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-17T01:03:18.482 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-17T01:03:18.566 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-17T01:03:18.600 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-17T01:03:18.633 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-17T01:03:18.746 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-17T01:03:18.812 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-17T01:03:18.846 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-17T01:03:19.028 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-17T01:03:19.031 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-17T01:03:19.041 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-17T01:03:19.218 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-17T01:03:19.257 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-17T01:03:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-17T01:03:19.389 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-17T01:03:19.465 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-17T01:03:19.646 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-17T01:03:19.666 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-17T01:03:19.792 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-17T01:03:19.856 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-17T01:03:19.938 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-17T01:03:20.075 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-17T01:03:20.099 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-17T01:03:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-17T01:03:20.342 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-17T01:03:20.393 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-17T01:03:20.572 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-17T01:03:20.645 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-17T01:03:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-17T01:03:20.854 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-17T01:03:21.022 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-17T01:03:21.125 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-17T01:03:21.249 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-17T01:03:21.407 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-17T01:03:21.424 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-17T01:03:21.429 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-17T01:03:21.582 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-17T01:03:21.633 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-17T01:03:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-17T01:03:21.841 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-17T01:03:21.868 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-17T01:03:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-17T01:03:22.061 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-17T01:03:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-17T01:03:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-17T01:03:22.374 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-17T01:03:22.392 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-17T01:03:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 36/134 2023-10-17T01:03:22.579 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-17T01:03:22.627 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-17T01:03:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 37/134 2023-10-17T01:03:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-17T01:03:22.769 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-17T01:03:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-17T01:03:22.913 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-17T01:03:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-17T01:03:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-17T01:03:23.070 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-17T01:03:23.127 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-17T01:03:23.167 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-17T01:03:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-17T01:03:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-17T01:03:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-17T01:03:23.423 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-17T01:03:23.598 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-17T01:03:23.623 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-17T01:03:23.624 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-17T01:03:23.779 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-17T01:03:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-17T01:03:23.854 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-17T01:03:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-17T01:03:24.088 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-17T01:03:24.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-17T01:03:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-17T01:03:24.289 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-17T01:03:24.297 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-17T01:03:24.429 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-17T01:03:24.454 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-17T01:03:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-17T01:03:24.625 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-17T01:03:24.664 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-17T01:03:24.838 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-17T01:03:24.845 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-17T01:03:24.871 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-17T01:03:25.069 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-17T01:03:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-17T01:03:25.105 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-17T01:03:25.200 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-17T01:03:25.216 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-17T01:03:25.357 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-17T01:03:25.365 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-17T01:03:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-17T01:03:25.534 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-17T01:03:25.573 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-17T01:03:25.716 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-17T01:03:25.770 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-17T01:03:25.806 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-17T01:03:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-17T01:03:25.949 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-17T01:03:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-17T01:03:26.065 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-17T01:03:26.148 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-17T01:03:26.185 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-17T01:03:26.194 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-17T01:03:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-17T01:03:26.396 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-17T01:03:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-17T01:03:26.559 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-17T01:03:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-17T01:03:26.617 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-17T01:03:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-17T01:03:26.791 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-17T01:03:26.826 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-17T01:03:27.001 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-17T01:03:27.057 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-17T01:03:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-17T01:03:27.234 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-17T01:03:27.253 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-17T01:03:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-17T01:03:27.412 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-17T01:03:27.461 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-17T01:03:27.617 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-17T01:03:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-17T01:03:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-17T01:03:27.834 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-17T01:03:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-17T01:03:28.080 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-17T01:03:28.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-17T01:03:28.462 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-17T01:03:28.567 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-17T01:03:28.777 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-17T01:03:28.809 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-17T01:03:28.969 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-17T01:03:28.986 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-17T01:03:29.034 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-17T01:03:29.171 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-17T01:03:29.240 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-17T01:03:29.270 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-17T01:03:29.288 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-17T01:03:29.469 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-17T01:03:29.477 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-17T01:03:29.592 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-17T01:03:29.674 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-17T01:03:29.788 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-17T01:03:29.839 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-17T01:03:29.912 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-17T01:03:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-17T01:03:30.085 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-17T01:03:30.101 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-17T01:03:30.286 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-17T01:03:30.309 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-17T01:03:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-17T01:03:30.726 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-17T01:03:30.859 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-17T01:03:30.888 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-17T01:03:31.030 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-17T01:03:31.035 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 92/134 2023-10-17T01:03:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-17T01:03:31.097 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 92/134 2023-10-17T01:03:31.194 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 92/134 2023-10-17T01:03:31.239 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-17T01:03:31.263 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 92/134 2023-10-17T01:03:31.270 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 93/134 2023-10-17T01:03:31.422 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-17T01:03:31.478 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 93/134 2023-10-17T01:03:31.599 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-17T01:03:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-17T01:03:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-17T01:03:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-17T01:03:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-17T01:03:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-17T01:03:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-17T01:03:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-17T01:03:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-17T01:03:34.718 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-17T01:03:35.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-17T01:03:35.109 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:35.153 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:35.235 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-17T01:03:35.344 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:35.352 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:35.600 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-17T01:03:35.938 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-17T01:03:35.951 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:35.984 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:36.153 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-17T01:03:36.199 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 95/134 2023-10-17T01:03:36.216 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 95/134 2023-10-17T01:03:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 95/134 2023-10-17T01:03:36.298 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 95/134 2023-10-17T01:03:36.426 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 96/134 2023-10-17T01:03:36.435 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 96/134 2023-10-17T01:03:36.962 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-17T01:03:37.152 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 92/134 2023-10-17T01:03:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 92/134 2023-10-17T01:03:37.403 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 93/134 2023-10-17T01:03:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:43.552 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 94/134 2023-10-17T01:03:43.790 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 95/134 2023-10-17T01:03:43.948 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 95/134 2023-10-17T01:03:44.052 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 96/134 2023-10-17T01:03:55.246 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 96/134 2023-10-17T01:03:55.247 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-10-17T01:03:55.247 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-10-17T01:03:55.247 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-10-17T01:03:55.247 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-10-17T01:03:55.248 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-10-17T01:03:55.248 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-10-17T01:03:55.248 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-10-17T01:03:55.248 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-10-17T01:03:55.248 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-10-17T01:03:55.249 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:03:55.582 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 97/134 2023-10-17T01:03:55.964 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 96/134 2023-10-17T01:03:55.964 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2023-10-17T01:03:55.965 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2023-10-17T01:03:55.965 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /dev 2023-10-17T01:03:55.965 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /run 2023-10-17T01:03:55.965 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2023-10-17T01:03:55.965 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2023-10-17T01:03:55.966 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2023-10-17T01:03:55.966 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2023-10-17T01:03:55.966 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /dev 2023-10-17T01:03:55.966 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:03:56.300 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 97/134 2023-10-17T01:03:57.158 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 97/134 2023-10-17T01:03:57.329 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 98/134 2023-10-17T01:03:57.547 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 98/134 2023-10-17T01:03:57.715 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 99/134 2023-10-17T01:03:57.972 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 97/134 2023-10-17T01:03:58.129 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 98/134 2023-10-17T01:03:58.186 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 99/134 2023-10-17T01:03:58.323 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 100/134 2023-10-17T01:03:58.341 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 100/134 2023-10-17T01:03:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 98/134 2023-10-17T01:03:58.516 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 99/134 2023-10-17T01:03:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 99/134 2023-10-17T01:03:59.149 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 100/134 2023-10-17T01:03:59.167 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 100/134 2023-10-17T01:03:59.993 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.2.0-668.gab9fe435.el8.noarch 101/134 2023-10-17T01:04:00.181 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.2.0-668.gab9fe435.el8.noarch 101/134 2023-10-17T01:04:00.295 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el 102/134 2023-10-17T01:04:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.e 103/134 2023-10-17T01:04:00.681 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 104/134 2023-10-17T01:04:00.902 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 104/134 2023-10-17T01:04:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.2.0-668.gab9fe435.el8.noarch 101/134 2023-10-17T01:04:01.157 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:18.2.0-668.gab9fe435.el8.noarch 101/134 2023-10-17T01:04:01.178 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 105/134 2023-10-17T01:04:01.272 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el 102/134 2023-10-17T01:04:01.460 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.e 103/134 2023-10-17T01:04:01.615 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 104/134 2023-10-17T01:04:01.845 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 104/134 2023-10-17T01:04:02.091 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 105/134 2023-10-17T01:04:02.142 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 105/134 2023-10-17T01:04:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 106/134 2023-10-17T01:04:02.414 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 106/134 2023-10-17T01:04:02.772 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8 107/134 2023-10-17T01:04:02.958 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 108/134 2023-10-17T01:04:02.993 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 108/134 2023-10-17T01:04:03.116 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 105/134 2023-10-17T01:04:03.156 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 109/134 2023-10-17T01:04:03.167 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 109/134 2023-10-17T01:04:03.315 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 106/134 2023-10-17T01:04:03.413 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 106/134 2023-10-17T01:04:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8 107/134 2023-10-17T01:04:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 108/134 2023-10-17T01:04:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 108/134 2023-10-17T01:04:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 109/134 2023-10-17T01:04:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 109/134 2023-10-17T01:04:04.425 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.2.0-668.gab9fe435.el8.x86_64 110/134 2023-10-17T01:04:04.656 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 111/134 2023-10-17T01:04:04.689 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 111/134 2023-10-17T01:04:04.831 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 112/134 2023-10-17T01:04:04.872 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 112/134 2023-10-17T01:04:05.022 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 113/134 2023-10-17T01:04:05.367 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 113/134 2023-10-17T01:04:05.398 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 96/134 2023-10-17T01:04:05.398 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2023-10-17T01:04:05.398 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2023-10-17T01:04:05.398 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-10-17T01:04:05.399 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /run 2023-10-17T01:04:05.399 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2023-10-17T01:04:05.399 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2023-10-17T01:04:05.399 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2023-10-17T01:04:05.399 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2023-10-17T01:04:05.400 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-10-17T01:04:05.400 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:04:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:18.2.0-668.gab9fe435.el8.x86_64 110/134 2023-10-17T01:04:05.564 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 114/134 2023-10-17T01:04:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 111/134 2023-10-17T01:04:05.710 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 111/134 2023-10-17T01:04:05.782 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 97/134 2023-10-17T01:04:05.857 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 112/134 2023-10-17T01:04:05.900 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 112/134 2023-10-17T01:04:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 113/134 2023-10-17T01:04:06.398 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 113/134 2023-10-17T01:04:06.557 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 114/134 2023-10-17T01:04:07.606 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 97/134 2023-10-17T01:04:07.819 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 98/134 2023-10-17T01:04:08.100 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 98/134 2023-10-17T01:04:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 99/134 2023-10-17T01:04:08.856 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 99/134 2023-10-17T01:04:09.040 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 100/134 2023-10-17T01:04:09.083 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 100/134 2023-10-17T01:04:09.983 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 114/134 2023-10-17T01:04:10.255 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 115/134 2023-10-17T01:04:10.421 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-17T01:04:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-17T01:04:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 114/134 2023-10-17T01:04:11.510 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-17T01:04:11.826 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-17T01:04:12.006 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:18.2.0-668.gab9fe435.el8.noarch 101/134 2023-10-17T01:04:12.015 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 119/134 2023-10-17T01:04:12.197 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:18.2.0-668.gab9fe435.el8.noarch 101/134 2023-10-17T01:04:12.233 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-17T01:04:12.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el 102/134 2023-10-17T01:04:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 115/134 2023-10-17T01:04:12.400 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_6 121/134 2023-10-17T01:04:12.480 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.e 103/134 2023-10-17T01:04:12.564 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-17T01:04:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 104/134 2023-10-17T01:04:12.776 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-17T01:04:12.934 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 117/134 2023-10-17T01:04:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 104/134 2023-10-17T01:04:13.173 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-17T01:04:13.210 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 105/134 2023-10-17T01:04:13.297 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-17T01:04:13.504 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-17T01:04:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-17T01:04:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_6 121/134 2023-10-17T01:04:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-17T01:04:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 105/134 2023-10-17T01:04:14.572 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 106/134 2023-10-17T01:04:14.706 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 106/134 2023-10-17T01:04:14.799 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:15.101 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8 107/134 2023-10-17T01:04:15.101 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:15.120 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:15.327 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 124/134 2023-10-17T01:04:15.352 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 108/134 2023-10-17T01:04:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 108/134 2023-10-17T01:04:15.561 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 125/134 2023-10-17T01:04:15.589 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 109/134 2023-10-17T01:04:15.618 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 109/134 2023-10-17T01:04:15.865 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 126/134 2023-10-17T01:04:16.181 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-17T01:04:16.320 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-17T01:04:16.330 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-17T01:04:16.385 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:16.509 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-17T01:04:16.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:16.695 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-17T01:04:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 124/134 2023-10-17T01:04:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-17T01:04:16.970 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-17T01:04:16.984 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-17T01:04:16.984 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:17.048 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:18.2.0-668.gab9fe435.el8.x86_64 110/134 2023-10-17T01:04:17.106 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 125/134 2023-10-17T01:04:17.198 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:17.198 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-17T01:04:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 111/134 2023-10-17T01:04:17.363 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 111/134 2023-10-17T01:04:17.405 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 126/134 2023-10-17T01:04:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-17T01:04:17.547 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 112/134 2023-10-17T01:04:17.620 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 112/134 2023-10-17T01:04:17.678 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:17.720 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-17T01:04:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 113/134 2023-10-17T01:04:17.905 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-17T01:04:18.002 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-17T01:04:18.174 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.2.0-668.gab9fe435.el8.x86_64 1/134 2023-10-17T01:04:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 2/134 2023-10-17T01:04:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 3/134 2023-10-17T01:04:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 4/134 2023-10-17T01:04:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 5/134 2023-10-17T01:04:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:04:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:04:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:04:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 9/134 2023-10-17T01:04:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 10/134 2023-10-17T01:04:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 11/134 2023-10-17T01:04:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 12/134 2023-10-17T01:04:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 13/134 2023-10-17T01:04:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_6 14/134 2023-10-17T01:04:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 15/134 2023-10-17T01:04:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 16/134 2023-10-17T01:04:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 17/134 2023-10-17T01:04:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 18/134 2023-10-17T01:04:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 19/134 2023-10-17T01:04:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8 20/134 2023-10-17T01:04:18.179 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x 21/134 2023-10-17T01:04:18.179 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 22/134 2023-10-17T01:04:18.179 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 23/134 2023-10-17T01:04:18.179 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 24/134 2023-10-17T01:04:18.180 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 25/134 2023-10-17T01:04:18.180 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 26/134 2023-10-17T01:04:18.180 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 27/134 2023-10-17T01:04:18.180 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 28/134 2023-10-17T01:04:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.e 29/134 2023-10-17T01:04:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 30/134 2023-10-17T01:04:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 31/134 2023-10-17T01:04:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 32/134 2023-10-17T01:04:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8 33/134 2023-10-17T01:04:18.182 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 34/134 2023-10-17T01:04:18.182 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el 35/134 2023-10-17T01:04:18.182 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 36/134 2023-10-17T01:04:18.182 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.2.0-668.gab9fe435.el8.noarch 37/134 2023-10-17T01:04:18.183 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 38/134 2023-10-17T01:04:18.183 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-17T01:04:18.183 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-17T01:04:18.183 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-17T01:04:18.183 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-17T01:04:18.184 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-17T01:04:18.184 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-17T01:04:18.184 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-17T01:04:18.184 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-17T01:04:18.184 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-17T01:04:18.185 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-17T01:04:18.185 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-17T01:04:18.185 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-17T01:04:18.185 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-17T01:04:18.186 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-17T01:04:18.186 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-17T01:04:18.186 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-17T01:04:18.186 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-17T01:04:18.186 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-17T01:04:18.187 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-17T01:04:18.187 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-17T01:04:18.187 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-17T01:04:18.187 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-17T01:04:18.188 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-17T01:04:18.188 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-17T01:04:18.188 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-17T01:04:18.188 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-17T01:04:18.188 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-17T01:04:18.189 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-17T01:04:18.189 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-17T01:04:18.189 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-17T01:04:18.189 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-17T01:04:18.189 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-17T01:04:18.190 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-17T01:04:18.190 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-17T01:04:18.190 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-17T01:04:18.190 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-17T01:04:18.191 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-17T01:04:18.191 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-17T01:04:18.191 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-17T01:04:18.191 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-17T01:04:18.191 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-17T01:04:18.191 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-17T01:04:18.192 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-17T01:04:18.192 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-17T01:04:18.192 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-17T01:04:18.192 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-17T01:04:18.192 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-17T01:04:18.193 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-17T01:04:18.193 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-17T01:04:18.193 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-17T01:04:18.193 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-17T01:04:18.193 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-17T01:04:18.194 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-17T01:04:18.194 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-17T01:04:18.194 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-17T01:04:18.194 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-17T01:04:18.195 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-17T01:04:18.195 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-17T01:04:18.195 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-17T01:04:18.195 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-17T01:04:18.195 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-17T01:04:18.196 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-17T01:04:18.196 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-17T01:04:18.196 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-17T01:04:18.196 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-17T01:04:18.197 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-17T01:04:18.197 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-17T01:04:18.197 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-17T01:04:18.197 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-17T01:04:18.197 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-17T01:04:18.198 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-17T01:04:18.198 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-17T01:04:18.198 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-17T01:04:18.198 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-17T01:04:18.199 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-17T01:04:18.199 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-17T01:04:18.199 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-17T01:04:18.199 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-17T01:04:18.200 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-17T01:04:18.200 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-17T01:04:18.200 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-17T01:04:18.200 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-17T01:04:18.200 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-17T01:04:18.201 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-17T01:04:18.201 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-17T01:04:18.201 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-17T01:04:18.201 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-17T01:04:18.201 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-17T01:04:18.202 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-17T01:04:18.202 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-17T01:04:18.202 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.2.0-668.gab9fe435.el8.x86_64 129/134 2023-10-17T01:04:18.202 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-17T01:04:18.203 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 131/134 2023-10-17T01:04:18.203 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:18.203 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-17T01:04:18.205 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-17T01:04:18.207 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 113/134 2023-10-17T01:04:18.218 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-17T01:04:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-17T01:04:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-17T01:04:18.405 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 114/134 2023-10-17T01:04:18.552 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-17T01:04:18.553 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:18.725 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:18.726 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-17T01:04:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-17T01:04:19.138 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:19.683 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:19.683 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:18.2.0-668.gab9fe435.el8.x86_64 1/134 2023-10-17T01:04:19.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 2/134 2023-10-17T01:04:19.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 3/134 2023-10-17T01:04:19.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 4/134 2023-10-17T01:04:19.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 5/134 2023-10-17T01:04:19.684 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:04:19.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:04:19.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:04:19.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 9/134 2023-10-17T01:04:19.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 10/134 2023-10-17T01:04:19.685 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 11/134 2023-10-17T01:04:19.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 12/134 2023-10-17T01:04:19.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 13/134 2023-10-17T01:04:19.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_6 14/134 2023-10-17T01:04:19.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 15/134 2023-10-17T01:04:19.686 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 16/134 2023-10-17T01:04:19.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 17/134 2023-10-17T01:04:19.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 18/134 2023-10-17T01:04:19.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 19/134 2023-10-17T01:04:19.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8 20/134 2023-10-17T01:04:19.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x 21/134 2023-10-17T01:04:19.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 22/134 2023-10-17T01:04:19.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 23/134 2023-10-17T01:04:19.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 24/134 2023-10-17T01:04:19.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 25/134 2023-10-17T01:04:19.688 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 26/134 2023-10-17T01:04:19.689 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 27/134 2023-10-17T01:04:19.689 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 28/134 2023-10-17T01:04:19.689 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.e 29/134 2023-10-17T01:04:19.689 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 30/134 2023-10-17T01:04:19.698 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 31/134 2023-10-17T01:04:19.698 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 32/134 2023-10-17T01:04:19.698 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8 33/134 2023-10-17T01:04:19.698 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 34/134 2023-10-17T01:04:19.698 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el 35/134 2023-10-17T01:04:19.699 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 36/134 2023-10-17T01:04:19.699 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:18.2.0-668.gab9fe435.el8.noarch 37/134 2023-10-17T01:04:19.699 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 38/134 2023-10-17T01:04:19.699 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-17T01:04:19.699 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-17T01:04:19.700 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-17T01:04:19.700 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-17T01:04:19.700 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-17T01:04:19.700 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-17T01:04:19.700 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-17T01:04:19.701 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-17T01:04:19.701 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-17T01:04:19.701 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-17T01:04:19.701 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-17T01:04:19.701 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-17T01:04:19.702 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-17T01:04:19.702 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-17T01:04:19.702 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-17T01:04:19.702 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-17T01:04:19.702 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-17T01:04:19.703 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-17T01:04:19.703 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-17T01:04:19.703 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-17T01:04:19.703 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-17T01:04:19.703 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-17T01:04:19.704 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-17T01:04:19.704 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-17T01:04:19.704 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-17T01:04:19.704 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-17T01:04:19.704 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-17T01:04:19.705 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-17T01:04:19.705 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-17T01:04:19.705 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-17T01:04:19.705 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-17T01:04:19.705 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-17T01:04:19.706 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-17T01:04:19.706 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-17T01:04:19.706 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-17T01:04:19.706 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-17T01:04:19.706 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-17T01:04:19.707 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-17T01:04:19.707 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-17T01:04:19.707 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-17T01:04:19.707 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-17T01:04:19.707 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-17T01:04:19.708 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-17T01:04:19.708 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-17T01:04:19.708 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-17T01:04:19.708 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-17T01:04:19.708 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-17T01:04:19.708 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-17T01:04:19.709 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-17T01:04:19.709 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-17T01:04:19.709 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-17T01:04:19.709 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-17T01:04:19.709 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-17T01:04:19.710 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-17T01:04:19.710 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-17T01:04:19.710 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-17T01:04:19.710 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-17T01:04:19.711 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-17T01:04:19.711 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-17T01:04:19.711 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-17T01:04:19.711 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-17T01:04:19.711 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-17T01:04:19.711 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-17T01:04:19.712 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-17T01:04:19.712 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-17T01:04:19.712 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-17T01:04:19.712 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-17T01:04:19.712 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-17T01:04:19.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-17T01:04:19.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-17T01:04:19.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-17T01:04:19.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-17T01:04:19.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-17T01:04:19.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-17T01:04:19.714 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-17T01:04:19.714 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-17T01:04:19.714 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-17T01:04:19.714 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-17T01:04:19.714 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-17T01:04:19.715 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-17T01:04:19.715 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-17T01:04:19.715 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-17T01:04:19.715 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-17T01:04:19.715 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-17T01:04:19.715 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-17T01:04:19.716 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-17T01:04:19.716 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-17T01:04:19.716 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-17T01:04:19.716 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-17T01:04:19.716 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-17T01:04:19.716 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:18.2.0-668.gab9fe435.el8.x86_64 129/134 2023-10-17T01:04:19.717 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-17T01:04:19.717 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 131/134 2023-10-17T01:04:19.717 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:19.717 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-17T01:04:20.630 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:20.630 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:02 ago on Tue 17 Oct 2023 01:02:18 AM UTC. 2023-10-17T01:04:22.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:22.098 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:01 ago on Tue 17 Oct 2023 01:02:21 AM UTC. 2023-10-17T01:04:23.252 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-10-17T01:04:23.469 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:04:23.470 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-10-17T01:04:23.470 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-17T01:04:23.470 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.470 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.471 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-10-17T01:04:23.471 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-17T01:04:23.471 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-7.el8.noarch 2023-10-17T01:04:23.471 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2023-10-17T01:04:23.471 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.472 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.472 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.472 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.472 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.472 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.473 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.473 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.473 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.473 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.473 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.474 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.474 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.474 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.474 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.474 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.475 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.475 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.475 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.475 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.475 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.476 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.476 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:23.476 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-17T01:04:23.476 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2023-10-17T01:04:23.476 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-17T01:04:23.476 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-17T01:04:23.477 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-3.el8.x86_64 2023-10-17T01:04:23.477 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-17T01:04:23.477 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-17T01:04:23.477 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-17T01:04:23.477 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.478 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.478 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.478 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-17T01:04:23.478 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-17T01:04:23.478 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-17T01:04:23.479 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.5-1.el8.x86_64 2023-10-17T01:04:23.479 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-17T01:04:23.479 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-17T01:04:23.479 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.479 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.480 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-17T01:04:23.480 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.480 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-17T01:04:23.480 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-17T01:04:23.480 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-17T01:04:23.480 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-17T01:04:23.481 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-17T01:04:23.481 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-17T01:04:23.481 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-17T01:04:23.481 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-17T01:04:23.481 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-17T01:04:23.482 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-17T01:04:23.482 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-17T01:04:23.482 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-17T01:04:23.482 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-17T01:04:23.482 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-17T01:04:23.482 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:23.483 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:23.483 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:23.483 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-17T01:04:23.483 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-17T01:04:23.483 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-17T01:04:23.484 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-17T01:04:23.484 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-17T01:04:23.484 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.484 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.484 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.485 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-17T01:04:23.485 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-17T01:04:23.485 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-17T01:04:23.485 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-17T01:04:23.485 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-17T01:04:23.485 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-17T01:04:23.486 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-17T01:04:23.486 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-17T01:04:23.486 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-17T01:04:23.486 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-17T01:04:23.486 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-17T01:04:23.487 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-17T01:04:23.487 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-17T01:04:23.487 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-17T01:04:23.487 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-17T01:04:23.487 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-17T01:04:23.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-17T01:04:23.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-17T01:04:23.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-17T01:04:23.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-17T01:04:23.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-17T01:04:23.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-17T01:04:23.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-10-17T01:04:23.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-17T01:04:23.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-17T01:04:23.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-17T01:04:23.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-17T01:04:23.490 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-17T01:04:23.490 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-17T01:04:23.490 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.490 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.490 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-17T01:04:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-17T01:04:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-17T01:04:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-17T01:04:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-17T01:04:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-17T01:04:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-17T01:04:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-17T01:04:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-17T01:04:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-17T01:04:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-17T01:04:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-17T01:04:23.493 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-17T01:04:23.493 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-17T01:04:23.493 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-17T01:04:23.493 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.493 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.494 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:23.494 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-17T01:04:23.494 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-17T01:04:23.494 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-17T01:04:23.494 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-17T01:04:23.495 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-17T01:04:23.495 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-17T01:04:23.495 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:04:23.495 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-17T01:04:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 114/134 2023-10-17T01:04:23.652 DEBUG:teuthology.parallel:result is None 2023-10-17T01:04:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 115/134 2023-10-17T01:04:24.087 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-17T01:04:24.349 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-17T01:04:24.710 INFO:teuthology.orchestra.run.smithi060.stdout:Installed products updated. 2023-10-17T01:04:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:04:24.958 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2023-10-17T01:04:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-17T01:04:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.959 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-10-17T01:04:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-17T01:04:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.1-7.el8.noarch 2023-10-17T01:04:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.0.4-10.el8.x86_64 2023-10-17T01:04:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.965 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:24.965 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-17T01:04:24.965 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.1-9.el8.x86_64 2023-10-17T01:04:24.965 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-17T01:04:24.965 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-17T01:04:24.966 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-3.el8.x86_64 2023-10-17T01:04:24.966 INFO:teuthology.orchestra.run.smithi060.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-17T01:04:24.966 INFO:teuthology.orchestra.run.smithi060.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-17T01:04:24.966 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-17T01:04:24.966 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.967 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.967 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.967 INFO:teuthology.orchestra.run.smithi060.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-17T01:04:24.967 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-17T01:04:24.967 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-17T01:04:24.968 INFO:teuthology.orchestra.run.smithi060.stdout: libpq-13.5-1.el8.x86_64 2023-10-17T01:04:24.968 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-17T01:04:24.968 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-17T01:04:24.968 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.968 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.969 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-17T01:04:24.969 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.969 INFO:teuthology.orchestra.run.smithi060.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-17T01:04:24.969 INFO:teuthology.orchestra.run.smithi060.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-17T01:04:24.969 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-17T01:04:24.970 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-17T01:04:24.970 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-17T01:04:24.970 INFO:teuthology.orchestra.run.smithi060.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-17T01:04:24.970 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-17T01:04:24.971 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-17T01:04:24.971 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-17T01:04:24.971 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-17T01:04:24.971 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-17T01:04:24.971 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-17T01:04:24.972 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-17T01:04:24.972 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-17T01:04:24.972 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:24.972 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:24.972 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:24.973 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-17T01:04:24.973 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-17T01:04:24.973 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-17T01:04:24.973 INFO:teuthology.orchestra.run.smithi060.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-17T01:04:24.974 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-17T01:04:24.974 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.974 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.974 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.974 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-17T01:04:24.975 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-17T01:04:24.975 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-17T01:04:24.975 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-17T01:04:24.975 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-17T01:04:24.975 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-17T01:04:24.976 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-17T01:04:24.976 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-17T01:04:24.976 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-17T01:04:24.976 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-17T01:04:24.976 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-17T01:04:24.977 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-17T01:04:24.977 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-17T01:04:24.977 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-17T01:04:24.977 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-17T01:04:24.977 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-17T01:04:24.978 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-17T01:04:24.978 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-17T01:04:24.978 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-17T01:04:24.978 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-17T01:04:24.978 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-17T01:04:24.979 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-17T01:04:24.979 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-2.6-1.el8.noarch 2023-10-17T01:04:24.979 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-17T01:04:24.979 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-17T01:04:24.979 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-17T01:04:24.980 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-17T01:04:24.980 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-17T01:04:24.980 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-17T01:04:24.980 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.981 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.981 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-17T01:04:24.981 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-17T01:04:24.981 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.981 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-17T01:04:24.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-17T01:04:24.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-17T01:04:24.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-17T01:04:24.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-17T01:04:24.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-17T01:04:24.983 INFO:teuthology.orchestra.run.smithi060.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-17T01:04:24.983 INFO:teuthology.orchestra.run.smithi060.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-17T01:04:24.983 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-17T01:04:24.983 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-17T01:04:24.983 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-17T01:04:24.984 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-17T01:04:24.984 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-17T01:04:24.984 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.984 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.985 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:24.985 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-17T01:04:24.985 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-17T01:04:24.985 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-17T01:04:24.985 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-17T01:04:24.986 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-17T01:04:24.986 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-17T01:04:24.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-10-17T01:04:24.987 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-10-17T01:04:25.157 DEBUG:teuthology.parallel:result is None 2023-10-17T01:04:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-17T01:04:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-17T01:04:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 119/134 2023-10-17T01:04:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-17T01:04:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_6 121/134 2023-10-17T01:04:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-17T01:04:30.491 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:30.720 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:30.759 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-17T01:04:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 124/134 2023-10-17T01:04:31.280 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 125/134 2023-10-17T01:04:31.603 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 126/134 2023-10-17T01:04:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-17T01:04:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-17T01:04:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-17T01:04:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-17T01:04:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-17T01:04:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-17T01:04:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-17T01:04:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-17T01:04:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-17T01:04:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-17T01:04:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:33.969 INFO:teuthology.orchestra.run.smithi045.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-17T01:04:33.969 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:04:34.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:18.2.0-668.gab9fe435.el8.x86_64 1/134 2023-10-17T01:04:34.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 2/134 2023-10-17T01:04:34.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 3/134 2023-10-17T01:04:34.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 4/134 2023-10-17T01:04:34.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-668.gab9fe4 5/134 2023-10-17T01:04:34.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 6/134 2023-10-17T01:04:34.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 7/134 2023-10-17T01:04:34.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 8/134 2023-10-17T01:04:34.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 9/134 2023-10-17T01:04:34.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 10/134 2023-10-17T01:04:34.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 11/134 2023-10-17T01:04:34.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 12/134 2023-10-17T01:04:34.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 13/134 2023-10-17T01:04:34.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_6 14/134 2023-10-17T01:04:34.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 15/134 2023-10-17T01:04:34.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 16/134 2023-10-17T01:04:34.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 17/134 2023-10-17T01:04:34.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_ 18/134 2023-10-17T01:04:34.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 19/134 2023-10-17T01:04:34.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8 20/134 2023-10-17T01:04:34.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x 21/134 2023-10-17T01:04:34.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 22/134 2023-10-17T01:04:34.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 23/134 2023-10-17T01:04:34.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 24/134 2023-10-17T01:04:34.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 25/134 2023-10-17T01:04:34.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 26/134 2023-10-17T01:04:34.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 27/134 2023-10-17T01:04:34.615 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 28/134 2023-10-17T01:04:34.615 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.e 29/134 2023-10-17T01:04:34.616 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noar 30/134 2023-10-17T01:04:34.616 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.no 31/134 2023-10-17T01:04:34.616 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-668.gab9f 32/134 2023-10-17T01:04:34.616 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8 33/134 2023-10-17T01:04:34.616 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 34/134 2023-10-17T01:04:34.617 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el 35/134 2023-10-17T01:04:34.617 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 36/134 2023-10-17T01:04:34.617 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:18.2.0-668.gab9fe435.el8.noarch 37/134 2023-10-17T01:04:34.617 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 38/134 2023-10-17T01:04:34.617 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-17T01:04:34.618 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-17T01:04:34.618 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-17T01:04:34.618 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-17T01:04:34.618 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-17T01:04:34.619 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-17T01:04:34.619 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-17T01:04:34.619 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-17T01:04:34.619 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-17T01:04:34.619 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-17T01:04:34.620 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-17T01:04:34.620 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-17T01:04:34.620 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-17T01:04:34.620 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-17T01:04:34.620 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-17T01:04:34.621 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-17T01:04:34.621 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-17T01:04:34.621 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-17T01:04:34.621 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-17T01:04:34.621 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-17T01:04:34.621 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-17T01:04:34.622 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-17T01:04:34.622 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-17T01:04:34.622 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-17T01:04:34.622 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-17T01:04:34.622 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-17T01:04:34.623 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-17T01:04:34.623 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-17T01:04:34.623 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-17T01:04:34.623 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-17T01:04:34.623 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-17T01:04:34.624 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-17T01:04:34.624 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-17T01:04:34.624 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-17T01:04:34.624 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-17T01:04:34.624 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-17T01:04:34.625 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-17T01:04:34.625 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-17T01:04:34.625 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-17T01:04:34.625 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-17T01:04:34.625 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-17T01:04:34.626 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-17T01:04:34.626 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-17T01:04:34.626 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-17T01:04:34.626 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-17T01:04:34.626 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-17T01:04:34.626 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-17T01:04:34.627 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-17T01:04:34.627 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-17T01:04:34.627 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-17T01:04:34.627 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-17T01:04:34.627 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-17T01:04:34.628 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-17T01:04:34.628 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-17T01:04:34.628 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-17T01:04:34.628 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-17T01:04:34.628 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-17T01:04:34.629 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-17T01:04:34.629 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-17T01:04:34.629 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-17T01:04:34.629 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-17T01:04:34.629 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-17T01:04:34.630 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-17T01:04:34.630 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-17T01:04:34.630 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-17T01:04:34.630 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-17T01:04:34.630 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-17T01:04:34.631 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-17T01:04:34.631 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-17T01:04:34.631 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-17T01:04:34.631 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-17T01:04:34.631 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-17T01:04:34.632 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-17T01:04:34.632 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-17T01:04:34.632 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-17T01:04:34.632 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-17T01:04:34.632 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-17T01:04:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-17T01:04:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-17T01:04:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-17T01:04:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-17T01:04:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-17T01:04:34.634 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-17T01:04:34.634 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-17T01:04:34.634 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-17T01:04:34.634 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-17T01:04:34.634 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-17T01:04:34.635 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-17T01:04:34.635 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-17T01:04:34.635 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-17T01:04:34.635 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:18.2.0-668.gab9fe435.el8.x86_64 129/134 2023-10-17T01:04:34.635 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-17T01:04:34.636 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 131/134 2023-10-17T01:04:34.636 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-17T01:04:34.636 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-17T01:04:37.416 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-17T01:04:37.416 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:19 ago on Tue 17 Oct 2023 01:02:18 AM UTC. 2023-10-17T01:04:40.395 INFO:teuthology.orchestra.run.smithi045.stdout:Installed products updated. 2023-10-17T01:04:40.678 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:04:40.679 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2023-10-17T01:04:40.679 INFO:teuthology.orchestra.run.smithi045.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-17T01:04:40.679 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.679 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.680 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-10-17T01:04:40.680 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-17T01:04:40.680 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.1-7.el8.noarch 2023-10-17T01:04:40.680 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.0.4-10.el8.x86_64 2023-10-17T01:04:40.680 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.682 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.682 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.682 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.682 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.682 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.683 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.683 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.683 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.683 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.683 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.684 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.684 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.684 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.684 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.685 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.685 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.685 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:18.2.0-668.gab9fe435.el8.noarch 2023-10-17T01:04:40.685 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-17T01:04:40.685 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.1-9.el8.x86_64 2023-10-17T01:04:40.686 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-17T01:04:40.686 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-17T01:04:40.686 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-3.el8.x86_64 2023-10-17T01:04:40.686 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-17T01:04:40.687 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-17T01:04:40.687 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-17T01:04:40.687 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.687 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.687 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.688 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-17T01:04:40.688 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-17T01:04:40.688 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-17T01:04:40.688 INFO:teuthology.orchestra.run.smithi045.stdout: libpq-13.5-1.el8.x86_64 2023-10-17T01:04:40.688 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-17T01:04:40.688 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-17T01:04:40.689 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.689 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.689 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-17T01:04:40.689 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.689 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-17T01:04:40.690 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-17T01:04:40.690 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-17T01:04:40.690 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-17T01:04:40.690 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-17T01:04:40.691 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-17T01:04:40.691 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-17T01:04:40.691 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-17T01:04:40.691 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-17T01:04:40.691 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-17T01:04:40.692 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-17T01:04:40.692 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-17T01:04:40.692 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-17T01:04:40.692 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-17T01:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-17T01:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-17T01:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-17T01:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-17T01:04:40.694 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-17T01:04:40.694 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-17T01:04:40.694 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.694 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.695 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.695 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-17T01:04:40.695 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-17T01:04:40.695 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-17T01:04:40.695 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-17T01:04:40.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-17T01:04:40.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-17T01:04:40.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-17T01:04:40.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-17T01:04:40.696 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-17T01:04:40.697 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-17T01:04:40.697 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-17T01:04:40.697 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-17T01:04:40.697 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-17T01:04:40.697 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-17T01:04:40.698 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-17T01:04:40.698 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-17T01:04:40.698 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-17T01:04:40.698 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-17T01:04:40.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-17T01:04:40.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-17T01:04:40.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-17T01:04:40.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-17T01:04:40.699 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-2.6-1.el8.noarch 2023-10-17T01:04:40.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-17T01:04:40.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-17T01:04:40.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-17T01:04:40.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-17T01:04:40.700 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-17T01:04:40.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-17T01:04:40.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-17T01:04:40.701 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-17T01:04:40.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-17T01:04:40.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-17T01:04:40.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-17T01:04:40.702 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-17T01:04:40.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-17T01:04:40.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-17T01:04:40.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-17T01:04:40.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-17T01:04:40.703 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-17T01:04:40.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-17T01:04:40.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-17T01:04:40.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-17T01:04:40.704 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-17T01:04:40.704 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.704 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.705 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:18.2.0-668.gab9fe435.el8.x86_64 2023-10-17T01:04:40.705 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-17T01:04:40.705 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-17T01:04:40.705 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-17T01:04:40.705 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-17T01:04:40.706 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-17T01:04:40.706 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-17T01:04:40.706 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:04:40.707 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-10-17T01:04:41.005 DEBUG:teuthology.parallel:result is None 2023-10-17T01:04:41.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:04:41.174 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-17T01:04:41.233 INFO:teuthology.orchestra.run.smithi045.stdout:18.2.0-668.gab9fe435.el8 2023-10-17T01:04:41.234 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-668.gab9fe435.el8 2023-10-17T01:04:41.234 INFO:teuthology.task.install:The correct ceph version 18.2.0-668.gab9fe435 is installed. 2023-10-17T01:04:41.235 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:04:41.540 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-17T01:04:41.569 INFO:teuthology.orchestra.run.smithi060.stdout:18.2.0-668.gab9fe435.el8 2023-10-17T01:04:41.569 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-668.gab9fe435.el8 2023-10-17T01:04:41.570 INFO:teuthology.task.install:The correct ceph version 18.2.0-668.gab9fe435 is installed. 2023-10-17T01:04:41.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:04:41.799 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-17T01:04:41.827 INFO:teuthology.orchestra.run.smithi181.stdout:18.2.0-668.gab9fe435.el8 2023-10-17T01:04:41.827 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-668.gab9fe435.el8 2023-10-17T01:04:41.828 INFO:teuthology.task.install:The correct ceph version 18.2.0-668.gab9fe435 is installed. 2023-10-17T01:04:41.828 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-17T01:04:41.829 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:04:41.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-17T01:04:41.921 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:04:41.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-17T01:04:41.971 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:04:41.972 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-17T01:04:42.019 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-17T01:04:42.019 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:04:42.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2023-10-17T01:04:42.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-17T01:04:42.203 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:04:42.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2023-10-17T01:04:42.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-17T01:04:42.310 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:04:42.310 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-10-17T01:04:42.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-17T01:04:42.414 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-17T01:04:42.414 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:04:42.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-17T01:04:42.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-17T01:04:42.602 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:04:42.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-17T01:04:42.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-17T01:04:42.707 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:04:42.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-17T01:04:42.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-17T01:04:42.810 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-17T01:04:42.810 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:04:42.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2023-10-17T01:04:42.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-17T01:04:42.993 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:04:42.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2023-10-17T01:04:43.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-17T01:04:43.099 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:04:43.099 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-10-17T01:04:43.132 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-17T01:04:43.202 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-17T01:04:43.277 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe', 'subvols': {'create': 2}} 2023-10-17T01:04:43.278 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:04:43.278 INFO:tasks.cephadm:Cluster fsid is 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:04:43.279 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-17T01:04:43.279 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.60', 'mon.c': '172.21.15.181'} 2023-10-17T01:04:43.279 INFO:tasks.cephadm:First mon is mon.a on smithi045 2023-10-17T01:04:43.280 INFO:tasks.cephadm:First mgr is x 2023-10-17T01:04:43.280 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-17T01:04:43.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2023-10-17T01:04:43.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2023-10-17T01:04:43.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2023-10-17T01:04:43.418 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-17T01:04:43.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:04:43.561 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ab9fe43522bc41de59746200b95f65d4fbcccffe', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73204/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.0-668-gab9fe435', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.0-668.gab9fe435'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-11 21:17:13.737497', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri4-testing-2023-10-11-0735-reef', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-17T01:04:43.774 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-yuri4-testing-2023-10-11-0735-reef, sha1 ab9fe43522bc41de59746200b95f65d4fbcccffe from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=ab9fe43522bc41de59746200b95f65d4fbcccffe 2023-10-17T01:04:43.774 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/x86_64/flavors/default/cephadm 2023-10-17T01:04:43.775 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:44.270 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-rw-r--. 1 ubuntu ubuntu 684730 Oct 17 01:04 /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:44.270 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:44.899 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-rw-r--. 1 ubuntu ubuntu 684730 Oct 17 01:04 /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:44.900 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-yuri4-testing-2023-10-11-0735-reef/ab9fe43522bc41de59746200b95f65d4fbcccffe/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:45.637 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-rw-r--. 1 ubuntu ubuntu 684730 Oct 17 01:04 /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:45.638 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:45.690 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:45.713 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-17T01:04:45.761 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe on all hosts... 2023-10-17T01:04:45.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe pull 2023-10-17T01:04:45.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe pull 2023-10-17T01:04:45.769 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe pull 2023-10-17T01:04:47.466 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe... 2023-10-17T01:04:47.621 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe... 2023-10-17T01:04:49.787 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe... 2023-10-17T01:05:12.643 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-17T01:05:12.644 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable)", 2023-10-17T01:05:12.644 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "be87ae50630c35eef3c504cf37a937655435531c9255af168926ada29bc8ef75", 2023-10-17T01:05:12.644 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2023-10-17T01:05:12.644 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6536720a640d6ca9b09ac7d971bfb92ef4c0272aba66aced923573c65b530ad0" 2023-10-17T01:05:12.645 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-17T01:05:12.645 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-17T01:05:13.604 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-10-17T01:05:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable)", 2023-10-17T01:05:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "be87ae50630c35eef3c504cf37a937655435531c9255af168926ada29bc8ef75", 2023-10-17T01:05:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2023-10-17T01:05:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6536720a640d6ca9b09ac7d971bfb92ef4c0272aba66aced923573c65b530ad0" 2023-10-17T01:05:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-10-17T01:05:13.606 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-10-17T01:05:21.373 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:05:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable)", 2023-10-17T01:05:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "be87ae50630c35eef3c504cf37a937655435531c9255af168926ada29bc8ef75", 2023-10-17T01:05:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2023-10-17T01:05:21.374 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6536720a640d6ca9b09ac7d971bfb92ef4c0272aba66aced923573c65b530ad0" 2023-10-17T01:05:21.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:05:21.375 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:05:21.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2023-10-17T01:05:21.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2023-10-17T01:05:21.516 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2023-10-17T01:05:21.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2023-10-17T01:05:21.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2023-10-17T01:05:21.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2023-10-17T01:05:21.688 INFO:tasks.cephadm:Writing seed config... 2023-10-17T01:05:21.690 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-10-17T01:05:21.690 INFO:tasks.cephadm: override: [client] debug client = 20 2023-10-17T01:05:21.690 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-10-17T01:05:21.691 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-10-17T01:05:21.691 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-10-17T01:05:21.691 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-17T01:05:21.691 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-10-17T01:05:21.692 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-10-17T01:05:21.692 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-10-17T01:05:21.692 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-10-17T01:05:21.692 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-10-17T01:05:21.693 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-10-17T01:05:21.693 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-10-17T01:05:21.693 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-10-17T01:05:21.693 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-10-17T01:05:21.694 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-10-17T01:05:21.694 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-10-17T01:05:21.694 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-10-17T01:05:21.695 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-10-17T01:05:21.695 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-10-17T01:05:21.695 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-17T01:05:21.695 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-17T01:05:21.696 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-17T01:05:21.696 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-17T01:05:21.696 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-17T01:05:21.697 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-17T01:05:21.697 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-10-17T01:05:21.697 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-17T01:05:21.697 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-10-17T01:05:21.698 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-17T01:05:21.698 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-17T01:05:21.698 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-17T01:05:21.698 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-17T01:05:21.699 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-17T01:05:21.699 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-17T01:05:21.699 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-17T01:05:21.699 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-17T01:05:21.700 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-17T01:05:21.700 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-17T01:05:21.700 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-17T01:05:21.700 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-10-17T01:05:21.701 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-17T01:05:21.701 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-17T01:05:21.702 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:05:21.702 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-17T01:05:21.746 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2804d2ac-6c89-11ee-8db6-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-10-17T01:05:21.746 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a.service 2023-10-17T01:05:21.790 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x.service 2023-10-17T01:05:21.836 INFO:tasks.cephadm:Bootstrapping... 2023-10-17T01:05:21.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe -v bootstrap --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:05:21.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:05:21.906 INFO:journalctl@ceph.mgr.x.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:05:22.068 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-10-17T01:05:22.068 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe', '-v', 'bootstrap', '--fsid', '2804d2ac-6c89-11ee-8db6-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2023-10-17T01:05:22.244 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.1.1 2023-10-17T01:05:22.245 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-17T01:05:22.246 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2023-10-17T01:05:22.417 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.1.1 2023-10-17T01:05:22.418 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2023-10-17T01:05:22.418 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2023-10-17T01:05:22.432 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-17T01:05:22.432 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-17T01:05:22.445 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-17T01:05:22.446 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-10-17T01:05:22.460 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2023-10-17T01:05:22.473 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2023-10-17T01:05:22.473 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2023-10-17T01:05:22.473 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2023-10-17T01:05:22.645 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.1.1 2023-10-17T01:05:22.646 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-17T01:05:22.646 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2023-10-17T01:05:22.646 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2023-10-17T01:05:22.659 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-17T01:05:22.660 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-17T01:05:22.673 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-17T01:05:22.673 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-10-17T01:05:22.691 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2023-10-17T01:05:22.700 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2023-10-17T01:05:22.700 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2023-10-17T01:05:22.701 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2023-10-17T01:05:22.701 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:22.701 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 140264195893736 on /run/cephadm/2804d2ac-6c89-11ee-8db6-212e2dc638e7.lock 2023-10-17T01:05:22.702 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 140264195893736 acquired on /run/cephadm/2804d2ac-6c89-11ee-8db6-212e2dc638e7.lock 2023-10-17T01:05:22.702 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2023-10-17T01:05:22.702 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2023-10-17T01:05:22.703 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2023-10-17T01:05:22.710 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-10-17T01:05:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2023-10-17T01:05:22.717 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-17T01:05:22.717 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-17T01:05:22.717 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2023-10-17T01:05:22.724 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-17T01:05:22.724 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-17T01:05:22.724 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-17T01:05:22.724 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-17T01:05:22.724 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-17T01:05:22.725 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-17T01:05:22.725 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2023-10-17T01:05:22.726 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2023-10-17T01:05:22.726 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-17T01:05:22.726 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-17T01:05:22.727 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe... 2023-10-17T01:05:23.202 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout be87ae50630c35eef3c504cf37a937655435531c9255af168926ada29bc8ef75 2023-10-17T01:05:23.203 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe... 2023-10-17T01:05:23.203 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2023-10-17T01:05:23.203 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:d9524c86468a2638c07fdac60b1cdd48dcba4dc5d350ccf8c962be09b5eaaa47 2023-10-17T01:05:23.203 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:8ed15414cb45b8b895a21f106743ceee45272cee0d9e7e5b1f49bd1e877e4a11 2023-10-17T01:05:23.204 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:be87ae50630c35eef3c504cf37a937655435531c9255af168926ada29bc8ef75 2023-10-17T01:05:23.204 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-17T01:05:23.204 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Storing signatures 2023-10-17T01:05:24.180 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable) 2023-10-17T01:05:24.181 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable) 2023-10-17T01:05:24.181 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2023-10-17T01:05:24.958 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2023-10-17T01:05:24.958 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2023-10-17T01:05:25.937 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDV3S1l1IT+FxAAAHto50U7pJ1QhjdZYOYW2Q== 2023-10-17T01:05:26.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDW3S1lJh/YHBAAjoGUTcZFTMgoAvbNyc1E1w== 2023-10-17T01:05:27.894 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDX3S1l8aH0EhAAZrxPt2uFwu6YNl52410LMw== 2023-10-17T01:05:27.895 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2023-10-17T01:05:28.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-17T01:05:28.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-17T01:05:28.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:28.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-17T01:05:28.874 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-17T01:05:28.875 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = pacific 2023-10-17T01:05:28.875 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:28.875 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-17T01:05:28.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:05:28.875 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2023-10-17T01:05:29.757 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2023-10-17T01:05:30.032 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-17T01:05:30.237 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-17T01:05:30.779 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7.target. 2023-10-17T01:05:30.779 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7.target. 2023-10-17T01:05:31.137 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a 2023-10-17T01:05:31.138 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a.service: Unit ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a.service not loaded. 2023-10-17T01:05:31.342 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7.target.wants/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@.service. 2023-10-17T01:05:31.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:31 smithi045 systemd[1]: Starting Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:05:32.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:31 smithi045 podman[97934]: 2023-10-17T01:05:32.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:32 smithi045 bash[97934]: fa1e9805fc27457ee534c6268e57804aee249d0a1e83449f95b46d6f6ce60a17 2023-10-17T01:05:32.166 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-17T01:05:32.167 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-10-17T01:05:32.183 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-17T01:05:32.183 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-10-17T01:05:32.184 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-10-17T01:05:32.184 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2023-10-17T01:05:32.185 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2023-10-17T01:05:32.185 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2023-10-17T01:05:32.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:32 smithi045 ceph-mon[97974]: mkfs 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:32.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:32 smithi045 ceph-mon[97974]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-17T01:05:32.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:32 smithi045 systemd[1]: Started Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:05:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2023-10-17T01:05:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-17T01:05:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:33.274 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2023-10-17T01:05:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.724791s) 2023-10-17T01:05:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-17T01:05:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-17T01:05:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:33.275 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2023-10-17T01:05:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-17T01:05:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-17T01:05:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-17T01:05:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2023-10-17T01:05:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.743+0000 7fccb6cef700 1 Processor -- start 2023-10-17T01:05:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.743+0000 7fccb6cef700 1 -- start start 2023-10-17T01:05:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.743+0000 7fccb6cef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.743+0000 7fccb6cef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb007a8a0 con 0x7fccb007be70 2023-10-17T01:05:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.744+0000 7fccb4a8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:33.277 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.744+0000 7fccb4a8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53988/0 (socket says 172.21.15.45:53988) 2023-10-17T01:05:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.744+0000 7fccb4a8b700 1 -- 172.21.15.45:0/57072936 learned_addr learned my addr 172.21.15.45:0/57072936 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.745+0000 7fccb4a8b700 1 -- 172.21.15.45:0/57072936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb007a9e0 con 0x7fccb007be70 2023-10-17T01:05:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.745+0000 7fccb4a8b700 1 --2- 172.21.15.45:0/57072936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb007a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcca40098a0 tx=0x7fcca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f605e652250f8c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.745+0000 7fccaf7fe700 1 -- 172.21.15.45:0/57072936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcca4004160 con 0x7fccb007be70 2023-10-17T01:05:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.745+0000 7fccaf7fe700 1 -- 172.21.15.45:0/57072936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca40042c0 con 0x7fccb007be70 2023-10-17T01:05:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.745+0000 7fccaf7fe700 1 -- 172.21.15.45:0/57072936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcca40045a0 con 0x7fccb007be70 2023-10-17T01:05:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 -- 172.21.15.45:0/57072936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 msgr2=0x7fccb007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 --2- 172.21.15.45:0/57072936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb007a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcca40098a0 tx=0x7fcca4009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 -- 172.21.15.45:0/57072936 shutdown_connections 2023-10-17T01:05:33.279 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 --2- 172.21.15.45:0/57072936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb007a2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 -- 172.21.15.45:0/57072936 >> 172.21.15.45:0/57072936 conn(0x7fccb0101a70 msgr2=0x7fccb0103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 -- 172.21.15.45:0/57072936 shutdown_connections 2023-10-17T01:05:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.746+0000 7fccb6cef700 1 -- 172.21.15.45:0/57072936 wait complete. 2023-10-17T01:05:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.747+0000 7fccb6cef700 1 Processor -- start 2023-10-17T01:05:33.280 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.747+0000 7fccb6cef700 1 -- start start 2023-10-17T01:05:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.747+0000 7fccb6cef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.747+0000 7fccb6cef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb019bfc0 con 0x7fccb007be70 2023-10-17T01:05:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.748+0000 7fccb4a8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.748+0000 7fccb4a8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54000/0 (socket says 172.21.15.45:54000) 2023-10-17T01:05:33.281 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.748+0000 7fccb4a8b700 1 -- 172.21.15.45:0/1848589753 learned_addr learned my addr 172.21.15.45:0/1848589753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.748+0000 7fccb4a8b700 1 -- 172.21.15.45:0/1848589753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca4009550 con 0x7fccb007be70 2023-10-17T01:05:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.748+0000 7fccb4a8b700 1 --2- 172.21.15.45:0/1848589753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb019ba80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fccb007b340 tx=0x7fcca4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcca4011630 con 0x7fccb007be70 2023-10-17T01:05:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4011790 con 0x7fccb007be70 2023-10-17T01:05:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb019c1c0 con 0x7fccb007be70 2023-10-17T01:05:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcca4011a30 con 0x7fccb007be70 2023-10-17T01:05:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb019c660 con 0x7fccb007be70 2023-10-17T01:05:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcca4022070 con 0x7fccb007be70 2023-10-17T01:05:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.749+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcca401c450 con 0x7fccb007be70 2023-10-17T01:05:33.283 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.750+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb0195a90 con 0x7fccb007be70 2023-10-17T01:05:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.753+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fcca401b450 con 0x7fccb007be70 2023-10-17T01:05:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.830+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fccb002d040 con 0x7fccb007be70 2023-10-17T01:05:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.831+0000 7fccadffb700 1 -- 172.21.15.45:0/1848589753 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fcca401b5f0 con 0x7fccb007be70 2023-10-17T01:05:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.831+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 msgr2=0x7fccb019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:33.284 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.831+0000 7fccb6cef700 1 --2- 172.21.15.45:0/1848589753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb019ba80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fccb007b340 tx=0x7fcca4004750 comp rx=0 tx=0).stop 2023-10-17T01:05:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.832+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 shutdown_connections 2023-10-17T01:05:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.832+0000 7fccb6cef700 1 --2- 172.21.15.45:0/1848589753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccb007be70 0x7fccb019ba80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.832+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 >> 172.21.15.45:0/1848589753 conn(0x7fccb0101a70 msgr2=0x7fccb0103320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.832+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 shutdown_connections 2023-10-17T01:05:33.285 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:32.832+0000 7fccb6cef700 1 -- 172.21.15.45:0/1848589753 wait complete. 2023-10-17T01:05:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2023-10-17T01:05:33.286 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2023-10-17T01:05:33.533 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:33 smithi045 ceph-mon[97974]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-17T01:05:33.533 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:33 smithi045 ceph-mon[97974]: monmap e1: 1 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} 2023-10-17T01:05:33.534 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:33 smithi045 ceph-mon[97974]: fsmap 2023-10-17T01:05:33.534 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:33 smithi045 ceph-mon[97974]: osdmap e1: 0 total, 0 up, 0 in 2023-10-17T01:05:33.534 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:33 smithi045 ceph-mon[97974]: mgrmap e1: no daemons active 2023-10-17T01:05:33.534 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:33 smithi045 ceph-mon[97974]: from='client.? 172.21.15.45:0/1848589753' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-17T01:05:34.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:34 smithi045 ceph-mon[97974]: from='client.? 172.21.15.45:0/3456842270' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-17T01:05:34.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:34 smithi045 ceph-mon[97974]: from='client.? 172.21.15.45:0/3456842270' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-17T01:05:34.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:34.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2023-10-17T01:05:34.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:34.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2023-10-17T01:05:34.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-17T01:05:34.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-17T01:05:34.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-17T01:05:34.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-17T01:05:34.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:34.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-17T01:05:34.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-17T01:05:34.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2023-10-17T01:05:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-17T01:05:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-17T01:05:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-17T01:05:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-17T01:05:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-17T01:05:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.809+0000 7f4fb3fff700 1 Processor -- start 2023-10-17T01:05:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.809+0000 7f4fb3fff700 1 -- start start 2023-10-17T01:05:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.809+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.809+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb407a8c0 con 0x7f4fb407be90 2023-10-17T01:05:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.810+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.810+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54016/0 (socket says 172.21.15.45:54016) 2023-10-17T01:05:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.810+0000 7f4fb2ffd700 1 -- 172.21.15.45:0/332993462 learned_addr learned my addr 172.21.15.45:0/332993462 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.810+0000 7f4fb2ffd700 1 -- 172.21.15.45:0/332993462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb407aa00 con 0x7f4fb407be90 2023-10-17T01:05:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.810+0000 7f4fb2ffd700 1 --2- 172.21.15.45:0/332993462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb407a2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40098a0 tx=0x7f4fa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fc7cb606d0e88d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb1ffb700 1 -- 172.21.15.45:0/332993462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fa4004160 con 0x7f4fb407be90 2023-10-17T01:05:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb1ffb700 1 -- 172.21.15.45:0/332993462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa40042c0 con 0x7f4fb407be90 2023-10-17T01:05:34.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb1ffb700 1 -- 172.21.15.45:0/332993462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fa40045a0 con 0x7f4fb407be90 2023-10-17T01:05:34.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 -- 172.21.15.45:0/332993462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 msgr2=0x7f4fb407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:34.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 --2- 172.21.15.45:0/332993462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb407a2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40098a0 tx=0x7f4fa4009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:34.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 -- 172.21.15.45:0/332993462 shutdown_connections 2023-10-17T01:05:34.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 --2- 172.21.15.45:0/332993462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb407a2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:34.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 -- 172.21.15.45:0/332993462 >> 172.21.15.45:0/332993462 conn(0x7f4fb41018f0 msgr2=0x7f4fb4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:34.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 -- 172.21.15.45:0/332993462 shutdown_connections 2023-10-17T01:05:34.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.811+0000 7f4fb3fff700 1 -- 172.21.15.45:0/332993462 wait complete. 2023-10-17T01:05:34.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.813+0000 7f4fb3fff700 1 Processor -- start 2023-10-17T01:05:34.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.813+0000 7f4fb3fff700 1 -- start start 2023-10-17T01:05:34.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.814+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:34.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.814+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb419be80 con 0x7f4fb407be90 2023-10-17T01:05:34.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.814+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:34.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.814+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54018/0 (socket says 172.21.15.45:54018) 2023-10-17T01:05:34.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.814+0000 7f4fb2ffd700 1 -- 172.21.15.45:0/3456842270 learned_addr learned my addr 172.21.15.45:0/3456842270 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:34.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.814+0000 7f4fb2ffd700 1 -- 172.21.15.45:0/3456842270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009550 con 0x7f4fb407be90 2023-10-17T01:05:34.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4fb2ffd700 1 --2- 172.21.15.45:0/3456842270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb419b940 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4004000 tx=0x7f4fa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:34.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fa4011630 con 0x7f4fb407be90 2023-10-17T01:05:34.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4011790 con 0x7f4fb407be90 2023-10-17T01:05:34.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fa4011a30 con 0x7f4fb407be90 2023-10-17T01:05:34.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb419c080 con 0x7f4fb407be90 2023-10-17T01:05:34.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb419c520 con 0x7f4fb407be90 2023-10-17T01:05:34.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4fa4022070 con 0x7f4fb407be90 2023-10-17T01:05:34.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.815+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4fa401c450 con 0x7f4fb407be90 2023-10-17T01:05:34.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.816+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb4195820 con 0x7f4fb407be90 2023-10-17T01:05:34.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.819+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4fa401b450 con 0x7f4fb407be90 2023-10-17T01:05:34.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.892+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4fb402cee0 con 0x7f4fb407be90 2023-10-17T01:05:34.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.897+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4fa401b5f0 con 0x7f4fb407be90 2023-10-17T01:05:34.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.897+0000 7f4f9bfff700 1 -- 172.21.15.45:0/3456842270 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f4fa40375f0 con 0x7f4fb407be90 2023-10-17T01:05:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.898+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 msgr2=0x7f4fb419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.898+0000 7f4fb3fff700 1 --2- 172.21.15.45:0/3456842270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb419b940 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4004000 tx=0x7f4fa4004750 comp rx=0 tx=0).stop 2023-10-17T01:05:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.898+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 shutdown_connections 2023-10-17T01:05:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.898+0000 7f4fb3fff700 1 --2- 172.21.15.45:0/3456842270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb407be90 0x7f4fb419b940 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.898+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 >> 172.21.15.45:0/3456842270 conn(0x7f4fb41018f0 msgr2=0x7f4fb41031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.898+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 shutdown_connections 2023-10-17T01:05:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:33.899+0000 7f4fb3fff700 1 -- 172.21.15.45:0/3456842270 wait complete. 2023-10-17T01:05:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2023-10-17T01:05:35.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:35 smithi045 ceph-mon[97974]: from='client.? 172.21.15.45:0/2126041284' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:05:35.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.895+0000 7fba8818e700 1 Processor -- start 2023-10-17T01:05:35.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.895+0000 7fba8818e700 1 -- start start 2023-10-17T01:05:35.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.895+0000 7fba8818e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba801068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:35.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.895+0000 7fba8818e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80106ea0 con 0x7fba801064b0 2023-10-17T01:05:35.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.896+0000 7fba85f2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba801068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:35.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.896+0000 7fba85f2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba801068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54028/0 (socket says 172.21.15.45:54028) 2023-10-17T01:05:35.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.896+0000 7fba85f2a700 1 -- 172.21.15.45:0/2992221132 learned_addr learned my addr 172.21.15.45:0/2992221132 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:35.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.896+0000 7fba85f2a700 1 -- 172.21.15.45:0/2992221132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba801076b0 con 0x7fba801064b0 2023-10-17T01:05:35.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.897+0000 7fba85f2a700 1 --2- 172.21.15.45:0/2992221132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba801068d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0098a0 tx=0x7fba7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5aeb21589de0c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:35.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.897+0000 7fba84f28700 1 -- 172.21.15.45:0/2992221132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba7c004160 con 0x7fba801064b0 2023-10-17T01:05:35.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.897+0000 7fba84f28700 1 -- 172.21.15.45:0/2992221132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba7c0042c0 con 0x7fba801064b0 2023-10-17T01:05:35.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.897+0000 7fba84f28700 1 -- 172.21.15.45:0/2992221132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba7c011a00 con 0x7fba801064b0 2023-10-17T01:05:35.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.897+0000 7fba8818e700 1 -- 172.21.15.45:0/2992221132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 msgr2=0x7fba801068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:35.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.897+0000 7fba8818e700 1 --2- 172.21.15.45:0/2992221132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba801068d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0098a0 tx=0x7fba7c009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:35.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.898+0000 7fba8818e700 1 -- 172.21.15.45:0/2992221132 shutdown_connections 2023-10-17T01:05:35.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.898+0000 7fba8818e700 1 --2- 172.21.15.45:0/2992221132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba801068d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:35.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.898+0000 7fba8818e700 1 -- 172.21.15.45:0/2992221132 >> 172.21.15.45:0/2992221132 conn(0x7fba80101a70 msgr2=0x7fba80103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:35.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.898+0000 7fba8818e700 1 -- 172.21.15.45:0/2992221132 shutdown_connections 2023-10-17T01:05:35.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.898+0000 7fba8818e700 1 -- 172.21.15.45:0/2992221132 wait complete. 2023-10-17T01:05:35.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.899+0000 7fba8818e700 1 Processor -- start 2023-10-17T01:05:35.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.899+0000 7fba8818e700 1 -- start start 2023-10-17T01:05:35.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.900+0000 7fba8818e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba8019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:35.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.900+0000 7fba8818e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8019c820 con 0x7fba801064b0 2023-10-17T01:05:35.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.900+0000 7fba85f2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba8019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:35.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.900+0000 7fba85f2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba8019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54034/0 (socket says 172.21.15.45:54034) 2023-10-17T01:05:35.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.900+0000 7fba85f2a700 1 -- 172.21.15.45:0/2126041284 learned_addr learned my addr 172.21.15.45:0/2126041284 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:35.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.900+0000 7fba85f2a700 1 -- 172.21.15.45:0/2126041284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c009550 con 0x7fba801064b0 2023-10-17T01:05:35.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba85f2a700 1 --2- 172.21.15.45:0/2126041284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba8019c2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba7c006af0 tx=0x7fba7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:35.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba7c003a40 con 0x7fba801064b0 2023-10-17T01:05:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba7c003ba0 con 0x7fba801064b0 2023-10-17T01:05:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba7c01a700 con 0x7fba801064b0 2023-10-17T01:05:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8019ca20 con 0x7fba801064b0 2023-10-17T01:05:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8019cec0 con 0x7fba801064b0 2023-10-17T01:05:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.901+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba7c01ac30 con 0x7fba801064b0 2023-10-17T01:05:35.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.902+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba7c019700 con 0x7fba801064b0 2023-10-17T01:05:35.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.902+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba80062380 con 0x7fba801064b0 2023-10-17T01:05:35.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.905+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fba7c024ad0 con 0x7fba801064b0 2023-10-17T01:05:35.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.979+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fba8010a960 con 0x7fba801064b0 2023-10-17T01:05:35.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.980+0000 7fba76ffd700 1 -- 172.21.15.45:0/2126041284 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fba7c024c70 con 0x7fba801064b0 2023-10-17T01:05:35.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 msgr2=0x7fba8019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:35.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 --2- 172.21.15.45:0/2126041284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba8019c2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba7c006af0 tx=0x7fba7c011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:35.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 shutdown_connections 2023-10-17T01:05:35.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 --2- 172.21.15.45:0/2126041284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba801064b0 0x7fba8019c2e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:35.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 >> 172.21.15.45:0/2126041284 conn(0x7fba80101a70 msgr2=0x7fba801027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:35.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 shutdown_connections 2023-10-17T01:05:35.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:34.981+0000 7fba8818e700 1 -- 172.21.15.45:0/2126041284 wait complete. 2023-10-17T01:05:35.447 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2023-10-17T01:05:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:35 smithi045 systemd[1]: Stopping Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:05:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a[97950]: 2023-10-17T01:05:36.528+0000 7f2067b4e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:05:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a[97950]: 2023-10-17T01:05:36.528+0000 7f2067b4e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-17T01:05:37.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:37 smithi045 bash[98310]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a 2023-10-17T01:05:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:37 smithi045 systemd[1]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a.service: Succeeded. 2023-10-17T01:05:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:37 smithi045 systemd[1]: Stopped Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:05:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:37 smithi045 systemd[1]: Starting Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:05:38.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:37 smithi045 podman[98434]: 2023-10-17T01:05:38.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 bash[98434]: a6f83c0b712c2a681d4532f54e5f01671a472badaedb8fd7a1376dfbcb061739 2023-10-17T01:05:38.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: set uid:gid to 167:167 (ceph:ceph) 2023-10-17T01:05:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-17T01:05:38.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable), process ceph-mon, pid 7 2023-10-17T01:05:38.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: pidfile_write: ignore empty --pid-file 2023-10-17T01:05:38.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: load: jerasure load: lrc 2023-10-17T01:05:38.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: RocksDB version: 7.9.2 2023-10-17T01:05:38.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Git sha 0 2023-10-17T01:05:38.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Compile date 2023-10-11 20:19:14 2023-10-17T01:05:38.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: DB SUMMARY 2023-10-17T01:05:38.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: DB Session ID: FQKV7YK45LT63WXXA8JQ 2023-10-17T01:05:38.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: CURRENT file: CURRENT 2023-10-17T01:05:38.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: IDENTITY file: IDENTITY 2023-10-17T01:05:38.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-17T01:05:38.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-17T01:05:38.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98294 ; 2023-10-17T01:05:38.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.error_if_exists: 0 2023-10-17T01:05:38.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.create_if_missing: 0 2023-10-17T01:05:38.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.paranoid_checks: 1 2023-10-17T01:05:38.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-17T01:05:38.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-17T01:05:38.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-17T01:05:38.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.env: 0x5581b52a6720 2023-10-17T01:05:38.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.fs: PosixFileSystem 2023-10-17T01:05:38.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.info_log: 0x5581b68d5420 2023-10-17T01:05:38.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_file_opening_threads: 16 2023-10-17T01:05:38.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.statistics: (nil) 2023-10-17T01:05:38.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.use_fsync: 0 2023-10-17T01:05:38.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_log_file_size: 0 2023-10-17T01:05:38.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-17T01:05:38.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-17T01:05:38.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.keep_log_file_num: 1000 2023-10-17T01:05:38.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.recycle_log_file_num: 0 2023-10-17T01:05:38.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.allow_fallocate: 1 2023-10-17T01:05:38.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.allow_mmap_reads: 0 2023-10-17T01:05:38.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.allow_mmap_writes: 0 2023-10-17T01:05:38.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.use_direct_reads: 0 2023-10-17T01:05:38.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-17T01:05:38.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.create_missing_column_families: 0 2023-10-17T01:05:38.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.db_log_dir: 2023-10-17T01:05:38.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.wal_dir: 2023-10-17T01:05:38.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-17T01:05:38.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-17T01:05:38.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-17T01:05:38.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-17T01:05:38.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-17T01:05:38.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-17T01:05:38.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.advise_random_on_open: 1 2023-10-17T01:05:38.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.db_write_buffer_size: 0 2023-10-17T01:05:38.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.write_buffer_manager: 0x5581b5b665a0 2023-10-17T01:05:38.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-17T01:05:38.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-17T01:05:38.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-17T01:05:38.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.rate_limiter: (nil) 2023-10-17T01:05:38.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-17T01:05:38.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.wal_recovery_mode: 2 2023-10-17T01:05:38.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.enable_thread_tracking: 0 2023-10-17T01:05:38.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.enable_pipelined_write: 0 2023-10-17T01:05:38.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.unordered_write: 0 2023-10-17T01:05:38.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-17T01:05:38.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-17T01:05:38.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-17T01:05:38.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-17T01:05:38.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.row_cache: None 2023-10-17T01:05:38.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.wal_filter: None 2023-10-17T01:05:38.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-17T01:05:38.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.allow_ingest_behind: 0 2023-10-17T01:05:38.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.two_write_queues: 0 2023-10-17T01:05:38.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.manual_wal_flush: 0 2023-10-17T01:05:38.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.wal_compression: 0 2023-10-17T01:05:38.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.atomic_flush: 0 2023-10-17T01:05:38.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-17T01:05:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-17T01:05:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-17T01:05:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.log_readahead_size: 0 2023-10-17T01:05:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-17T01:05:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.best_efforts_recovery: 0 2023-10-17T01:05:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-17T01:05:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-17T01:05:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.allow_data_in_errors: 0 2023-10-17T01:05:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.db_host_id: __hostname__ 2023-10-17T01:05:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-17T01:05:38.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_background_jobs: 2 2023-10-17T01:05:38.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_background_compactions: -1 2023-10-17T01:05:38.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_subcompactions: 1 2023-10-17T01:05:38.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-17T01:05:38.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-17T01:05:38.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-17T01:05:38.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_total_wal_size: 0 2023-10-17T01:05:38.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-17T01:05:38.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-17T01:05:38.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-17T01:05:38.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-17T01:05:38.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_open_files: -1 2023-10-17T01:05:38.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bytes_per_sync: 0 2023-10-17T01:05:38.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-17T01:05:38.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-17T01:05:38.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_readahead_size: 0 2023-10-17T01:05:38.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_background_flushes: -1 2023-10-17T01:05:38.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Compression algorithms supported: 2023-10-17T01:05:38.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-17T01:05:38.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kZSTD supported: 0 2023-10-17T01:05:38.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kXpressCompression supported: 0 2023-10-17T01:05:38.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kLZ4HCCompression supported: 1 2023-10-17T01:05:38.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kZlibCompression supported: 1 2023-10-17T01:05:38.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kSnappyCompression supported: 1 2023-10-17T01:05:38.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kLZ4Compression supported: 1 2023-10-17T01:05:38.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: kBZip2Compression supported: 0 2023-10-17T01:05:38.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-17T01:05:38.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-17T01:05:38.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-17T01:05:38.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-17T01:05:38.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-17T01:05:38.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.merge_operator: 2023-10-17T01:05:38.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_filter: None 2023-10-17T01:05:38.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_filter_factory: None 2023-10-17T01:05:38.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.sst_partitioner_factory: None 2023-10-17T01:05:38.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-17T01:05:38.576 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-17T01:05:38.576 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5581b68d5520) 2023-10-17T01:05:38.576 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2023-10-17T01:05:38.576 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-17T01:05:38.576 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-17T01:05:38.577 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2023-10-17T01:05:38.577 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2023-10-17T01:05:38.577 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2023-10-17T01:05:38.577 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2023-10-17T01:05:38.577 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-17T01:05:38.578 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2023-10-17T01:05:38.578 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2023-10-17T01:05:38.578 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x5581b5be7350 2023-10-17T01:05:38.578 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2023-10-17T01:05:38.578 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2023-10-17T01:05:38.579 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2023-10-17T01:05:38.579 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2023-10-17T01:05:38.579 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2023-10-17T01:05:38.579 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2023-10-17T01:05:38.579 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2023-10-17T01:05:38.580 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2023-10-17T01:05:38.580 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2023-10-17T01:05:38.580 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2023-10-17T01:05:38.580 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2023-10-17T01:05:38.580 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2023-10-17T01:05:38.580 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2023-10-17T01:05:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2023-10-17T01:05:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2023-10-17T01:05:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2023-10-17T01:05:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2023-10-17T01:05:38.581 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2023-10-17T01:05:38.582 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2023-10-17T01:05:38.582 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2023-10-17T01:05:38.582 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2023-10-17T01:05:38.582 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2023-10-17T01:05:38.582 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2023-10-17T01:05:38.583 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2023-10-17T01:05:38.583 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2023-10-17T01:05:38.583 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2023-10-17T01:05:38.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.write_buffer_size: 33554432 2023-10-17T01:05:38.583 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_write_buffer_number: 2 2023-10-17T01:05:38.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression: NoCompression 2023-10-17T01:05:38.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression: Disabled 2023-10-17T01:05:38.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.prefix_extractor: nullptr 2023-10-17T01:05:38.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-17T01:05:38.584 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.num_levels: 7 2023-10-17T01:05:38.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-17T01:05:38.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-17T01:05:38.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-17T01:05:38.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-17T01:05:38.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-17T01:05:38.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-17T01:05:38.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-17T01:05:38.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-17T01:05:38.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-17T01:05:38.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-17T01:05:38.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-17T01:05:38.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-17T01:05:38.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-17T01:05:38.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.level: 32767 2023-10-17T01:05:38.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.strategy: 0 2023-10-17T01:05:38.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-17T01:05:38.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-17T01:05:38.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-17T01:05:38.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-17T01:05:38.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.enabled: false 2023-10-17T01:05:38.588 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-17T01:05:38.589 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-17T01:05:38.589 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-17T01:05:38.589 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-17T01:05:38.589 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.target_file_size_base: 67108864 2023-10-17T01:05:38.589 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-17T01:05:38.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-17T01:05:38.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-17T01:05:38.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-17T01:05:38.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-17T01:05:38.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-17T01:05:38.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-17T01:05:38.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-17T01:05:38.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-17T01:05:38.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-17T01:05:38.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-17T01:05:38.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-17T01:05:38.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-17T01:05:38.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-17T01:05:38.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.arena_block_size: 1048576 2023-10-17T01:05:38.592 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-17T01:05:38.593 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-17T01:05:38.593 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.disable_auto_compactions: 0 2023-10-17T01:05:38.593 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-17T01:05:38.593 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-17T01:05:38.593 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-17T01:05:38.593 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-17T01:05:38.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-17T01:05:38.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-17T01:05:38.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-17T01:05:38.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-17T01:05:38.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-17T01:05:38.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-17T01:05:38.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.table_properties_collectors: 2023-10-17T01:05:38.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.inplace_update_support: 0 2023-10-17T01:05:38.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-17T01:05:38.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-17T01:05:38.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-17T01:05:38.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-17T01:05:38.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.bloom_locality: 0 2023-10-17T01:05:38.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.max_successive_merges: 0 2023-10-17T01:05:38.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-17T01:05:38.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.paranoid_file_checks: 0 2023-10-17T01:05:38.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.force_consistency_checks: 1 2023-10-17T01:05:38.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.report_bg_io_stats: 0 2023-10-17T01:05:38.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.ttl: 2592000 2023-10-17T01:05:38.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-17T01:05:38.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-17T01:05:38.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-17T01:05:38.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.enable_blob_files: false 2023-10-17T01:05:38.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.min_blob_size: 0 2023-10-17T01:05:38.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.blob_file_size: 268435456 2023-10-17T01:05:38.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-17T01:05:38.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-17T01:05:38.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-17T01:05:38.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-17T01:05:38.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-17T01:05:38.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.blob_file_starting_level: 0 2023-10-17T01:05:38.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-17T01:05:38.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-17T01:05:38.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-17T01:05:38.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 92bed1af-6db7-4229-83e4-8955e5fab085 2023-10-17T01:05:38.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697504738258506, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-17T01:05:38.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-17T01:05:38.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697504738260244, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93631, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91626, "index_size": 287, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13529, "raw_average_key_size": 49, "raw_value_size": 84213, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697504738, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "92bed1af-6db7-4229-83e4-8955e5fab085", "db_session_id": "FQKV7YK45LT63WXXA8JQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-17T01:05:38.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697504738260391, "job": 1, "event": "recovery_finished"} 2023-10-17T01:05:38.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-17T01:05:38.602 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-17T01:05:38.602 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5581b5c84000 2023-10-17T01:05:38.602 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: DB pointer 0x5581b5c70000 2023-10-17T01:05:38.602 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-17T01:05:38.602 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-17T01:05:38.603 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2023-10-17T01:05:38.603 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-17T01:05:38.603 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-17T01:05:38.603 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-17T01:05:38.603 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-17T01:05:38.604 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-17T01:05:38.604 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-17T01:05:38.604 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-17T01:05:38.604 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-10-17T01:05:38.604 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2023-10-17T01:05:38.605 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-17T01:05:38.605 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-17T01:05:38.605 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 93.27 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 87.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:05:38.605 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 93.27 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 87.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:05:38.605 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 87.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:05:38.606 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-10-17T01:05:38.606 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2023-10-17T01:05:38.606 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-17T01:05:38.606 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-17T01:05:38.606 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 87.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:05:38.607 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-10-17T01:05:38.607 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-17T01:05:38.607 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-10-17T01:05:38.607 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-17T01:05:38.607 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-17T01:05:38.607 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-17T01:05:38.608 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-17T01:05:38.608 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-17T01:05:38.608 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-17T01:05:38.608 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 12.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-17T01:05:38.608 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 12.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-17T01:05:38.609 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-17T01:05:38.609 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x5581b5be7350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-10-17T01:05:38.609 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-10-17T01:05:38.609 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-10-17T01:05:38.609 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-17T01:05:38.610 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:38.610 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???) e1 preinit fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:38.610 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).mds e1 new map 2023-10-17T01:05:38.610 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).mds e1 print_map 2023-10-17T01:05:38.610 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2023-10-17T01:05:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-17T01:05:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-17T01:05:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2023-10-17T01:05:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2023-10-17T01:05:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2023-10-17T01:05:38.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-17T01:05:38.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-17T01:05:38.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-17T01:05:38.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-17T01:05:38.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-17T01:05:38.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 systemd[1]: Started Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:05:38.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-17T01:05:38.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: monmap e1: 1 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} 2023-10-17T01:05:38.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: fsmap 2023-10-17T01:05:38.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: osdmap e1: 0 total, 0 up, 0 in 2023-10-17T01:05:38.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:38 smithi045 ceph-mon[98474]: mgrmap e1: no daemons active 2023-10-17T01:05:39.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.837+0000 7f6ec09c6700 1 Processor -- start 2023-10-17T01:05:39.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.838+0000 7f6ec09c6700 1 -- start start 2023-10-17T01:05:39.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.838+0000 7f6ec09c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc07a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:39.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.838+0000 7f6ec09c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc07a880 con 0x7f6ebc07be50 2023-10-17T01:05:39.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.839+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc07a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:39.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.839+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc07a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42180/0 (socket says 172.21.15.45:42180) 2023-10-17T01:05:39.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.839+0000 7f6eba59c700 1 -- 172.21.15.45:0/1143539859 learned_addr learned my addr 172.21.15.45:0/1143539859 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:39.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.839+0000 7f6eba59c700 1 -- 172.21.15.45:0/1143539859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc07a9c0 con 0x7f6ebc07be50 2023-10-17T01:05:39.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.839+0000 7f6eba59c700 1 --2- 172.21.15.45:0/1143539859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc07a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098a0 tx=0x7f6eac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d097f02a0dc131 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:39.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6eb959a700 1 -- 172.21.15.45:0/1143539859 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6eac004160 con 0x7f6ebc07be50 2023-10-17T01:05:39.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6eb959a700 1 -- 172.21.15.45:0/1143539859 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6eac0042c0 con 0x7f6ebc07be50 2023-10-17T01:05:39.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6eb959a700 1 -- 172.21.15.45:0/1143539859 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6eac011a00 con 0x7f6ebc07be50 2023-10-17T01:05:39.370 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6ec09c6700 1 -- 172.21.15.45:0/1143539859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 msgr2=0x7f6ebc07a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:39.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6ec09c6700 1 --2- 172.21.15.45:0/1143539859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc07a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0098a0 tx=0x7f6eac009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:39.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6ec09c6700 1 -- 172.21.15.45:0/1143539859 shutdown_connections 2023-10-17T01:05:39.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6ec09c6700 1 --2- 172.21.15.45:0/1143539859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc07a2b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:39.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.840+0000 7f6ec09c6700 1 -- 172.21.15.45:0/1143539859 >> 172.21.15.45:0/1143539859 conn(0x7f6ebc101a30 msgr2=0x7f6ebc103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:39.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.841+0000 7f6ec09c6700 1 -- 172.21.15.45:0/1143539859 shutdown_connections 2023-10-17T01:05:39.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.841+0000 7f6ec09c6700 1 -- 172.21.15.45:0/1143539859 wait complete. 2023-10-17T01:05:39.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6ec09c6700 1 Processor -- start 2023-10-17T01:05:39.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6ec09c6700 1 -- start start 2023-10-17T01:05:39.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6ec09c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:39.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6ec09c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc19c860 con 0x7f6ebc07be50 2023-10-17T01:05:39.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:39.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42196/0 (socket says 172.21.15.45:42196) 2023-10-17T01:05:39.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.842+0000 7f6eba59c700 1 -- 172.21.15.45:0/196258944 learned_addr learned my addr 172.21.15.45:0/196258944 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:39.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6eba59c700 1 -- 172.21.15.45:0/196258944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009550 con 0x7f6ebc07be50 2023-10-17T01:05:39.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6eba59c700 1 --2- 172.21.15.45:0/196258944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc19c320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009fb0 tx=0x7f6eac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:39.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6eac003a40 con 0x7f6ebc07be50 2023-10-17T01:05:39.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6eac003ba0 con 0x7f6ebc07be50 2023-10-17T01:05:39.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc19ca60 con 0x7f6ebc07be50 2023-10-17T01:05:39.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6eac01a750 con 0x7f6ebc07be50 2023-10-17T01:05:39.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.843+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc19cf00 con 0x7f6ebc07be50 2023-10-17T01:05:39.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.844+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6eac01acd0 con 0x7f6ebc07be50 2023-10-17T01:05:39.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.844+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6eac0193c0 con 0x7f6ebc07be50 2023-10-17T01:05:39.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.844+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc062380 con 0x7f6ebc07be50 2023-10-17T01:05:39.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.847+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6eac02de40 con 0x7f6ebc07be50 2023-10-17T01:05:39.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.921+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6ebc19f920 con 0x7f6ebc07be50 2023-10-17T01:05:39.376 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.924+0000 7f6eab7fe700 1 -- 172.21.15.45:0/196258944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6eac024b60 con 0x7f6ebc07be50 2023-10-17T01:05:39.376 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 msgr2=0x7f6ebc19c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:39.376 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 --2- 172.21.15.45:0/196258944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc19c320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009fb0 tx=0x7f6eac011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:39.376 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 shutdown_connections 2023-10-17T01:05:39.376 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 --2- 172.21.15.45:0/196258944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ebc07be50 0x7f6ebc19c320 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:39.376 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 >> 172.21.15.45:0/196258944 conn(0x7f6ebc101a30 msgr2=0x7f6ebc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:39.377 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 shutdown_connections 2023-10-17T01:05:39.377 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:38.925+0000 7f6ec09c6700 1 -- 172.21.15.45:0/196258944 wait complete. 2023-10-17T01:05:39.377 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-17T01:05:39.386 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:05:39.386 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2023-10-17T01:05:39.387 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-17T01:05:39.387 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-17T01:05:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x 2023-10-17T01:05:39.749 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x.service: Unit ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x.service not loaded. 2023-10-17T01:05:39.963 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7.target.wants/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@.service. 2023-10-17T01:05:40.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:39 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/196258944' entity='client.admin' 2023-10-17T01:05:40.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:39 smithi045 systemd[1]: Starting Ceph mgr.x for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:05:40.719 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:40 smithi045 podman[98706]: 2023-10-17T01:05:40.799 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-17T01:05:40.799 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-10-17T01:05:40.818 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-17T01:05:40.819 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-10-17T01:05:40.819 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-10-17T01:05:40.819 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2023-10-17T01:05:40.840 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-17T01:05:40.841 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-10-17T01:05:40.857 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-17T01:05:40.858 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-10-17T01:05:40.858 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-10-17T01:05:40.858 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-17T01:05:40.858 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2023-10-17T01:05:40.858 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2023-10-17T01:05:41.002 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:40 smithi045 bash[98706]: 62c2e090a59370506ed44a2049b1535abb514b9fd43bfbd5231d69cd88487b60 2023-10-17T01:05:41.002 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:40 smithi045 systemd[1]: Started Ceph mgr.x for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:05:41.003 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:40 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:40.882+0000 7fa37fb82200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-17T01:05:41.003 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:41 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:41.000+0000 7fa37fb82200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-17T01:05:41.252 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:41 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:41.120+0000 7fa37fb82200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-17T01:05:41.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:41 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:41.250+0000 7fa37fb82200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-17T01:05:41.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:41 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:41.496+0000 7fa37fb82200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-17T01:05:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:41 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2087261398' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-17T01:05:41.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:41.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:05:41.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:05:41.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-10-17T01:05:41.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-17T01:05:41.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-17T01:05:41.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-17T01:05:41.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-17T01:05:41.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-17T01:05:41.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-10-17T01:05:41.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:41.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-17T01:05:41.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-10-17T01:05:41.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:41.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-17T01:05:41.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-17T01:05:41.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:41.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-17T01:05:41.950 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-17T01:05:41.950 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.950 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-17T01:05:41.950 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:41.950 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-17T01:05:41.951 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-17T01:05:41.951 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-17T01:05:41.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-17T01:05:41.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-17T01:05:41.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-17T01:05:41.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-17T01:05:41.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-17T01:05:41.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-17T01:05:41.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-17T01:05:41.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-17T01:05:41.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-17T01:05:41.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-17T01:05:41.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-17T01:05:41.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-17T01:05:41.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-17T01:05:41.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:41.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-17T01:05:41.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-17T01:05:41.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-17T01:05:41.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-17T01:05:41.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-17T01:05:41.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-17T01:05:41.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-17T01:05:41.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-17T01:05:41.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-10-17T01:05:41.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:41.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-17T01:05:41.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-17T01:05:41.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:41.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-10-17T01:05:32.108829+0000", 2023-10-17T01:05:41.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-17T01:05:41.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:41.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-17T01:05:41.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:05:41.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.439+0000 7fb28b7e2700 1 Processor -- start 2023-10-17T01:05:41.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.440+0000 7fb28b7e2700 1 -- start start 2023-10-17T01:05:41.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.440+0000 7fb28b7e2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb284107140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:41.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.440+0000 7fb28b7e2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284079ea0 con 0x7fb284104d10 2023-10-17T01:05:41.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.440+0000 7fb28957e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb284107140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:41.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb28957e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb284107140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42210/0 (socket says 172.21.15.45:42210) 2023-10-17T01:05:41.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb28957e700 1 -- 172.21.15.45:0/2114540649 learned_addr learned my addr 172.21.15.45:0/2114540649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:41.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb28957e700 1 -- 172.21.15.45:0/2114540649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284107680 con 0x7fb284104d10 2023-10-17T01:05:41.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb28957e700 1 --2- 172.21.15.45:0/2114540649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb284107140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098a0 tx=0x7fb280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ba9dbbdcabe4fb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:41.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb27bfff700 1 -- 172.21.15.45:0/2114540649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb280004160 con 0x7fb284104d10 2023-10-17T01:05:41.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb27bfff700 1 -- 172.21.15.45:0/2114540649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2800042c0 con 0x7fb284104d10 2023-10-17T01:05:41.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.441+0000 7fb27bfff700 1 -- 172.21.15.45:0/2114540649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb280011a00 con 0x7fb284104d10 2023-10-17T01:05:41.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2114540649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 msgr2=0x7fb284107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:41.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 --2- 172.21.15.45:0/2114540649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb284107140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098a0 tx=0x7fb280009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:41.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2114540649 shutdown_connections 2023-10-17T01:05:41.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 --2- 172.21.15.45:0/2114540649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb284107140 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:41.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2114540649 >> 172.21.15.45:0/2114540649 conn(0x7fb284100990 msgr2=0x7fb284102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:41.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2114540649 shutdown_connections 2023-10-17T01:05:41.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.442+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2114540649 wait complete. 2023-10-17T01:05:41.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.443+0000 7fb28b7e2700 1 Processor -- start 2023-10-17T01:05:41.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.443+0000 7fb28b7e2700 1 -- start start 2023-10-17T01:05:41.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.444+0000 7fb28b7e2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb2841a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:41.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.444+0000 7fb28b7e2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2841a0c30 con 0x7fb284104d10 2023-10-17T01:05:41.971 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.444+0000 7fb28957e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb2841a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:41.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.444+0000 7fb28957e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb2841a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42222/0 (socket says 172.21.15.45:42222) 2023-10-17T01:05:41.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.444+0000 7fb28957e700 1 -- 172.21.15.45:0/2087261398 learned_addr learned my addr 172.21.15.45:0/2087261398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:41.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.444+0000 7fb28957e700 1 -- 172.21.15.45:0/2087261398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009550 con 0x7fb284104d10 2023-10-17T01:05:41.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb28957e700 1 --2- 172.21.15.45:0/2087261398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb2841a06f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb280006af0 tx=0x7fb280011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:41.972 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb280003a40 con 0x7fb284104d10 2023-10-17T01:05:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb280003ba0 con 0x7fb284104d10 2023-10-17T01:05:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb28001a6e0 con 0x7fb284104d10 2023-10-17T01:05:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2841a0e30 con 0x7fb284104d10 2023-10-17T01:05:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2841a12d0 con 0x7fb284104d10 2023-10-17T01:05:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb28001ac50 con 0x7fb284104d10 2023-10-17T01:05:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.445+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb280019700 con 0x7fb284104d10 2023-10-17T01:05:41.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.446+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284199ff0 con 0x7fb284104d10 2023-10-17T01:05:41.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.449+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb28002de30 con 0x7fb284104d10 2023-10-17T01:05:41.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.526+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb284062380 con 0x7fb284104d10 2023-10-17T01:05:41.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.526+0000 7fb27a7fc700 1 -- 172.21.15.45:0/2087261398 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fb280024a60 con 0x7fb284104d10 2023-10-17T01:05:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 msgr2=0x7fb2841a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 --2- 172.21.15.45:0/2087261398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb2841a06f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb280006af0 tx=0x7fb280011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 shutdown_connections 2023-10-17T01:05:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 --2- 172.21.15.45:0/2087261398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb284104d10 0x7fb2841a06f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 >> 172.21.15.45:0/2087261398 conn(0x7fb284100990 msgr2=0x7fb284102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:41.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 shutdown_connections 2023-10-17T01:05:41.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:41.527+0000 7fb28b7e2700 1 -- 172.21.15.45:0/2087261398 wait complete. 2023-10-17T01:05:41.976 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2023-10-17T01:05:42.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:42 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:42.013+0000 7fa37fb82200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-17T01:05:42.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:42 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:42.188+0000 7fa37fb82200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-17T01:05:42.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:42 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:42.308+0000 7fa37fb82200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-17T01:05:42.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:42 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:42.437+0000 7fa37fb82200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-17T01:05:43.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:43 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:43.003+0000 7fa37fb82200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-17T01:05:44.251 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:43 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:43.923+0000 7fa37fb82200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-17T01:05:44.252 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:44 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:44.037+0000 7fa37fb82200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-17T01:05:45.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:45.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:05:45.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:05:45.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-10-17T01:05:45.034 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-17T01:05:45.034 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-17T01:05:45.034 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-17T01:05:45.034 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.034 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-17T01:05:45.035 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-17T01:05:45.035 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-10-17T01:05:45.035 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:45.035 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-17T01:05:45.035 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-10-17T01:05:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-17T01:05:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-17T01:05:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-17T01:05:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-17T01:05:45.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-17T01:05:45.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:45.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-17T01:05:45.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-17T01:05:45.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-17T01:05:45.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-17T01:05:45.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-17T01:05:45.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-17T01:05:45.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-17T01:05:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-17T01:05:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-17T01:05:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-17T01:05:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-17T01:05:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-17T01:05:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-17T01:05:45.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-17T01:05:45.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-17T01:05:45.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-17T01:05:45.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:45.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-17T01:05:45.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-17T01:05:45.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-17T01:05:45.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-17T01:05:45.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-17T01:05:45.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-17T01:05:45.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-17T01:05:45.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-17T01:05:45.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-10-17T01:05:45.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:45.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-17T01:05:45.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-17T01:05:45.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:45.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-10-17T01:05:32.108829+0000", 2023-10-17T01:05:45.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-17T01:05:45.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:45.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-17T01:05:45.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:05:45.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.513+0000 7fcd28cdd700 1 Processor -- start 2023-10-17T01:05:45.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.513+0000 7fcd28cdd700 1 -- start start 2023-10-17T01:05:45.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.514+0000 7fcd28cdd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:45.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.514+0000 7fcd28cdd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2407a8a0 con 0x7fcd2407be70 2023-10-17T01:05:45.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.514+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:45.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.514+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42232/0 (socket says 172.21.15.45:42232) 2023-10-17T01:05:45.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.514+0000 7fcd2259c700 1 -- 172.21.15.45:0/4022759486 learned_addr learned my addr 172.21.15.45:0/4022759486 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:45.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.515+0000 7fcd2259c700 1 -- 172.21.15.45:0/4022759486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2407a9e0 con 0x7fcd2407be70 2023-10-17T01:05:45.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.515+0000 7fcd2259c700 1 --2- 172.21.15.45:0/4022759486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2407a2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd140098a0 tx=0x7fcd14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f5aa897d49f812 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:45.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:44 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2620998893' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-17T01:05:45.055 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.515+0000 7fcd2159a700 1 -- 172.21.15.45:0/4022759486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd14004160 con 0x7fcd2407be70 2023-10-17T01:05:45.055 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.515+0000 7fcd2159a700 1 -- 172.21.15.45:0/4022759486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd140042c0 con 0x7fcd2407be70 2023-10-17T01:05:45.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.515+0000 7fcd2159a700 1 -- 172.21.15.45:0/4022759486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd14011a00 con 0x7fcd2407be70 2023-10-17T01:05:45.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 -- 172.21.15.45:0/4022759486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 msgr2=0x7fcd2407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:45.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 --2- 172.21.15.45:0/4022759486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2407a2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd140098a0 tx=0x7fcd14009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:45.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 -- 172.21.15.45:0/4022759486 shutdown_connections 2023-10-17T01:05:45.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 --2- 172.21.15.45:0/4022759486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2407a2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:45.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 -- 172.21.15.45:0/4022759486 >> 172.21.15.45:0/4022759486 conn(0x7fcd24101a70 msgr2=0x7fcd24103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:45.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 -- 172.21.15.45:0/4022759486 shutdown_connections 2023-10-17T01:05:45.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.516+0000 7fcd28cdd700 1 -- 172.21.15.45:0/4022759486 wait complete. 2023-10-17T01:05:45.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.517+0000 7fcd28cdd700 1 Processor -- start 2023-10-17T01:05:45.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.517+0000 7fcd28cdd700 1 -- start start 2023-10-17T01:05:45.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.517+0000 7fcd28cdd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:45.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.518+0000 7fcd28cdd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2419c7f0 con 0x7fcd2407be70 2023-10-17T01:05:45.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.518+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:45.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.518+0000 7fcd2259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42246/0 (socket says 172.21.15.45:42246) 2023-10-17T01:05:45.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.518+0000 7fcd2259c700 1 -- 172.21.15.45:0/2620998893 learned_addr learned my addr 172.21.15.45:0/2620998893 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:45.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.518+0000 7fcd2259c700 1 -- 172.21.15.45:0/2620998893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14009550 con 0x7fcd2407be70 2023-10-17T01:05:45.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.518+0000 7fcd2259c700 1 --2- 172.21.15.45:0/2620998893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2419c2b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009fb0 tx=0x7fcd14011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:45.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.519+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd14003930 con 0x7fcd2407be70 2023-10-17T01:05:45.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.519+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd14003a90 con 0x7fcd2407be70 2023-10-17T01:05:45.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.519+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2419c9f0 con 0x7fcd2407be70 2023-10-17T01:05:45.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.519+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd14019610 con 0x7fcd2407be70 2023-10-17T01:05:45.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.519+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2419ce90 con 0x7fcd2407be70 2023-10-17T01:05:45.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.520+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcd14019880 con 0x7fcd2407be70 2023-10-17T01:05:45.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.520+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcd1402da10 con 0x7fcd2407be70 2023-10-17T01:05:45.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.520+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd24062380 con 0x7fcd2407be70 2023-10-17T01:05:45.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.524+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fcd14003c00 con 0x7fcd2407be70 2023-10-17T01:05:45.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.602+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcd2419f8b0 con 0x7fcd2407be70 2023-10-17T01:05:45.062 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.603+0000 7fcd137fe700 1 -- 172.21.15.45:0/2620998893 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fcd1402d440 con 0x7fcd2407be70 2023-10-17T01:05:45.062 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 msgr2=0x7fcd2419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:45.062 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 --2- 172.21.15.45:0/2620998893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2419c2b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009fb0 tx=0x7fcd14011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:45.063 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 shutdown_connections 2023-10-17T01:05:45.063 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 --2- 172.21.15.45:0/2620998893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2407be70 0x7fcd2419c2b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:45.063 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 >> 172.21.15.45:0/2620998893 conn(0x7fcd24101a70 msgr2=0x7fcd24102850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:45.063 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 shutdown_connections 2023-10-17T01:05:45.063 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:44.604+0000 7fcd28cdd700 1 -- 172.21.15.45:0/2620998893 wait complete. 2023-10-17T01:05:45.064 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2023-10-17T01:05:45.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:45 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:45.071+0000 7fa37fb82200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:05:45.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:45 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:45.199+0000 7fa37fb82200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-17T01:05:45.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:45 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:45.432+0000 7fa37fb82200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:05:46.095 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:45 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:45.811+0000 7fa37fb82200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-17T01:05:46.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:46 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:46.094+0000 7fa37fb82200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-17T01:05:46.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:46 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:46.302+0000 7fa37fb82200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-17T01:05:47.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:46 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:46.665+0000 7fa37fb82200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-17T01:05:48.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:47 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:47.649+0000 7fa37fb82200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-17T01:05:48.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:47 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:47.833+0000 7fa37fb82200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-17T01:05:48.541 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:48 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:48.093+0000 7fa37fb82200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-17T01:05:48.542 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:48 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:48.249+0000 7fa37fb82200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-17T01:05:48.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:48 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:48.540+0000 7fa37fb82200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-17T01:05:50.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:49.663+0000 7fa37fb82200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-17T01:05:50.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: Activating manager daemon x 2023-10-17T01:05:50.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: mgrmap e2: x(active, starting, since 0.00448124s) 2023-10-17T01:05:50.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-17T01:05:50.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-17T01:05:50.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-17T01:05:50.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:05:50.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-17T01:05:50.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: Manager daemon x is now available 2023-10-17T01:05:50.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-17T01:05:50.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-17T01:05:50.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' 2023-10-17T01:05:50.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' 2023-10-17T01:05:50.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:49 smithi045 ceph-mon[98474]: from='mgr.14100 172.21.15.45:0/3906735406' entity='mgr.x' 2023-10-17T01:05:52.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:51 smithi045 ceph-mon[98474]: mgrmap e3: x(active, since 1.0096s) 2023-10-17T01:05:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:52 smithi045 ceph-mon[98474]: mgrmap e4: x(active, since 2s) 2023-10-17T01:05:54.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:54 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3616001879' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-17T01:05:54.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:54.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:05:54.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:05:54.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-10-17T01:05:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-17T01:05:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-17T01:05:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-17T01:05:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-17T01:05:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-17T01:05:54.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-10-17T01:05:54.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:54.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-17T01:05:54.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2023-10-17T01:05:54.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-10-17T01:05:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-17T01:05:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-17T01:05:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-17T01:05:54.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-17T01:05:54.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:54.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-17T01:05:54.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-17T01:05:54.952 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-17T01:05:54.952 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-17T01:05:54.953 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-17T01:05:54.953 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-17T01:05:54.953 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.953 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-17T01:05:54.953 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-17T01:05:54.954 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-17T01:05:54.954 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-17T01:05:54.954 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-17T01:05:54.954 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-17T01:05:54.954 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-17T01:05:54.955 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-17T01:05:54.955 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-17T01:05:54.955 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.955 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-17T01:05:54.955 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:54.955 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-17T01:05:54.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-17T01:05:54.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-17T01:05:54.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-17T01:05:54.956 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-17T01:05:54.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-17T01:05:54.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-17T01:05:54.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-17T01:05:54.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-10-17T01:05:54.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-10-17T01:05:54.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-17T01:05:54.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-17T01:05:54.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-17T01:05:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-10-17T01:05:32.108829+0000", 2023-10-17T01:05:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-17T01:05:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-10-17T01:05:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-17T01:05:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:05:54.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.139+0000 7f8af7016700 1 Processor -- start 2023-10-17T01:05:54.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.139+0000 7f8af7016700 1 -- start start 2023-10-17T01:05:54.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.140+0000 7f8af7016700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:54.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.140+0000 7f8af7016700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af007a8a0 con 0x7f8af007be70 2023-10-17T01:05:54.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.140+0000 7f8af4db2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.140+0000 7f8af4db2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41086/0 (socket says 172.21.15.45:41086) 2023-10-17T01:05:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.140+0000 7f8af4db2700 1 -- 172.21.15.45:0/500665513 learned_addr learned my addr 172.21.15.45:0/500665513 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.140+0000 7f8af4db2700 1 -- 172.21.15.45:0/500665513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af007a9e0 con 0x7f8af007be70 2023-10-17T01:05:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.141+0000 7f8af4db2700 1 --2- 172.21.15.45:0/500665513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af007a2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40098a0 tx=0x7f8ae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de98bee301d8cb6f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.141+0000 7f8aef7fe700 1 -- 172.21.15.45:0/500665513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4004160 con 0x7f8af007be70 2023-10-17T01:05:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.141+0000 7f8aef7fe700 1 -- 172.21.15.45:0/500665513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ae40042c0 con 0x7f8af007be70 2023-10-17T01:05:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.141+0000 7f8aef7fe700 1 -- 172.21.15.45:0/500665513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4011a00 con 0x7f8af007be70 2023-10-17T01:05:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.141+0000 7f8af7016700 1 -- 172.21.15.45:0/500665513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 msgr2=0x7f8af007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.141+0000 7f8af7016700 1 --2- 172.21.15.45:0/500665513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af007a2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40098a0 tx=0x7f8ae4009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.142+0000 7f8af7016700 1 -- 172.21.15.45:0/500665513 shutdown_connections 2023-10-17T01:05:54.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.142+0000 7f8af7016700 1 --2- 172.21.15.45:0/500665513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af007a2d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:54.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.142+0000 7f8af7016700 1 -- 172.21.15.45:0/500665513 >> 172.21.15.45:0/500665513 conn(0x7f8af0101a70 msgr2=0x7f8af0103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:54.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.142+0000 7f8af7016700 1 -- 172.21.15.45:0/500665513 shutdown_connections 2023-10-17T01:05:54.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.142+0000 7f8af7016700 1 -- 172.21.15.45:0/500665513 wait complete. 2023-10-17T01:05:54.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.143+0000 7f8af7016700 1 Processor -- start 2023-10-17T01:05:54.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.143+0000 7f8af7016700 1 -- start start 2023-10-17T01:05:54.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.143+0000 7f8af7016700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:54.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.143+0000 7f8af7016700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af019c7f0 con 0x7f8af007be70 2023-10-17T01:05:54.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8af4db2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:54.964 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8af4db2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41100/0 (socket says 172.21.15.45:41100) 2023-10-17T01:05:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8af4db2700 1 -- 172.21.15.45:0/3616001879 learned_addr learned my addr 172.21.15.45:0/3616001879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8af4db2700 1 -- 172.21.15.45:0/3616001879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4009550 con 0x7f8af007be70 2023-10-17T01:05:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8af4db2700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af019c2b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4006af0 tx=0x7f8ae4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae4003a40 con 0x7f8af007be70 2023-10-17T01:05:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ae4003ba0 con 0x7f8af007be70 2023-10-17T01:05:54.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af019c9f0 con 0x7f8af007be70 2023-10-17T01:05:54.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.144+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ae401a6e0 con 0x7f8af007be70 2023-10-17T01:05:54.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.145+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af019ce90 con 0x7f8af007be70 2023-10-17T01:05:54.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.146+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45411+0+0 (secure 0 0 0) 0x7f8ae4024430 con 0x7f8af007be70 2023-10-17T01:05:54.966 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.145+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af010c8a0 con 0x7f8af007be70 2023-10-17T01:05:54.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.146+0000 7f8aedffb700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f8ae00384b0 0x7f8ae003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:54.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.146+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8ae404c3f0 con 0x7f8af007be70 2023-10-17T01:05:54.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.146+0000 7f8aeffff700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f8ae00384b0 0x7f8ae003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:54.967 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.147+0000 7f8aeffff700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f8ae00384b0 0x7f8ae003a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8adc006fd0 tx=0x7f8adc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:54.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.152+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f8ae40246e0 con 0x7f8af007be70 2023-10-17T01:05:54.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.446+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8af019fc70 con 0x7f8af007be70 2023-10-17T01:05:54.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.446+0000 7f8aedffb700 1 -- 172.21.15.45:0/3616001879 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f8ae4018350 con 0x7f8af007be70 2023-10-17T01:05:54.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f8ae00384b0 msgr2=0x7f8ae003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:54.968 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f8ae00384b0 0x7f8ae003a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8adc006fd0 tx=0x7f8adc006e40 comp rx=0 tx=0).stop 2023-10-17T01:05:54.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 msgr2=0x7f8af019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:54.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af019c2b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4006af0 tx=0x7f8ae4011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:54.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 shutdown_connections 2023-10-17T01:05:54.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f8ae00384b0 0x7f8ae003a970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:54.969 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 --2- 172.21.15.45:0/3616001879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af007be70 0x7f8af019c2b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:54.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 >> 172.21.15.45:0/3616001879 conn(0x7f8af0101a70 msgr2=0x7f8af0102850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:54.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 shutdown_connections 2023-10-17T01:05:54.970 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:54.449+0000 7f8af7016700 1 -- 172.21.15.45:0/3616001879 wait complete. 2023-10-17T01:05:54.970 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2023-10-17T01:05:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:55 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4178529956' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-17T01:05:56.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:56.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2023-10-17T01:05:56.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:05:56.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-17T01:05:56.128 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-17T01:05:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-17T01:05:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-17T01:05:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-17T01:05:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-17T01:05:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-10-17T01:05:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2023-10-17T01:05:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-17T01:05:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-17T01:05:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-17T01:05:56.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-17T01:05:56.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-17T01:05:56.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.479+0000 7fe5d75ed700 1 Processor -- start 2023-10-17T01:05:56.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.480+0000 7fe5d75ed700 1 -- start start 2023-10-17T01:05:56.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.480+0000 7fe5d75ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:56.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.480+0000 7fe5d75ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d007a8a0 con 0x7fe5d007be70 2023-10-17T01:05:56.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.480+0000 7fe5d5389700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:56.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.480+0000 7fe5d5389700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41104/0 (socket says 172.21.15.45:41104) 2023-10-17T01:05:56.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.480+0000 7fe5d5389700 1 -- 172.21.15.45:0/1648840631 learned_addr learned my addr 172.21.15.45:0/1648840631 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:56.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.481+0000 7fe5d5389700 1 -- 172.21.15.45:0/1648840631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d007a9e0 con 0x7fe5d007be70 2023-10-17T01:05:56.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.481+0000 7fe5d5389700 1 --2- 172.21.15.45:0/1648840631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d007a2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098a0 tx=0x7fe5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67891fb518de7bc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:56.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.481+0000 7fe5c7fff700 1 -- 172.21.15.45:0/1648840631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc004160 con 0x7fe5d007be70 2023-10-17T01:05:56.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.481+0000 7fe5c7fff700 1 -- 172.21.15.45:0/1648840631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5cc0042c0 con 0x7fe5d007be70 2023-10-17T01:05:56.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.481+0000 7fe5c7fff700 1 -- 172.21.15.45:0/1648840631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc011a00 con 0x7fe5d007be70 2023-10-17T01:05:56.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 -- 172.21.15.45:0/1648840631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 msgr2=0x7fe5d007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:56.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 --2- 172.21.15.45:0/1648840631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d007a2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098a0 tx=0x7fe5cc009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:56.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 -- 172.21.15.45:0/1648840631 shutdown_connections 2023-10-17T01:05:56.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 --2- 172.21.15.45:0/1648840631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d007a2d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:56.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 -- 172.21.15.45:0/1648840631 >> 172.21.15.45:0/1648840631 conn(0x7fe5d0101a70 msgr2=0x7fe5d0103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:56.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 -- 172.21.15.45:0/1648840631 shutdown_connections 2023-10-17T01:05:56.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.482+0000 7fe5d75ed700 1 -- 172.21.15.45:0/1648840631 wait complete. 2023-10-17T01:05:56.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.483+0000 7fe5d75ed700 1 Processor -- start 2023-10-17T01:05:56.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.483+0000 7fe5d75ed700 1 -- start start 2023-10-17T01:05:56.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d75ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:56.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d75ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d019c7f0 con 0x7fe5d007be70 2023-10-17T01:05:56.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d5389700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:56.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d5389700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41108/0 (socket says 172.21.15.45:41108) 2023-10-17T01:05:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d5389700 1 -- 172.21.15.45:0/4178529956 learned_addr learned my addr 172.21.15.45:0/4178529956 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d5389700 1 -- 172.21.15.45:0/4178529956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc009550 con 0x7fe5d007be70 2023-10-17T01:05:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.484+0000 7fe5d5389700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d019c2b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006af0 tx=0x7fe5cc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.485+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc003a40 con 0x7fe5d007be70 2023-10-17T01:05:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.485+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5cc003ba0 con 0x7fe5d007be70 2023-10-17T01:05:56.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.485+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc01a6e0 con 0x7fe5d007be70 2023-10-17T01:05:56.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.485+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d019c9f0 con 0x7fe5d007be70 2023-10-17T01:05:56.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.485+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d019ce90 con 0x7fe5d007be70 2023-10-17T01:05:56.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.486+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45411+0+0 (secure 0 0 0) 0x7fe5cc024a60 con 0x7fe5d007be70 2023-10-17T01:05:56.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.486+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d010c8a0 con 0x7fe5d007be70 2023-10-17T01:05:56.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.486+0000 7fe5c67fc700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fe5bc038500 0x7fe5bc03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:56.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.487+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe5cc04c3c0 con 0x7fe5d007be70 2023-10-17T01:05:56.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.487+0000 7fe5d4b88700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fe5bc038500 0x7fe5bc03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:56.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.487+0000 7fe5d4b88700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fe5bc038500 0x7fe5bc03a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0006fd0 tx=0x7fe5c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:56.139 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.491+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe5cc024430 con 0x7fe5d007be70 2023-10-17T01:05:56.139 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.700+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe5d010ca90 con 0x7fe5d007be70 2023-10-17T01:05:56.139 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.701+0000 7fe5c67fc700 1 -- 172.21.15.45:0/4178529956 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe5cc018350 con 0x7fe5d007be70 2023-10-17T01:05:56.139 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fe5bc038500 msgr2=0x7fe5bc03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:56.139 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fe5bc038500 0x7fe5bc03a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0006fd0 tx=0x7fe5c0006e40 comp rx=0 tx=0).stop 2023-10-17T01:05:56.140 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 msgr2=0x7fe5d019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:56.140 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d019c2b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006af0 tx=0x7fe5cc011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:56.140 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 shutdown_connections 2023-10-17T01:05:56.140 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fe5bc038500 0x7fe5bc03a9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:56.140 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 --2- 172.21.15.45:0/4178529956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d007be70 0x7fe5d019c2b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:56.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 >> 172.21.15.45:0/4178529956 conn(0x7fe5d0101a70 msgr2=0x7fe5d0102850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:56.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 shutdown_connections 2023-10-17T01:05:56.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:55.704+0000 7fe5d75ed700 1 -- 172.21.15.45:0/4178529956 wait complete. 2023-10-17T01:05:56.141 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2023-10-17T01:05:57.268 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1609452977' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-17T01:05:57.269 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: ignoring --setuser ceph since I am not root 2023-10-17T01:05:57.269 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: ignoring --setgroup ceph since I am not root 2023-10-17T01:05:57.269 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:57.267+0000 7f1ef27d0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-17T01:05:57.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.671+0000 7fd0fdb56700 1 Processor -- start 2023-10-17T01:05:57.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.671+0000 7fd0fdb56700 1 -- start start 2023-10-17T01:05:57.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.672+0000 7fd0fdb56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:57.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.672+0000 7fd0fdb56700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f807a8a0 con 0x7fd0f807be70 2023-10-17T01:05:57.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.672+0000 7fd0f77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:57.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.672+0000 7fd0f77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41116/0 (socket says 172.21.15.45:41116) 2023-10-17T01:05:57.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.672+0000 7fd0f77fe700 1 -- 172.21.15.45:0/2865399763 learned_addr learned my addr 172.21.15.45:0/2865399763 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:57.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.672+0000 7fd0f77fe700 1 -- 172.21.15.45:0/2865399763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f807a9e0 con 0x7fd0f807be70 2023-10-17T01:05:57.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.673+0000 7fd0f77fe700 1 --2- 172.21.15.45:0/2865399763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f807a2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80098a0 tx=0x7fd0e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81436514da6d631 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:57.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.673+0000 7fd0f67fc700 1 -- 172.21.15.45:0/2865399763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0e8004160 con 0x7fd0f807be70 2023-10-17T01:05:57.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.673+0000 7fd0f67fc700 1 -- 172.21.15.45:0/2865399763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0e80042c0 con 0x7fd0f807be70 2023-10-17T01:05:57.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.673+0000 7fd0f67fc700 1 -- 172.21.15.45:0/2865399763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0e8011a00 con 0x7fd0f807be70 2023-10-17T01:05:57.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.673+0000 7fd0fdb56700 1 -- 172.21.15.45:0/2865399763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 msgr2=0x7fd0f807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.673+0000 7fd0fdb56700 1 --2- 172.21.15.45:0/2865399763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f807a2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80098a0 tx=0x7fd0e8009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.674+0000 7fd0fdb56700 1 -- 172.21.15.45:0/2865399763 shutdown_connections 2023-10-17T01:05:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.674+0000 7fd0fdb56700 1 --2- 172.21.15.45:0/2865399763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f807a2d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.674+0000 7fd0fdb56700 1 -- 172.21.15.45:0/2865399763 >> 172.21.15.45:0/2865399763 conn(0x7fd0f8101a30 msgr2=0x7fd0f8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.674+0000 7fd0fdb56700 1 -- 172.21.15.45:0/2865399763 shutdown_connections 2023-10-17T01:05:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.674+0000 7fd0fdb56700 1 -- 172.21.15.45:0/2865399763 wait complete. 2023-10-17T01:05:57.414 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0fdb56700 1 Processor -- start 2023-10-17T01:05:57.414 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0fdb56700 1 -- start start 2023-10-17T01:05:57.414 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0fdb56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:57.414 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0fdb56700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819c870 con 0x7fd0f807be70 2023-10-17T01:05:57.415 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0f77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:57.415 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0f77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41118/0 (socket says 172.21.15.45:41118) 2023-10-17T01:05:57.415 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.675+0000 7fd0f77fe700 1 -- 172.21.15.45:0/1609452977 learned_addr learned my addr 172.21.15.45:0/1609452977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:57.415 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0f77fe700 1 -- 172.21.15.45:0/1609452977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8009550 con 0x7fd0f807be70 2023-10-17T01:05:57.415 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0f77fe700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f819c330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009130 tx=0x7fd0e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:57.416 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0e8003b70 con 0x7fd0f807be70 2023-10-17T01:05:57.416 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0e8003cd0 con 0x7fd0f807be70 2023-10-17T01:05:57.416 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0e80197d0 con 0x7fd0f807be70 2023-10-17T01:05:57.416 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f819ca70 con 0x7fd0f807be70 2023-10-17T01:05:57.417 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.676+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f819cf10 con 0x7fd0f807be70 2023-10-17T01:05:57.417 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.677+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f810c880 con 0x7fd0f807be70 2023-10-17T01:05:57.417 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.678+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 45411+0+0 (secure 0 0 0) 0x7fd0e801abb0 con 0x7fd0f807be70 2023-10-17T01:05:57.417 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.678+0000 7fd0f4ff9700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd0e0038500 0x7fd0e003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:57.417 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.678+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd0e804c630 con 0x7fd0f807be70 2023-10-17T01:05:57.418 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.678+0000 7fd0f6ffd700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd0e0038500 0x7fd0e003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:57.418 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.679+0000 7fd0f6ffd700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd0e0038500 0x7fd0e003a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec006fd0 tx=0x7fd0ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:57.418 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.683+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd0e801a430 con 0x7fd0f807be70 2023-10-17T01:05:57.418 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:56.951+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd0f819fcf0 con 0x7fd0f807be70 2023-10-17T01:05:57.418 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.004+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 5) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fd0e802bd40 con 0x7fd0f807be70 2023-10-17T01:05:57.419 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.006+0000 7fd0f4ff9700 1 -- 172.21.15.45:0/1609452977 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd0e8023a00 con 0x7fd0f807be70 2023-10-17T01:05:57.419 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd0e0038500 msgr2=0x7fd0e003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:57.419 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd0e0038500 0x7fd0e003a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec006fd0 tx=0x7fd0ec006e40 comp rx=0 tx=0).stop 2023-10-17T01:05:57.419 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 msgr2=0x7fd0f819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:57.420 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f819c330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009130 tx=0x7fd0e8011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:57.420 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 shutdown_connections 2023-10-17T01:05:57.420 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd0e0038500 0x7fd0e003a9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:57.420 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 --2- 172.21.15.45:0/1609452977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f807be70 0x7fd0f819c330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:57.420 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.009+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 >> 172.21.15.45:0/1609452977 conn(0x7fd0f8101a30 msgr2=0x7fd0f81030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:57.421 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.010+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 shutdown_connections 2023-10-17T01:05:57.421 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.010+0000 7fd0fdb56700 1 -- 172.21.15.45:0/1609452977 wait complete. 2023-10-17T01:05:57.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:57.376+0000 7f1ef27d0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-17T01:05:57.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:57.503+0000 7f1ef27d0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-17T01:05:58.010 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:57.629+0000 7f1ef27d0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-17T01:05:58.010 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:57.874+0000 7f1ef27d0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-17T01:05:58.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:58 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1609452977' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-17T01:05:58.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:58 smithi045 ceph-mon[98474]: mgrmap e5: x(active, since 7s) 2023-10-17T01:05:58.659 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:58.369+0000 7f1ef27d0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-17T01:05:58.659 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:58.539+0000 7f1ef27d0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-17T01:05:58.659 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:58.657+0000 7f1ef27d0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-17T01:05:58.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:05:58.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-17T01:05:58.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-17T01:05:58.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-17T01:05:58.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-17T01:05:58.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:05:58.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.938+0000 7f1cd6f86700 1 Processor -- start 2023-10-17T01:05:58.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.938+0000 7f1cd6f86700 1 -- start start 2023-10-17T01:05:58.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.938+0000 7f1cd6f86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd01068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:58.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.938+0000 7f1cd6f86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0106ea0 con 0x7f1cd01064b0 2023-10-17T01:05:58.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.939+0000 7f1cd4d22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd01068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:58.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.939+0000 7f1cd4d22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd01068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34422/0 (socket says 172.21.15.45:34422) 2023-10-17T01:05:58.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.939+0000 7f1cd4d22700 1 -- 172.21.15.45:0/2407231766 learned_addr learned my addr 172.21.15.45:0/2407231766 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:58.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.939+0000 7f1cd4d22700 1 -- 172.21.15.45:0/2407231766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd01076b0 con 0x7f1cd01064b0 2023-10-17T01:05:58.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.939+0000 7f1cd4d22700 1 --2- 172.21.15.45:0/2407231766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd01068d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40098a0 tx=0x7f1cc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce5ed6b42f77287 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:58.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2407231766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4004160 con 0x7f1cd01064b0 2023-10-17T01:05:58.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2407231766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1cc40042c0 con 0x7f1cd01064b0 2023-10-17T01:05:58.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2407231766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4011a00 con 0x7f1cd01064b0 2023-10-17T01:05:58.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1cd6f86700 1 -- 172.21.15.45:0/2407231766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 msgr2=0x7f1cd01068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:58.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1cd6f86700 1 --2- 172.21.15.45:0/2407231766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd01068d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40098a0 tx=0x7f1cc4009bb0 comp rx=0 tx=0).stop 2023-10-17T01:05:58.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1cd6f86700 1 -- 172.21.15.45:0/2407231766 shutdown_connections 2023-10-17T01:05:58.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1cd6f86700 1 --2- 172.21.15.45:0/2407231766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd01068d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:58.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.940+0000 7f1cd6f86700 1 -- 172.21.15.45:0/2407231766 >> 172.21.15.45:0/2407231766 conn(0x7f1cd0101a70 msgr2=0x7f1cd0103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:58.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.941+0000 7f1cd6f86700 1 -- 172.21.15.45:0/2407231766 shutdown_connections 2023-10-17T01:05:58.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.941+0000 7f1cd6f86700 1 -- 172.21.15.45:0/2407231766 wait complete. 2023-10-17T01:05:58.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.942+0000 7f1cd6f86700 1 Processor -- start 2023-10-17T01:05:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.942+0000 7f1cd6f86700 1 -- start start 2023-10-17T01:05:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.942+0000 7f1cd6f86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.942+0000 7f1cd6f86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd019c820 con 0x7f1cd01064b0 2023-10-17T01:05:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.943+0000 7f1cd4d22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:05:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.943+0000 7f1cd4d22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34434/0 (socket says 172.21.15.45:34434) 2023-10-17T01:05:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.943+0000 7f1cd4d22700 1 -- 172.21.15.45:0/3988546249 learned_addr learned my addr 172.21.15.45:0/3988546249 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:05:58.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.943+0000 7f1cd4d22700 1 -- 172.21.15.45:0/3988546249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4009550 con 0x7f1cd01064b0 2023-10-17T01:05:58.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.943+0000 7f1cd4d22700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd019c2e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4006af0 tx=0x7f1cc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:05:58.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.943+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4003a40 con 0x7f1cd01064b0 2023-10-17T01:05:58.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.944+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1cc4003ba0 con 0x7f1cd01064b0 2023-10-17T01:05:58.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.944+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd019ca20 con 0x7f1cd01064b0 2023-10-17T01:05:58.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.944+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cc401a700 con 0x7f1cd01064b0 2023-10-17T01:05:58.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.944+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd019cec0 con 0x7f1cd01064b0 2023-10-17T01:05:58.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.945+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f1cc401a930 con 0x7f1cd01064b0 2023-10-17T01:05:58.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.945+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd0062380 con 0x7f1cd01064b0 2023-10-17T01:05:58.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.945+0000 7f1ccdffb700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 0x7f1cc003aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:05:58.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.945+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1cc404c3d0 con 0x7f1cd01064b0 2023-10-17T01:05:58.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.945+0000 7f1ccffff700 1 -- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 msgr2=0x7f1cc003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:05:58.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.945+0000 7f1ccffff700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 0x7f1cc003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:05:58.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:57.951+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1cc4018b80 con 0x7f1cd01064b0 2023-10-17T01:05:58.704 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.146+0000 7f1ccffff700 1 -- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 msgr2=0x7f1cc003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:05:58.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.146+0000 7f1ccffff700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 0x7f1cc003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-17T01:05:58.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.231+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1cd010a8e0 con 0x7f1cd01064b0 2023-10-17T01:05:58.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.232+0000 7f1ccdffb700 1 -- 172.21.15.45:0/3988546249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1cc4018350 con 0x7f1cd01064b0 2023-10-17T01:05:58.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 msgr2=0x7f1cc003aa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:05:58.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 0x7f1cc003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 msgr2=0x7f1cd019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:05:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd019c2e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4006af0 tx=0x7f1cc4011ad0 comp rx=0 tx=0).stop 2023-10-17T01:05:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 shutdown_connections 2023-10-17T01:05:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7f1cc0038550 0x7f1cc003aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 --2- 172.21.15.45:0/3988546249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd01064b0 0x7f1cd019c2e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:05:58.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 >> 172.21.15.45:0/3988546249 conn(0x7f1cd0101a70 msgr2=0x7f1cd01027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:05:58.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 shutdown_connections 2023-10-17T01:05:58.707 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:58.234+0000 7f1cd6f86700 1 -- 172.21.15.45:0/3988546249 wait complete. 2023-10-17T01:05:58.707 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2023-10-17T01:05:58.707 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2023-10-17T01:05:58.932 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:58.798+0000 7f1ef27d0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-17T01:05:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:05:59 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3988546249' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-17T01:05:59.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:05:59 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:05:59.372+0000 7f1ef27d0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-17T01:06:00.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:00 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:00.246+0000 7f1ef27d0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-17T01:06:00.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:00 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:00.354+0000 7f1ef27d0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-17T01:06:01.708 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:01.348+0000 7f1ef27d0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:06:01.708 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:01.476+0000 7f1ef27d0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-17T01:06:02.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:01.707+0000 7f1ef27d0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:06:02.369 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:02.086+0000 7f1ef27d0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-17T01:06:02.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:02.369+0000 7f1ef27d0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-17T01:06:02.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:02.571+0000 7f1ef27d0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-17T01:06:03.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:02.928+0000 7f1ef27d0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-17T01:06:04.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:03 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:03.900+0000 7f1ef27d0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-17T01:06:04.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:04.081+0000 7f1ef27d0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-17T01:06:04.787 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:04.343+0000 7f1ef27d0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-17T01:06:04.788 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:04.499+0000 7f1ef27d0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-17T01:06:05.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:04.787+0000 7f1ef27d0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-17T01:06:06.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:05.895+0000 7f1ef27d0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-17T01:06:06.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: Active manager daemon x restarted 2023-10-17T01:06:06.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: Activating manager daemon x 2023-10-17T01:06:06.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: osdmap e2: 0 total, 0 up, 0 in 2023-10-17T01:06:06.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: mgrmap e6: x(active, starting, since 0.00401849s) 2023-10-17T01:06:06.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:06:06.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-17T01:06:06.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-17T01:06:06.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-17T01:06:06.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-17T01:06:06.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: Manager daemon x is now available 2023-10-17T01:06:06.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:06.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:06.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:06.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:05 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: Found migration_current of "None". Setting to last migration. 2023-10-17T01:06:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-17T01:06:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-17T01:06:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:06 smithi045 ceph-mon[98474]: mgrmap e7: x(active, since 1.00767s) 2023-10-17T01:06:07.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:06:07.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-17T01:06:07.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-17T01:06:07.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:06:07.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.194+0000 7fd4aa198700 1 Processor -- start 2023-10-17T01:06:07.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.194+0000 7fd4aa198700 1 -- start start 2023-10-17T01:06:07.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.194+0000 7fd4aa198700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a4106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:07.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.194+0000 7fd4aa198700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4106ce0 con 0x7fd4a41062f0 2023-10-17T01:06:07.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.195+0000 7fd4a9196700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a4106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:07.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.195+0000 7fd4a9196700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a4106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34438/0 (socket says 172.21.15.45:34438) 2023-10-17T01:06:07.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.195+0000 7fd4a9196700 1 -- 172.21.15.45:0/1977937594 learned_addr learned my addr 172.21.15.45:0/1977937594 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:07.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.195+0000 7fd4a9196700 1 -- 172.21.15.45:0/1977937594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a41074f0 con 0x7fd4a41062f0 2023-10-17T01:06:07.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.195+0000 7fd4a9196700 1 --2- 172.21.15.45:0/1977937594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a4106710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098a0 tx=0x7fd4a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eee6f16b3d12fb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:07.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd49bfff700 1 -- 172.21.15.45:0/1977937594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a0004160 con 0x7fd4a41062f0 2023-10-17T01:06:07.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd49bfff700 1 -- 172.21.15.45:0/1977937594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4a00042c0 con 0x7fd4a41062f0 2023-10-17T01:06:07.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd49bfff700 1 -- 172.21.15.45:0/1977937594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a0011a00 con 0x7fd4a41062f0 2023-10-17T01:06:07.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd4aa198700 1 -- 172.21.15.45:0/1977937594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 msgr2=0x7fd4a4106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:07.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd4aa198700 1 --2- 172.21.15.45:0/1977937594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a4106710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098a0 tx=0x7fd4a0009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd4aa198700 1 -- 172.21.15.45:0/1977937594 shutdown_connections 2023-10-17T01:06:07.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd4aa198700 1 --2- 172.21.15.45:0/1977937594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a4106710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.196+0000 7fd4aa198700 1 -- 172.21.15.45:0/1977937594 >> 172.21.15.45:0/1977937594 conn(0x7fd4a41018f0 msgr2=0x7fd4a4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:07.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.197+0000 7fd4aa198700 1 -- 172.21.15.45:0/1977937594 shutdown_connections 2023-10-17T01:06:07.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.197+0000 7fd4aa198700 1 -- 172.21.15.45:0/1977937594 wait complete. 2023-10-17T01:06:07.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4aa198700 1 Processor -- start 2023-10-17T01:06:07.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4aa198700 1 -- start start 2023-10-17T01:06:07.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4aa198700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a407bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:07.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4aa198700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a407c180 con 0x7fd4a41062f0 2023-10-17T01:06:07.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4a9196700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a407bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:07.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4a9196700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a407bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34442/0 (socket says 172.21.15.45:34442) 2023-10-17T01:06:07.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.198+0000 7fd4a9196700 1 -- 172.21.15.45:0/896712528 learned_addr learned my addr 172.21.15.45:0/896712528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:07.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.199+0000 7fd4a9196700 1 -- 172.21.15.45:0/896712528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009550 con 0x7fd4a41062f0 2023-10-17T01:06:07.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.199+0000 7fd4a9196700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a407bc40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009130 tx=0x7fd4a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:07.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.199+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a0003ab0 con 0x7fd4a41062f0 2023-10-17T01:06:07.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.199+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4a0003c10 con 0x7fd4a41062f0 2023-10-17T01:06:07.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.199+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4a00247d0 con 0x7fd4a41062f0 2023-10-17T01:06:07.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.199+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a407a290 con 0x7fd4a41062f0 2023-10-17T01:06:07.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.200+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a407a730 con 0x7fd4a41062f0 2023-10-17T01:06:07.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.201+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fd4a001abb0 con 0x7fd4a41062f0 2023-10-17T01:06:07.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.201+0000 7fd49a7fc700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:07.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.201+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 --> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd49003b120 con 0x7fd490038550 2023-10-17T01:06:07.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.201+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd4a004c1d0 con 0x7fd4a41062f0 2023-10-17T01:06:07.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.201+0000 7fd4a8995700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:06:07.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.201+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:06:07.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.402+0000 7fd4a8995700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:06:07.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.402+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-17T01:06:07.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.803+0000 7fd4a8995700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:06:07.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:05:59.803+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-17T01:06:07.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:00.604+0000 7fd4a8995700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:06:07.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:00.604+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-17T01:06:07.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:02.205+0000 7fd4a8995700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:06:07.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:02.206+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-17T01:06:07.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:05.409+0000 7fd4a8995700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3224370568 2023-10-17T01:06:07.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:05.409+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-17T01:06:07.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:05.900+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) v1 ==== 45189+0+0 (secure 0 0 0) 0x7fd4a002d770 con 0x7fd4a41062f0 2023-10-17T01:06:07.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:05.901+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 msgr2=0x7fd49003aa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:06:07.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:05.901+0000 7fd49a7fc700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.902+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) v1 ==== 45316+0+0 (secure 0 0 0) 0x7fd4a0016d40 con 0x7fd4a41062f0 2023-10-17T01:06:07.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.903+0000 7fd49a7fc700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd49003bf50 0x7fd49003e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:07.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.903+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd49003b120 con 0x7fd49003bf50 2023-10-17T01:06:07.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.903+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd49003bf50 0x7fd49003e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:07.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.904+0000 7fd4a8995700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd49003bf50 0x7fd49003e340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd494003a10 tx=0x7fd4940092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:07.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.905+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fd49003b120 con 0x7fd49003bf50 2023-10-17T01:06:07.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.912+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd4a407b7e0 con 0x7fd49003bf50 2023-10-17T01:06:07.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.912+0000 7fd49a7fc700 1 -- 172.21.15.45:0/896712528 <== mgr.14118 v2:172.21.15.45:6800/1775908551 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd4a407b7e0 con 0x7fd49003bf50 2023-10-17T01:06:07.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd49003bf50 msgr2=0x7fd49003e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:07.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd49003bf50 0x7fd49003e340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd494003a10 tx=0x7fd4940092b0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 msgr2=0x7fd4a407bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:07.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a407bc40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009130 tx=0x7fd4a0011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 shutdown_connections 2023-10-17T01:06:07.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd49003bf50 0x7fd49003e340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a41062f0 0x7fd4a407bc40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 --2- 172.21.15.45:0/896712528 >> [v2:172.21.15.45:6800/3224370568,v1:172.21.15.45:6801/3224370568] conn(0x7fd490038550 0x7fd49003aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:07.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 >> 172.21.15.45:0/896712528 conn(0x7fd4a41018f0 msgr2=0x7fd4a4102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:07.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 shutdown_connections 2023-10-17T01:06:07.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:06.913+0000 7fd4aa198700 1 -- 172.21.15.45:0/896712528 wait complete. 2023-10-17T01:06:07.414 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2023-10-17T01:06:07.414 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2023-10-17T01:06:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:07 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:06] ENGINE Bus STARTING 2023-10-17T01:06:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:07 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:06] ENGINE Serving on http://172.21.15.45:8765 2023-10-17T01:06:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:07 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:06] ENGINE Serving on https://172.21.15.45:7150 2023-10-17T01:06:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:07 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:06] ENGINE Bus STARTED 2023-10-17T01:06:08.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:07 smithi045 ceph-mon[98474]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-17T01:06:08.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:07 smithi045 ceph-mon[98474]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-17T01:06:08.573 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.923+0000 7f2ae04c4700 1 Processor -- start 2023-10-17T01:06:08.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.923+0000 7f2ae04c4700 1 -- start start 2023-10-17T01:06:08.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.923+0000 7f2ae04c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad807a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:08.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.924+0000 7f2ae04c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad807a8e0 con 0x7f2ad807beb0 2023-10-17T01:06:08.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.924+0000 7f2ade260700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad807a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:08.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.924+0000 7f2ade260700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad807a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43164/0 (socket says 172.21.15.45:43164) 2023-10-17T01:06:08.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.924+0000 7f2ade260700 1 -- 172.21.15.45:0/2658537338 learned_addr learned my addr 172.21.15.45:0/2658537338 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:08.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.924+0000 7f2ade260700 1 -- 172.21.15.45:0/2658537338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad807aa20 con 0x7f2ad807beb0 2023-10-17T01:06:08.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.925+0000 7f2ade260700 1 --2- 172.21.15.45:0/2658537338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad807a310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40098a0 tx=0x7f2ad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4560d18599352eda server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:08.581 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.925+0000 7f2add25e700 1 -- 172.21.15.45:0/2658537338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad4004160 con 0x7f2ad807beb0 2023-10-17T01:06:08.581 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.925+0000 7f2add25e700 1 -- 172.21.15.45:0/2658537338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ad40042c0 con 0x7f2ad807beb0 2023-10-17T01:06:08.581 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.925+0000 7f2add25e700 1 -- 172.21.15.45:0/2658537338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad4011a00 con 0x7f2ad807beb0 2023-10-17T01:06:08.581 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.925+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2658537338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 msgr2=0x7f2ad807a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:08.582 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.925+0000 7f2ae04c4700 1 --2- 172.21.15.45:0/2658537338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad807a310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40098a0 tx=0x7f2ad4009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:08.582 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.926+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2658537338 shutdown_connections 2023-10-17T01:06:08.582 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.926+0000 7f2ae04c4700 1 --2- 172.21.15.45:0/2658537338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad807a310 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:08.582 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.926+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2658537338 >> 172.21.15.45:0/2658537338 conn(0x7f2ad8101a30 msgr2=0x7f2ad8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:08.582 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.926+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2658537338 shutdown_connections 2023-10-17T01:06:08.583 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.926+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2658537338 wait complete. 2023-10-17T01:06:08.583 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ae04c4700 1 Processor -- start 2023-10-17T01:06:08.583 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ae04c4700 1 -- start start 2023-10-17T01:06:08.583 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ae04c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:08.583 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ae04c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad819c870 con 0x7f2ad807beb0 2023-10-17T01:06:08.584 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ade260700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:08.584 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ade260700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43168/0 (socket says 172.21.15.45:43168) 2023-10-17T01:06:08.584 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.927+0000 7f2ade260700 1 -- 172.21.15.45:0/2724172005 learned_addr learned my addr 172.21.15.45:0/2724172005 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:08.584 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2ade260700 1 -- 172.21.15.45:0/2724172005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4009550 con 0x7f2ad807beb0 2023-10-17T01:06:08.584 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2ade260700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad819c330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009130 tx=0x7f2ad4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:08.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad4003ab0 con 0x7f2ad807beb0 2023-10-17T01:06:08.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ad4003c10 con 0x7f2ad807beb0 2023-10-17T01:06:08.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad40247d0 con 0x7f2ad807beb0 2023-10-17T01:06:08.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad819ca70 con 0x7f2ad807beb0 2023-10-17T01:06:08.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.928+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad819cf10 con 0x7f2ad807beb0 2023-10-17T01:06:08.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.929+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) v1 ==== 45316+0+0 (secure 0 0 0) 0x7f2ad401abb0 con 0x7f2ad807beb0 2023-10-17T01:06:08.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.929+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad810c840 con 0x7f2ad807beb0 2023-10-17T01:06:08.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.930+0000 7f2acf7fe700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ac4038420 0x7f2ac403a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:08.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.930+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2ad404c3f0 con 0x7f2ad807beb0 2023-10-17T01:06:08.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.930+0000 7f2adda5f700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ac4038420 0x7f2ac403a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.931+0000 7f2adda5f700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ac4038420 0x7f2ac403a8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8006fd0 tx=0x7f2ac8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.935+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2ad4018530 con 0x7f2ad807beb0 2023-10-17T01:06:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:07.953+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f2ad402d770 con 0x7f2ad807beb0 2023-10-17T01:06:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.164+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ad8103d00 con 0x7f2ac4038420 2023-10-17T01:06:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.171+0000 7f2acf7fe700 1 -- 172.21.15.45:0/2724172005 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2ad8103d00 con 0x7f2ac4038420 2023-10-17T01:06:08.588 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ac4038420 msgr2=0x7f2ac403a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:08.588 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ac4038420 0x7f2ac403a8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8006fd0 tx=0x7f2ac8006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:08.588 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 msgr2=0x7f2ad819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:08.588 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad819c330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009130 tx=0x7f2ad4011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:08.588 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 shutdown_connections 2023-10-17T01:06:08.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ac4038420 0x7f2ac403a8e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:08.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 --2- 172.21.15.45:0/2724172005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad807beb0 0x7f2ad819c330 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:08.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 >> 172.21.15.45:0/2724172005 conn(0x7f2ad8101a30 msgr2=0x7f2ad81030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:08.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 shutdown_connections 2023-10-17T01:06:08.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:08.174+0000 7f2ae04c4700 1 -- 172.21.15.45:0/2724172005 wait complete. 2023-10-17T01:06:09.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:08 smithi045 ceph-mon[98474]: mgrmap e8: x(active, since 2s) 2023-10-17T01:06:09.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:08 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:09.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:08 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:09.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-17T01:06:09.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.111+0000 7f574ae45700 1 Processor -- start 2023-10-17T01:06:09.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.111+0000 7f574ae45700 1 -- start start 2023-10-17T01:06:09.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.111+0000 7f574ae45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:09.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.111+0000 7f574ae45700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744106ce0 con 0x7f57441062f0 2023-10-17T01:06:09.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.112+0000 7f5749e43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:09.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.112+0000 7f5749e43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43176/0 (socket says 172.21.15.45:43176) 2023-10-17T01:06:09.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.112+0000 7f5749e43700 1 -- 172.21.15.45:0/2035022797 learned_addr learned my addr 172.21.15.45:0/2035022797 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:09.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.112+0000 7f5749e43700 1 -- 172.21.15.45:0/2035022797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57441074f0 con 0x7f57441062f0 2023-10-17T01:06:09.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.112+0000 7f5749e43700 1 --2- 172.21.15.45:0/2035022797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744106710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57400098a0 tx=0x7f5740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bcdcdc97a00995 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:09.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.112+0000 7f5748e41700 1 -- 172.21.15.45:0/2035022797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5740004160 con 0x7f57441062f0 2023-10-17T01:06:09.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f5748e41700 1 -- 172.21.15.45:0/2035022797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57400042c0 con 0x7f57441062f0 2023-10-17T01:06:09.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f5748e41700 1 -- 172.21.15.45:0/2035022797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5740011a00 con 0x7f57441062f0 2023-10-17T01:06:09.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 -- 172.21.15.45:0/2035022797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 msgr2=0x7f5744106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:09.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 --2- 172.21.15.45:0/2035022797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744106710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57400098a0 tx=0x7f5740009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:09.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 -- 172.21.15.45:0/2035022797 shutdown_connections 2023-10-17T01:06:09.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 --2- 172.21.15.45:0/2035022797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744106710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:09.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 -- 172.21.15.45:0/2035022797 >> 172.21.15.45:0/2035022797 conn(0x7f57441018f0 msgr2=0x7f5744103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:09.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 -- 172.21.15.45:0/2035022797 shutdown_connections 2023-10-17T01:06:09.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.113+0000 7f574ae45700 1 -- 172.21.15.45:0/2035022797 wait complete. 2023-10-17T01:06:09.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.115+0000 7f574ae45700 1 Processor -- start 2023-10-17T01:06:09.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.115+0000 7f574ae45700 1 -- start start 2023-10-17T01:06:09.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.115+0000 7f574ae45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.115+0000 7f574ae45700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574419a4e0 con 0x7f57441062f0 2023-10-17T01:06:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f5749e43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f5749e43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43184/0 (socket says 172.21.15.45:43184) 2023-10-17T01:06:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f5749e43700 1 -- 172.21.15.45:0/4058237627 learned_addr learned my addr 172.21.15.45:0/4058237627 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f5749e43700 1 -- 172.21.15.45:0/4058237627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740009550 con 0x7f57441062f0 2023-10-17T01:06:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f5749e43700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744199fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5740009130 tx=0x7f5740011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:09.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5740003ab0 con 0x7f57441062f0 2023-10-17T01:06:09.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.116+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5740003c10 con 0x7f57441062f0 2023-10-17T01:06:09.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.117+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574419a6e0 con 0x7f57441062f0 2023-10-17T01:06:09.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.117+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57400247d0 con 0x7f57441062f0 2023-10-17T01:06:09.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.117+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574419ab80 con 0x7f57441062f0 2023-10-17T01:06:09.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.118+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5744062380 con 0x7f57441062f0 2023-10-17T01:06:09.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.118+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f574001abb0 con 0x7f57441062f0 2023-10-17T01:06:09.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.118+0000 7f573affd700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f5730038500 0x7f573003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:09.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.119+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f574004c6a0 con 0x7f57441062f0 2023-10-17T01:06:09.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.119+0000 7f5749642700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f5730038500 0x7f573003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:09.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.119+0000 7f5749642700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f5730038500 0x7f573003a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5734006fd0 tx=0x7f5734006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:09.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.124+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f5740018ba0 con 0x7f57441062f0 2023-10-17T01:06:09.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.335+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5744103bb0 con 0x7f5730038500 2023-10-17T01:06:09.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.335+0000 7f573affd700 1 -- 172.21.15.45:0/4058237627 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5744103bb0 con 0x7f5730038500 2023-10-17T01:06:09.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f5730038500 msgr2=0x7f573003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:09.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f5730038500 0x7f573003a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5734006fd0 tx=0x7f5734006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:09.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 msgr2=0x7f5744199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:09.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744199fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5740009130 tx=0x7f5740011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:09.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 shutdown_connections 2023-10-17T01:06:09.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f5730038500 0x7f573003a9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:09.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 --2- 172.21.15.45:0/4058237627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57441062f0 0x7f5744199fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:09.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 >> 172.21.15.45:0/4058237627 conn(0x7f57441018f0 msgr2=0x7f5744102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:09.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 shutdown_connections 2023-10-17T01:06:09.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:09.338+0000 7f574ae45700 1 -- 172.21.15.45:0/4058237627 wait complete. 2023-10-17T01:06:09.774 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2023-10-17T01:06:10.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:09 smithi045 ceph-mon[98474]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-mon[98474]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:11.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:11.052 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: Generating public/private rsa key pair. 2023-10-17T01:06:11.052 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: Your identification has been saved in /tmp/tmpgvk2zgky/key. 2023-10-17T01:06:11.052 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: Your public key has been saved in /tmp/tmpgvk2zgky/key.pub. 2023-10-17T01:06:11.053 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: The key fingerprint is: 2023-10-17T01:06:11.053 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: SHA256:XkmJ9KVxTzP4DysUWO8lGzG/vCT+oX5v4/neVpdEYwk ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:06:11.053 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: The key's randomart image is: 2023-10-17T01:06:11.053 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: +---[RSA 3072]----+ 2023-10-17T01:06:11.053 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | . .o+EB .| 2023-10-17T01:06:11.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | . o.*o+ @ | 2023-10-17T01:06:11.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | . = oB +| 2023-10-17T01:06:11.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | . ...+*.| 2023-10-17T01:06:11.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | S o. .+B.| 2023-10-17T01:06:11.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | . . o +.=| 2023-10-17T01:06:11.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | . o oo| 2023-10-17T01:06:11.055 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | oo*| 2023-10-17T01:06:11.055 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: | .ooBO| 2023-10-17T01:06:11.055 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: +----[SHA256]-----+ 2023-10-17T01:06:11.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.328+0000 7fe059b30700 1 Processor -- start 2023-10-17T01:06:11.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.328+0000 7fe059b30700 1 -- start start 2023-10-17T01:06:11.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.328+0000 7fe059b30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05407a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:11.144 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.328+0000 7fe059b30700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05407a880 con 0x7fe05407be50 2023-10-17T01:06:11.145 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.329+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05407a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:11.145 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.329+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05407a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43186/0 (socket says 172.21.15.45:43186) 2023-10-17T01:06:11.145 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.329+0000 7fe0537fe700 1 -- 172.21.15.45:0/2245157878 learned_addr learned my addr 172.21.15.45:0/2245157878 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:11.145 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.329+0000 7fe0537fe700 1 -- 172.21.15.45:0/2245157878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05407a9c0 con 0x7fe05407be50 2023-10-17T01:06:11.145 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.329+0000 7fe0537fe700 1 --2- 172.21.15.45:0/2245157878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05407a2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe0440098a0 tx=0x7fe044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30a18c6eb9cc9f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:11.146 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe0527fc700 1 -- 172.21.15.45:0/2245157878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe044004160 con 0x7fe05407be50 2023-10-17T01:06:11.146 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe0527fc700 1 -- 172.21.15.45:0/2245157878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0440042c0 con 0x7fe05407be50 2023-10-17T01:06:11.146 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe0527fc700 1 -- 172.21.15.45:0/2245157878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe044011a00 con 0x7fe05407be50 2023-10-17T01:06:11.146 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 -- 172.21.15.45:0/2245157878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 msgr2=0x7fe05407a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:11.146 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 --2- 172.21.15.45:0/2245157878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05407a2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe0440098a0 tx=0x7fe044009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:11.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 -- 172.21.15.45:0/2245157878 shutdown_connections 2023-10-17T01:06:11.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 --2- 172.21.15.45:0/2245157878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05407a2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:11.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 -- 172.21.15.45:0/2245157878 >> 172.21.15.45:0/2245157878 conn(0x7fe054101a70 msgr2=0x7fe054103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:11.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 -- 172.21.15.45:0/2245157878 shutdown_connections 2023-10-17T01:06:11.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.330+0000 7fe059b30700 1 -- 172.21.15.45:0/2245157878 wait complete. 2023-10-17T01:06:11.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.331+0000 7fe059b30700 1 Processor -- start 2023-10-17T01:06:11.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe059b30700 1 -- start start 2023-10-17T01:06:11.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe059b30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:11.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe059b30700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419c7b0 con 0x7fe05407be50 2023-10-17T01:06:11.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:11.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43196/0 (socket says 172.21.15.45:43196) 2023-10-17T01:06:11.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe0537fe700 1 -- 172.21.15.45:0/2471095669 learned_addr learned my addr 172.21.15.45:0/2471095669 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:11.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.332+0000 7fe0537fe700 1 -- 172.21.15.45:0/2471095669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044009550 con 0x7fe05407be50 2023-10-17T01:06:11.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.333+0000 7fe0537fe700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05419c270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe044006af0 tx=0x7fe044011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:11.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.333+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe044003a40 con 0x7fe05407be50 2023-10-17T01:06:11.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.333+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe044003ba0 con 0x7fe05407be50 2023-10-17T01:06:11.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.333+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe04401a6e0 con 0x7fe05407be50 2023-10-17T01:06:11.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.333+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05419c9b0 con 0x7fe05407be50 2023-10-17T01:06:11.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.333+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05419ce50 con 0x7fe05407be50 2023-10-17T01:06:11.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.334+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fe044024a60 con 0x7fe05407be50 2023-10-17T01:06:11.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.334+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05410c860 con 0x7fe05407be50 2023-10-17T01:06:11.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.335+0000 7fe050ff9700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe03c038500 0x7fe03c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:11.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.335+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe04404c510 con 0x7fe05407be50 2023-10-17T01:06:11.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.335+0000 7fe052ffd700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe03c038500 0x7fe03c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:11.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.335+0000 7fe052ffd700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe03c038500 0x7fe03c03a9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe048006fd0 tx=0x7fe048006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.340+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe044024430 con 0x7fe05407be50 2023-10-17T01:06:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.550+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05419fc90 con 0x7fe03c038500 2023-10-17T01:06:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.748+0000 7fe050ff9700 1 -- 172.21.15.45:0/2471095669 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe05419fc90 con 0x7fe03c038500 2023-10-17T01:06:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe03c038500 msgr2=0x7fe03c03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe03c038500 0x7fe03c03a9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe048006fd0 tx=0x7fe048006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 msgr2=0x7fe05419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:11.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05419c270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe044006af0 tx=0x7fe044011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:11.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 shutdown_connections 2023-10-17T01:06:11.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe03c038500 0x7fe03c03a9c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:11.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 --2- 172.21.15.45:0/2471095669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05407be50 0x7fe05419c270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:11.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 >> 172.21.15.45:0/2471095669 conn(0x7fe054101a70 msgr2=0x7fe054102810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:11.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 shutdown_connections 2023-10-17T01:06:11.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:10.750+0000 7fe059b30700 1 -- 172.21.15.45:0/2471095669 wait complete. 2023-10-17T01:06:12.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:11 smithi045 ceph-mon[98474]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:12.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:11 smithi045 ceph-mon[98474]: Generating ssh key... 2023-10-17T01:06:12.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:06:12.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fed0e700 1 Processor -- start 2023-10-17T01:06:12.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fed0e700 1 -- start start 2023-10-17T01:06:12.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fed0e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f81068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:12.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fed0e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8106ea0 con 0x7fe3f81064b0 2023-10-17T01:06:12.329 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fcaaa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f81068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:12.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fcaaa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f81068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43200/0 (socket says 172.21.15.45:43200) 2023-10-17T01:06:12.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.668+0000 7fe3fcaaa700 1 -- 172.21.15.45:0/204318289 learned_addr learned my addr 172.21.15.45:0/204318289 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:12.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.669+0000 7fe3fcaaa700 1 -- 172.21.15.45:0/204318289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f81076b0 con 0x7fe3f81064b0 2023-10-17T01:06:12.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.669+0000 7fe3fcaaa700 1 --2- 172.21.15.45:0/204318289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f81068d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0098a0 tx=0x7fe3ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6696bc40c9e91d9e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:12.330 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.669+0000 7fe3f77fe700 1 -- 172.21.15.45:0/204318289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec004160 con 0x7fe3f81064b0 2023-10-17T01:06:12.331 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.669+0000 7fe3f77fe700 1 -- 172.21.15.45:0/204318289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3ec0042c0 con 0x7fe3f81064b0 2023-10-17T01:06:12.331 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.669+0000 7fe3f77fe700 1 -- 172.21.15.45:0/204318289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec011a00 con 0x7fe3f81064b0 2023-10-17T01:06:12.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 -- 172.21.15.45:0/204318289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 msgr2=0x7fe3f81068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:12.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 --2- 172.21.15.45:0/204318289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f81068d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0098a0 tx=0x7fe3ec009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:12.335 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 -- 172.21.15.45:0/204318289 shutdown_connections 2023-10-17T01:06:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 --2- 172.21.15.45:0/204318289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f81068d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 -- 172.21.15.45:0/204318289 >> 172.21.15.45:0/204318289 conn(0x7fe3f8101a70 msgr2=0x7fe3f8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 -- 172.21.15.45:0/204318289 shutdown_connections 2023-10-17T01:06:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.670+0000 7fe3fed0e700 1 -- 172.21.15.45:0/204318289 wait complete. 2023-10-17T01:06:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.672+0000 7fe3fed0e700 1 Processor -- start 2023-10-17T01:06:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.672+0000 7fe3fed0e700 1 -- start start 2023-10-17T01:06:12.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.672+0000 7fe3fed0e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:12.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.672+0000 7fe3fed0e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f819a610 con 0x7fe3f81064b0 2023-10-17T01:06:12.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.672+0000 7fe3fcaaa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:12.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.672+0000 7fe3fcaaa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43216/0 (socket says 172.21.15.45:43216) 2023-10-17T01:06:12.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3fcaaa700 1 -- 172.21.15.45:0/1695943041 learned_addr learned my addr 172.21.15.45:0/1695943041 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:12.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3fcaaa700 1 -- 172.21.15.45:0/1695943041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec009550 con 0x7fe3f81064b0 2023-10-17T01:06:12.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3fcaaa700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f819a0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006af0 tx=0x7fe3ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:12.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec003a40 con 0x7fe3f81064b0 2023-10-17T01:06:12.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3ec003ba0 con 0x7fe3f81064b0 2023-10-17T01:06:12.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f819a810 con 0x7fe3f81064b0 2023-10-17T01:06:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec01a700 con 0x7fe3f81064b0 2023-10-17T01:06:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.673+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f819acb0 con 0x7fe3f81064b0 2023-10-17T01:06:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.674+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fe3ec024ad0 con 0x7fe3f81064b0 2023-10-17T01:06:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.675+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f8062380 con 0x7fe3f81064b0 2023-10-17T01:06:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.675+0000 7fe3f5ffb700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe3e8038550 0x7fe3e803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:12.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.675+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe3ec04c430 con 0x7fe3f81064b0 2023-10-17T01:06:12.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.675+0000 7fe3f7fff700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe3e8038550 0x7fe3e803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:12.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.676+0000 7fe3f7fff700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe3e8038550 0x7fe3e803aa10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4006fd0 tx=0x7fe3e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:12.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.680+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe3ec018b80 con 0x7fe3f81064b0 2023-10-17T01:06:12.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.890+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3f8102ef0 con 0x7fe3e8038550 2023-10-17T01:06:12.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.891+0000 7fe3f5ffb700 1 -- 172.21.15.45:0/1695943041 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe3f8102ef0 con 0x7fe3e8038550 2023-10-17T01:06:12.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe3e8038550 msgr2=0x7fe3e803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:12.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe3e8038550 0x7fe3e803aa10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4006fd0 tx=0x7fe3e4006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:12.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 msgr2=0x7fe3f819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f819a0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006af0 tx=0x7fe3ec011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 shutdown_connections 2023-10-17T01:06:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fe3e8038550 0x7fe3e803aa10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 --2- 172.21.15.45:0/1695943041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3f81064b0 0x7fe3f819a0d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 >> 172.21.15.45:0/1695943041 conn(0x7fe3f8101a70 msgr2=0x7fe3f81027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:12.343 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 shutdown_connections 2023-10-17T01:06:12.343 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:11.894+0000 7fe3fed0e700 1 -- 172.21.15.45:0/1695943041 wait complete. 2023-10-17T01:06:12.343 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-17T01:06:12.343 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2023-10-17T01:06:12.343 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2023-10-17T01:06:13.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:12 smithi045 ceph-mon[98474]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:14.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:13 smithi045 ceph-mon[98474]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:15.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:14 smithi045 ceph-mon[98474]: Deploying cephadm binary to smithi045 2023-10-17T01:06:16.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2023-10-17T01:06:16.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.855+0000 7f86e52df700 1 Processor -- start 2023-10-17T01:06:16.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.855+0000 7f86e52df700 1 -- start start 2023-10-17T01:06:16.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.855+0000 7f86e52df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e01068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:16.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.855+0000 7f86e52df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0106e80 con 0x7f86e0106490 2023-10-17T01:06:16.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.856+0000 7f86deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e01068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:16.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.856+0000 7f86deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e01068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43218/0 (socket says 172.21.15.45:43218) 2023-10-17T01:06:16.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.856+0000 7f86deffd700 1 -- 172.21.15.45:0/580706247 learned_addr learned my addr 172.21.15.45:0/580706247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:16.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.856+0000 7f86deffd700 1 -- 172.21.15.45:0/580706247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0107690 con 0x7f86e0106490 2023-10-17T01:06:16.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.856+0000 7f86deffd700 1 --2- 172.21.15.45:0/580706247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e01068b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86d00098a0 tx=0x7f86d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecaf1930049d7cb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:16.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86ddffb700 1 -- 172.21.15.45:0/580706247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86d0004160 con 0x7f86e0106490 2023-10-17T01:06:16.732 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86ddffb700 1 -- 172.21.15.45:0/580706247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86d00042c0 con 0x7f86e0106490 2023-10-17T01:06:16.732 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86ddffb700 1 -- 172.21.15.45:0/580706247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86d0011a00 con 0x7f86e0106490 2023-10-17T01:06:16.732 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 -- 172.21.15.45:0/580706247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 msgr2=0x7f86e01068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:16.733 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 --2- 172.21.15.45:0/580706247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e01068b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86d00098a0 tx=0x7f86d0009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:16.733 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 -- 172.21.15.45:0/580706247 shutdown_connections 2023-10-17T01:06:16.733 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 --2- 172.21.15.45:0/580706247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e01068b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:16.734 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 -- 172.21.15.45:0/580706247 >> 172.21.15.45:0/580706247 conn(0x7f86e0101a30 msgr2=0x7f86e0103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:16.734 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 -- 172.21.15.45:0/580706247 shutdown_connections 2023-10-17T01:06:16.734 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.857+0000 7f86e52df700 1 -- 172.21.15.45:0/580706247 wait complete. 2023-10-17T01:06:16.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.858+0000 7f86e52df700 1 Processor -- start 2023-10-17T01:06:16.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86e52df700 1 -- start start 2023-10-17T01:06:16.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86e52df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:16.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86e52df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e019c8d0 con 0x7f86e0106490 2023-10-17T01:06:16.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:16.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e019c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43222/0 (socket says 172.21.15.45:43222) 2023-10-17T01:06:16.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86deffd700 1 -- 172.21.15.45:0/663680534 learned_addr learned my addr 172.21.15.45:0/663680534 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:16.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.859+0000 7f86deffd700 1 -- 172.21.15.45:0/663680534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0009550 con 0x7f86e0106490 2023-10-17T01:06:16.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.860+0000 7f86deffd700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e019c390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009130 tx=0x7f86d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:16.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.860+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86d0003ab0 con 0x7f86e0106490 2023-10-17T01:06:16.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.860+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86d0003c10 con 0x7f86e0106490 2023-10-17T01:06:16.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.860+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e019cad0 con 0x7f86e0106490 2023-10-17T01:06:16.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.860+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86d00247d0 con 0x7f86e0106490 2023-10-17T01:06:16.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.860+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e019cf70 con 0x7f86e0106490 2023-10-17T01:06:16.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.861+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f86d001abb0 con 0x7f86e0106490 2023-10-17T01:06:16.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.861+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e0062380 con 0x7f86e0106490 2023-10-17T01:06:16.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.862+0000 7f86c7fff700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f86c8038550 0x7f86c803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:16.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.862+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f86d004c520 con 0x7f86e0106490 2023-10-17T01:06:16.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.862+0000 7f86de7fc700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f86c8038550 0x7f86c803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:16.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.863+0000 7f86de7fc700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f86c8038550 0x7f86c803aa10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86d4006fd0 tx=0x7f86d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:16.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:12.867+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f86d0018530 con 0x7f86e0106490 2023-10-17T01:06:16.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:13.081+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) v1 -- 0x7f86e010a890 con 0x7f86c8038550 2023-10-17T01:06:16.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.302+0000 7f86c7fff700 1 -- 172.21.15.45:0/663680534 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f86e010a890 con 0x7f86c8038550 2023-10-17T01:06:16.740 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f86c8038550 msgr2=0x7f86c803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:16.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f86c8038550 0x7f86c803aa10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86d4006fd0 tx=0x7f86d4006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:16.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 msgr2=0x7f86e019c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:16.741 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e019c390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009130 tx=0x7f86d0011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:16.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 shutdown_connections 2023-10-17T01:06:16.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f86c8038550 0x7f86c803aa10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:16.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 --2- 172.21.15.45:0/663680534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e0106490 0x7f86e019c390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:16.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.305+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 >> 172.21.15.45:0/663680534 conn(0x7f86e0101a30 msgr2=0x7f86e0103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:16.742 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.306+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 shutdown_connections 2023-10-17T01:06:16.743 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:16.306+0000 7f86e52df700 1 -- 172.21.15.45:0/663680534 wait complete. 2023-10-17T01:06:16.743 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2023-10-17T01:06:17.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:17 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:17.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:17 smithi045 ceph-mon[98474]: Added host smithi045 2023-10-17T01:06:17.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:17 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:18.109 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-17T01:06:18.110 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.272+0000 7f42559c2700 1 Processor -- start 2023-10-17T01:06:18.110 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.272+0000 7f42559c2700 1 -- start start 2023-10-17T01:06:18.110 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.273+0000 7f42559c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f42501068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:18.111 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.273+0000 7f42559c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250106e80 con 0x7f4250106490 2023-10-17T01:06:18.111 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.273+0000 7f424effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f42501068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:18.111 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.273+0000 7f424effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f42501068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43232/0 (socket says 172.21.15.45:43232) 2023-10-17T01:06:18.111 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.273+0000 7f424effd700 1 -- 172.21.15.45:0/50749485 learned_addr learned my addr 172.21.15.45:0/50749485 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:18.112 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.273+0000 7f424effd700 1 -- 172.21.15.45:0/50749485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250107690 con 0x7f4250106490 2023-10-17T01:06:18.112 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.274+0000 7f424effd700 1 --2- 172.21.15.45:0/50749485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f42501068b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42400098a0 tx=0x7f4240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d77fd36cb5dd52 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:18.112 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.274+0000 7f424dffb700 1 -- 172.21.15.45:0/50749485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4240004160 con 0x7f4250106490 2023-10-17T01:06:18.112 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.274+0000 7f424dffb700 1 -- 172.21.15.45:0/50749485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f42400042c0 con 0x7f4250106490 2023-10-17T01:06:18.112 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.274+0000 7f424dffb700 1 -- 172.21.15.45:0/50749485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4240011a00 con 0x7f4250106490 2023-10-17T01:06:18.113 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.274+0000 7f42559c2700 1 -- 172.21.15.45:0/50749485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 msgr2=0x7f42501068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:18.113 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.274+0000 7f42559c2700 1 --2- 172.21.15.45:0/50749485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f42501068b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42400098a0 tx=0x7f4240009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:18.113 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.275+0000 7f42559c2700 1 -- 172.21.15.45:0/50749485 shutdown_connections 2023-10-17T01:06:18.113 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.275+0000 7f42559c2700 1 --2- 172.21.15.45:0/50749485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f42501068b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:18.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.275+0000 7f42559c2700 1 -- 172.21.15.45:0/50749485 >> 172.21.15.45:0/50749485 conn(0x7f4250101a70 msgr2=0x7f4250103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:18.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.275+0000 7f42559c2700 1 -- 172.21.15.45:0/50749485 shutdown_connections 2023-10-17T01:06:18.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.275+0000 7f42559c2700 1 -- 172.21.15.45:0/50749485 wait complete. 2023-10-17T01:06:18.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.276+0000 7f42559c2700 1 Processor -- start 2023-10-17T01:06:18.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.276+0000 7f42559c2700 1 -- start start 2023-10-17T01:06:18.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.276+0000 7f42559c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f425007bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:18.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.276+0000 7f42559c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425007c180 con 0x7f4250106490 2023-10-17T01:06:18.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.276+0000 7f424effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f425007bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:18.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.276+0000 7f424effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f425007bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43242/0 (socket says 172.21.15.45:43242) 2023-10-17T01:06:18.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f424effd700 1 -- 172.21.15.45:0/1561085111 learned_addr learned my addr 172.21.15.45:0/1561085111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:18.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f424effd700 1 -- 172.21.15.45:0/1561085111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4240009550 con 0x7f4250106490 2023-10-17T01:06:18.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f424effd700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f425007bc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4240009130 tx=0x7f4240011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:18.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4240003ab0 con 0x7f4250106490 2023-10-17T01:06:18.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4240003c10 con 0x7f4250106490 2023-10-17T01:06:18.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425007a290 con 0x7f4250106490 2023-10-17T01:06:18.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42400247d0 con 0x7f4250106490 2023-10-17T01:06:18.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.277+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425007a730 con 0x7f4250106490 2023-10-17T01:06:18.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.278+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f424001a430 con 0x7f4250106490 2023-10-17T01:06:18.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.279+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250062380 con 0x7f4250106490 2023-10-17T01:06:18.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.279+0000 7f42549c0700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f4238038550 0x7f423803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:18.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.279+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f424004c640 con 0x7f4250106490 2023-10-17T01:06:18.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.279+0000 7f424e7fc700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f4238038550 0x7f423803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:18.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.280+0000 7f424e7fc700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f4238038550 0x7f423803aa10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4244006fd0 tx=0x7f4244006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:18.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.285+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f4240018b40 con 0x7f4250106490 2023-10-17T01:06:18.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.518+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4250103d90 con 0x7f4238038550 2023-10-17T01:06:18.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.523+0000 7f42549c0700 1 -- 172.21.15.45:0/1561085111 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4250103d90 con 0x7f4238038550 2023-10-17T01:06:18.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f4238038550 msgr2=0x7f423803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:18.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f4238038550 0x7f423803aa10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4244006fd0 tx=0x7f4244006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:18.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 msgr2=0x7f425007bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:18.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f425007bc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4240009130 tx=0x7f4240011ad0 comp rx=0 tx=0).stop 2023-10-17T01:06:18.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 shutdown_connections 2023-10-17T01:06:18.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f4238038550 0x7f423803aa10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:18.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 --2- 172.21.15.45:0/1561085111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4250106490 0x7f425007bc40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:18.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 >> 172.21.15.45:0/1561085111 conn(0x7f4250101a70 msgr2=0x7f4250103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:18.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 shutdown_connections 2023-10-17T01:06:18.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:17.526+0000 7f42559c2700 1 -- 172.21.15.45:0/1561085111 wait complete. 2023-10-17T01:06:18.122 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2023-10-17T01:06:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:18 smithi045 ceph-mon[98474]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:18 smithi045 ceph-mon[98474]: Saving service mon spec with placement count:5 2023-10-17T01:06:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:18 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:18 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:19.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-17T01:06:19.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.723+0000 7fd542d69700 1 Processor -- start 2023-10-17T01:06:19.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.723+0000 7fd542d69700 1 -- start start 2023-10-17T01:06:19.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.724+0000 7fd542d69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:19.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.724+0000 7fd542d69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c07a8a0 con 0x7fd53c07be70 2023-10-17T01:06:19.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.724+0000 7fd540b05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:19.400 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.724+0000 7fd540b05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42238/0 (socket says 172.21.15.45:42238) 2023-10-17T01:06:19.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.724+0000 7fd540b05700 1 -- 172.21.15.45:0/1793735494 learned_addr learned my addr 172.21.15.45:0/1793735494 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:19.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.725+0000 7fd540b05700 1 -- 172.21.15.45:0/1793735494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c07a9e0 con 0x7fd53c07be70 2023-10-17T01:06:19.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.725+0000 7fd540b05700 1 --2- 172.21.15.45:0/1793735494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c07a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd5300098a0 tx=0x7fd530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75487f4a0042301 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:19.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.725+0000 7fd53b7fe700 1 -- 172.21.15.45:0/1793735494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd530004160 con 0x7fd53c07be70 2023-10-17T01:06:19.401 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.725+0000 7fd53b7fe700 1 -- 172.21.15.45:0/1793735494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5300042c0 con 0x7fd53c07be70 2023-10-17T01:06:19.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.725+0000 7fd53b7fe700 1 -- 172.21.15.45:0/1793735494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5300119b0 con 0x7fd53c07be70 2023-10-17T01:06:19.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 -- 172.21.15.45:0/1793735494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 msgr2=0x7fd53c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:19.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 --2- 172.21.15.45:0/1793735494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c07a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd5300098a0 tx=0x7fd530009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:19.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 -- 172.21.15.45:0/1793735494 shutdown_connections 2023-10-17T01:06:19.402 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 --2- 172.21.15.45:0/1793735494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c07a2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 -- 172.21.15.45:0/1793735494 >> 172.21.15.45:0/1793735494 conn(0x7fd53c101a30 msgr2=0x7fd53c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 -- 172.21.15.45:0/1793735494 shutdown_connections 2023-10-17T01:06:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.726+0000 7fd542d69700 1 -- 172.21.15.45:0/1793735494 wait complete. 2023-10-17T01:06:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.728+0000 7fd542d69700 1 Processor -- start 2023-10-17T01:06:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.728+0000 7fd542d69700 1 -- start start 2023-10-17T01:06:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.728+0000 7fd542d69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:19.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.728+0000 7fd542d69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19c880 con 0x7fd53c07be70 2023-10-17T01:06:19.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.729+0000 7fd540b05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:19.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.729+0000 7fd540b05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42250/0 (socket says 172.21.15.45:42250) 2023-10-17T01:06:19.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.729+0000 7fd540b05700 1 -- 172.21.15.45:0/3456849801 learned_addr learned my addr 172.21.15.45:0/3456849801 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:19.404 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.729+0000 7fd540b05700 1 -- 172.21.15.45:0/3456849801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530009550 con 0x7fd53c07be70 2023-10-17T01:06:19.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.729+0000 7fd540b05700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c19c340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd530003f00 tx=0x7fd530011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:19.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.730+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd530003b20 con 0x7fd53c07be70 2023-10-17T01:06:19.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.730+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd530003c80 con 0x7fd53c07be70 2023-10-17T01:06:19.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.730+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c19ca80 con 0x7fd53c07be70 2023-10-17T01:06:19.405 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.730+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd530019770 con 0x7fd53c07be70 2023-10-17T01:06:19.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.730+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c19cf20 con 0x7fd53c07be70 2023-10-17T01:06:19.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.731+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c10c8d0 con 0x7fd53c07be70 2023-10-17T01:06:19.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.732+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fd53001ab60 con 0x7fd53c07be70 2023-10-17T01:06:19.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.732+0000 7fd539ffb700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd52c038500 0x7fd52c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:19.406 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.732+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd53004c7b0 con 0x7fd53c07be70 2023-10-17T01:06:19.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.732+0000 7fd53bfff700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd52c038500 0x7fd52c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:19.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.733+0000 7fd53bfff700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd52c038500 0x7fd52c03a9c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd528006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:19.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.738+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd53001a430 con 0x7fd53c07be70 2023-10-17T01:06:19.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.977+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd53c19fd60 con 0x7fd52c038500 2023-10-17T01:06:19.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.982+0000 7fd539ffb700 1 -- 172.21.15.45:0/3456849801 <== mgr.14118 v2:172.21.15.45:6800/1775908551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd53c19fd60 con 0x7fd52c038500 2023-10-17T01:06:19.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd52c038500 msgr2=0x7fd52c03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:19.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd52c038500 0x7fd52c03a9c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd528006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:19.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 msgr2=0x7fd53c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:19.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c19c340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd530003f00 tx=0x7fd530011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:19.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 shutdown_connections 2023-10-17T01:06:19.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fd52c038500 0x7fd52c03a9c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:19.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 --2- 172.21.15.45:0/3456849801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd53c07be70 0x7fd53c19c340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:19.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 >> 172.21.15.45:0/3456849801 conn(0x7fd53c101a30 msgr2=0x7fd53c102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:19.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.985+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 shutdown_connections 2023-10-17T01:06:19.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:18.986+0000 7fd542d69700 1 -- 172.21.15.45:0/3456849801 wait complete. 2023-10-17T01:06:20.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:19 smithi045 ceph-mon[98474]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:20.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:19 smithi045 ceph-mon[98474]: Saving service mgr spec with placement count:2 2023-10-17T01:06:20.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:19 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:20.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:19 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:20.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.892+0000 7f300e586700 1 Processor -- start 2023-10-17T01:06:20.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.892+0000 7f300e586700 1 -- start start 2023-10-17T01:06:20.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.892+0000 7f300e586700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:20.683 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.892+0000 7f300e586700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300807a8c0 con 0x7f300807be90 2023-10-17T01:06:20.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.893+0000 7f3007fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:20.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.893+0000 7f3007fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42258/0 (socket says 172.21.15.45:42258) 2023-10-17T01:06:20.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.893+0000 7f3007fff700 1 -- 172.21.15.45:0/388553520 learned_addr learned my addr 172.21.15.45:0/388553520 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:20.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.893+0000 7f3007fff700 1 -- 172.21.15.45:0/388553520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300807aa00 con 0x7f300807be90 2023-10-17T01:06:20.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.893+0000 7f3007fff700 1 --2- 172.21.15.45:0/388553520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300807a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80098a0 tx=0x7f2ff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e927e842d5542a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:20.684 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.894+0000 7f3006ffd700 1 -- 172.21.15.45:0/388553520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8004160 con 0x7f300807be90 2023-10-17T01:06:20.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.894+0000 7f3006ffd700 1 -- 172.21.15.45:0/388553520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ff80042c0 con 0x7f300807be90 2023-10-17T01:06:20.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.894+0000 7f3006ffd700 1 -- 172.21.15.45:0/388553520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff80119b0 con 0x7f300807be90 2023-10-17T01:06:20.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.894+0000 7f300e586700 1 -- 172.21.15.45:0/388553520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 msgr2=0x7f300807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:20.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.894+0000 7f300e586700 1 --2- 172.21.15.45:0/388553520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300807a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80098a0 tx=0x7f2ff8009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:20.685 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.895+0000 7f300e586700 1 -- 172.21.15.45:0/388553520 shutdown_connections 2023-10-17T01:06:20.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.895+0000 7f300e586700 1 --2- 172.21.15.45:0/388553520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300807a2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:20.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.895+0000 7f300e586700 1 -- 172.21.15.45:0/388553520 >> 172.21.15.45:0/388553520 conn(0x7f3008101a70 msgr2=0x7f3008103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:20.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.895+0000 7f300e586700 1 -- 172.21.15.45:0/388553520 shutdown_connections 2023-10-17T01:06:20.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.895+0000 7f300e586700 1 -- 172.21.15.45:0/388553520 wait complete. 2023-10-17T01:06:20.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.896+0000 7f300e586700 1 Processor -- start 2023-10-17T01:06:20.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.896+0000 7f300e586700 1 -- start start 2023-10-17T01:06:20.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.896+0000 7f300e586700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:20.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.896+0000 7f300e586700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300819c820 con 0x7f300807be90 2023-10-17T01:06:20.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f3007fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:20.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f3007fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42268/0 (socket says 172.21.15.45:42268) 2023-10-17T01:06:20.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f3007fff700 1 -- 172.21.15.45:0/195721184 learned_addr learned my addr 172.21.15.45:0/195721184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:20.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f3007fff700 1 -- 172.21.15.45:0/195721184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8009550 con 0x7f300807be90 2023-10-17T01:06:20.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f3007fff700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300819c2e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000c00 tx=0x7f2ff8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:20.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff80039f0 con 0x7f300807be90 2023-10-17T01:06:20.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ff8003b50 con 0x7f300807be90 2023-10-17T01:06:20.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.897+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300819ca20 con 0x7f300807be90 2023-10-17T01:06:20.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.898+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff801a780 con 0x7f300807be90 2023-10-17T01:06:20.689 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.898+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300819cec0 con 0x7f300807be90 2023-10-17T01:06:20.689 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.899+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f2ff801a9f0 con 0x7f300807be90 2023-10-17T01:06:20.689 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.899+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300810c8d0 con 0x7f300807be90 2023-10-17T01:06:20.689 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.899+0000 7f30057fa700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ff0038550 0x7f2ff003aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:20.689 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.899+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2ff804c5d0 con 0x7f300807be90 2023-10-17T01:06:20.690 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.900+0000 7f30077fe700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ff0038550 0x7f2ff003aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:20.690 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.901+0000 7f30077fe700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ff0038550 0x7f2ff003aa10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:20.690 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:19.905+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2ff801aca0 con 0x7f300807be90 2023-10-17T01:06:20.690 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.148+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f300819fd00 con 0x7f300807be90 2023-10-17T01:06:20.690 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.152+0000 7f30057fa700 1 -- 172.21.15.45:0/195721184 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2ff80187c0 con 0x7f300807be90 2023-10-17T01:06:20.691 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.155+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ff0038550 msgr2=0x7f2ff003aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:20.691 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.155+0000 7f300e586700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ff0038550 0x7f2ff003aa10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:20.691 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.155+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 msgr2=0x7f300819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:20.691 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.155+0000 7f300e586700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300819c2e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000c00 tx=0x7f2ff8011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:20.691 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.156+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 shutdown_connections 2023-10-17T01:06:20.692 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.156+0000 7f300e586700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f2ff0038550 0x7f2ff003aa10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:20.692 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.156+0000 7f300e586700 1 --2- 172.21.15.45:0/195721184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300807be90 0x7f300819c2e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:20.692 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.156+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 >> 172.21.15.45:0/195721184 conn(0x7f3008101a70 msgr2=0x7f3008102850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:20.692 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.156+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 shutdown_connections 2023-10-17T01:06:20.692 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:20.156+0000 7f300e586700 1 -- 172.21.15.45:0/195721184 wait complete. 2023-10-17T01:06:21.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:21 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/195721184' entity='client.admin' 2023-10-17T01:06:22.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.394+0000 7f70c828f700 1 Processor -- start 2023-10-17T01:06:22.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.395+0000 7f70c828f700 1 -- start start 2023-10-17T01:06:22.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.395+0000 7f70c828f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:22.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.395+0000 7f70c828f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c007a8a0 con 0x7f70c007be70 2023-10-17T01:06:22.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.395+0000 7f70c602b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:22.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.395+0000 7f70c602b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42272/0 (socket says 172.21.15.45:42272) 2023-10-17T01:06:22.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.395+0000 7f70c602b700 1 -- 172.21.15.45:0/3044286338 learned_addr learned my addr 172.21.15.45:0/3044286338 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:22.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.396+0000 7f70c602b700 1 -- 172.21.15.45:0/3044286338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c007a9e0 con 0x7f70c007be70 2023-10-17T01:06:22.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.396+0000 7f70c602b700 1 --2- 172.21.15.45:0/3044286338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c007a2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0098a0 tx=0x7f70bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d050b18f82332826 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:22.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.396+0000 7f70c5029700 1 -- 172.21.15.45:0/3044286338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc004160 con 0x7f70c007be70 2023-10-17T01:06:22.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.396+0000 7f70c5029700 1 -- 172.21.15.45:0/3044286338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70bc0042c0 con 0x7f70c007be70 2023-10-17T01:06:22.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.396+0000 7f70c5029700 1 -- 172.21.15.45:0/3044286338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc0119b0 con 0x7f70c007be70 2023-10-17T01:06:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.397+0000 7f70c828f700 1 -- 172.21.15.45:0/3044286338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 msgr2=0x7f70c007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.397+0000 7f70c828f700 1 --2- 172.21.15.45:0/3044286338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c007a2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0098a0 tx=0x7f70bc009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.397+0000 7f70c828f700 1 -- 172.21.15.45:0/3044286338 shutdown_connections 2023-10-17T01:06:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.397+0000 7f70c828f700 1 --2- 172.21.15.45:0/3044286338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c007a2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.397+0000 7f70c828f700 1 -- 172.21.15.45:0/3044286338 >> 172.21.15.45:0/3044286338 conn(0x7f70c0101a30 msgr2=0x7f70c0103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:22.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.397+0000 7f70c828f700 1 -- 172.21.15.45:0/3044286338 shutdown_connections 2023-10-17T01:06:22.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.398+0000 7f70c828f700 1 -- 172.21.15.45:0/3044286338 wait complete. 2023-10-17T01:06:22.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.399+0000 7f70c828f700 1 Processor -- start 2023-10-17T01:06:22.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.399+0000 7f70c828f700 1 -- start start 2023-10-17T01:06:22.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.399+0000 7f70c828f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:22.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.399+0000 7f70c828f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c019c880 con 0x7f70c007be70 2023-10-17T01:06:22.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.400+0000 7f70c602b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:22.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.400+0000 7f70c602b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42278/0 (socket says 172.21.15.45:42278) 2023-10-17T01:06:22.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.400+0000 7f70c602b700 1 -- 172.21.15.45:0/4271754231 learned_addr learned my addr 172.21.15.45:0/4271754231 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:22.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.400+0000 7f70c602b700 1 -- 172.21.15.45:0/4271754231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc009550 con 0x7f70c007be70 2023-10-17T01:06:22.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.400+0000 7f70c602b700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c019c340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70bc003f00 tx=0x7f70bc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:22.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.401+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc003b20 con 0x7f70c007be70 2023-10-17T01:06:22.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.401+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c019ca80 con 0x7f70c007be70 2023-10-17T01:06:22.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.401+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70bc003c80 con 0x7f70c007be70 2023-10-17T01:06:22.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.401+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc019770 con 0x7f70c007be70 2023-10-17T01:06:22.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.401+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c019cf20 con 0x7f70c007be70 2023-10-17T01:06:22.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.402+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7f70bc01ab60 con 0x7f70c007be70 2023-10-17T01:06:22.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.402+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c010c8d0 con 0x7f70c007be70 2023-10-17T01:06:22.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.402+0000 7f70b77fe700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f70ac038500 0x7f70ac03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:22.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.403+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f70bc04c7e0 con 0x7f70c007be70 2023-10-17T01:06:22.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.403+0000 7f70c582a700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f70ac038500 0x7f70ac03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:22.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.403+0000 7f70c582a700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f70ac038500 0x7f70ac03a9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:22.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.408+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f70bc01a430 con 0x7f70c007be70 2023-10-17T01:06:22.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.620+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f70c010caf0 con 0x7f70c007be70 2023-10-17T01:06:22.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.625+0000 7f70b77fe700 1 -- 172.21.15.45:0/4271754231 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f70bc02db40 con 0x7f70c007be70 2023-10-17T01:06:22.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.627+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f70ac038500 msgr2=0x7f70ac03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:22.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.627+0000 7f70c828f700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f70ac038500 0x7f70ac03a9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:22.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.627+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 msgr2=0x7f70c019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:22.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.627+0000 7f70c828f700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c019c340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70bc003f00 tx=0x7f70bc011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:22.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.628+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 shutdown_connections 2023-10-17T01:06:22.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.628+0000 7f70c828f700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f70ac038500 0x7f70ac03a9c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:22.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.628+0000 7f70c828f700 1 --2- 172.21.15.45:0/4271754231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c007be70 0x7f70c019c340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:22.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.628+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 >> 172.21.15.45:0/4271754231 conn(0x7f70c0101a30 msgr2=0x7f70c0102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:22.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.628+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 shutdown_connections 2023-10-17T01:06:22.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:21.628+0000 7f70c828f700 1 -- 172.21.15.45:0/4271754231 wait complete. 2023-10-17T01:06:22.028 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2023-10-17T01:06:22.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:22 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:22.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:22 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4271754231' entity='client.admin' 2023-10-17T01:06:23.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3878236775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-17T01:06:23.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-mon[98474]: from='mgr.14118 172.21.15.45:0/2570663473' entity='mgr.x' 2023-10-17T01:06:23.552 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: ignoring --setuser ceph since I am not root 2023-10-17T01:06:23.552 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: ignoring --setgroup ceph since I am not root 2023-10-17T01:06:23.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.585+0000 7fb537caf700 1 Processor -- start 2023-10-17T01:06:23.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.585+0000 7fb537caf700 1 -- start start 2023-10-17T01:06:23.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.585+0000 7fb537caf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:23.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.585+0000 7fb537caf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53007a8a0 con 0x7fb53007be70 2023-10-17T01:06:23.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.586+0000 7fb535a4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:23.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.586+0000 7fb535a4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42294/0 (socket says 172.21.15.45:42294) 2023-10-17T01:06:23.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.586+0000 7fb535a4b700 1 -- 172.21.15.45:0/3410269468 learned_addr learned my addr 172.21.15.45:0/3410269468 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:23.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.586+0000 7fb535a4b700 1 -- 172.21.15.45:0/3410269468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53007a9e0 con 0x7fb53007be70 2023-10-17T01:06:23.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.586+0000 7fb535a4b700 1 --2- 172.21.15.45:0/3410269468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53007a2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0098a0 tx=0x7fb52c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69671781b8d5c44b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:23.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb534a49700 1 -- 172.21.15.45:0/3410269468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb52c004160 con 0x7fb53007be70 2023-10-17T01:06:23.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb534a49700 1 -- 172.21.15.45:0/3410269468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52c0042c0 con 0x7fb53007be70 2023-10-17T01:06:23.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb534a49700 1 -- 172.21.15.45:0/3410269468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb52c0119b0 con 0x7fb53007be70 2023-10-17T01:06:23.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb537caf700 1 -- 172.21.15.45:0/3410269468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 msgr2=0x7fb53007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:23.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb537caf700 1 --2- 172.21.15.45:0/3410269468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53007a2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0098a0 tx=0x7fb52c009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:23.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb537caf700 1 -- 172.21.15.45:0/3410269468 shutdown_connections 2023-10-17T01:06:23.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb537caf700 1 --2- 172.21.15.45:0/3410269468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53007a2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:23.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb537caf700 1 -- 172.21.15.45:0/3410269468 >> 172.21.15.45:0/3410269468 conn(0x7fb530101a30 msgr2=0x7fb530103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.587+0000 7fb537caf700 1 -- 172.21.15.45:0/3410269468 shutdown_connections 2023-10-17T01:06:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.588+0000 7fb537caf700 1 -- 172.21.15.45:0/3410269468 wait complete. 2023-10-17T01:06:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.588+0000 7fb537caf700 1 Processor -- start 2023-10-17T01:06:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.589+0000 7fb537caf700 1 -- start start 2023-10-17T01:06:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.589+0000 7fb537caf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.589+0000 7fb537caf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019c880 con 0x7fb53007be70 2023-10-17T01:06:23.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.589+0000 7fb535a4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:23.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.589+0000 7fb535a4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42310/0 (socket says 172.21.15.45:42310) 2023-10-17T01:06:23.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.589+0000 7fb535a4b700 1 -- 172.21.15.45:0/3878236775 learned_addr learned my addr 172.21.15.45:0/3878236775 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:23.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb535a4b700 1 -- 172.21.15.45:0/3878236775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c009550 con 0x7fb53007be70 2023-10-17T01:06:23.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb535a4b700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53019c340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb52c003f00 tx=0x7fb52c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:23.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb52c003b20 con 0x7fb53007be70 2023-10-17T01:06:23.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52c003c80 con 0x7fb53007be70 2023-10-17T01:06:23.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53019ca80 con 0x7fb53007be70 2023-10-17T01:06:23.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb52c019770 con 0x7fb53007be70 2023-10-17T01:06:23.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.590+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53019cf20 con 0x7fb53007be70 2023-10-17T01:06:23.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.591+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fb52c0199e0 con 0x7fb53007be70 2023-10-17T01:06:23.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.592+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53010c8d0 con 0x7fb53007be70 2023-10-17T01:06:23.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.592+0000 7fb526ffd700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fb51c038550 0x7fb51c03aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:23.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.592+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb52c04c8d0 con 0x7fb53007be70 2023-10-17T01:06:23.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.592+0000 7fb53524a700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fb51c038550 0x7fb51c03aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:23.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.593+0000 7fb53524a700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fb51c038550 0x7fb51c03aa10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb520006fd0 tx=0x7fb520006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:23.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.597+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb52c018ba0 con 0x7fb53007be70 2023-10-17T01:06:23.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:22.875+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb53010caf0 con 0x7fb53007be70 2023-10-17T01:06:23.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.303+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 9) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fb52c04ecd0 con 0x7fb53007be70 2023-10-17T01:06:23.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.305+0000 7fb526ffd700 1 -- 172.21.15.45:0/3878236775 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb52c023550 con 0x7fb53007be70 2023-10-17T01:06:23.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fb51c038550 msgr2=0x7fb51c03aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:23.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fb51c038550 0x7fb51c03aa10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb520006fd0 tx=0x7fb520006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:23.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 msgr2=0x7fb53019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:23.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53019c340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb52c003f00 tx=0x7fb52c011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:23.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 shutdown_connections 2023-10-17T01:06:23.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7fb51c038550 0x7fb51c03aa10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:23.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 --2- 172.21.15.45:0/3878236775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb53007be70 0x7fb53019c340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 >> 172.21.15.45:0/3878236775 conn(0x7fb530101a30 msgr2=0x7fb530102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 shutdown_connections 2023-10-17T01:06:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:23.308+0000 7fb537caf700 1 -- 172.21.15.45:0/3878236775 wait complete. 2023-10-17T01:06:24.001 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:23.613+0000 7f30cdbdc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-17T01:06:24.001 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:23.722+0000 7f30cdbdc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-17T01:06:24.002 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:23.855+0000 7f30cdbdc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-17T01:06:24.002 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:24.000+0000 7f30cdbdc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-17T01:06:24.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:24 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3878236775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-17T01:06:24.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:24 smithi045 ceph-mon[98474]: mgrmap e9: x(active, since 17s) 2023-10-17T01:06:24.552 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:24.255+0000 7f30cdbdc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-17T01:06:24.915 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:24.764+0000 7f30cdbdc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-17T01:06:25.176 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:24.948+0000 7f30cdbdc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-17T01:06:25.176 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:25.068+0000 7f30cdbdc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-17T01:06:25.222 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:06:25.222 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-17T01:06:25.222 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-17T01:06:25.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-17T01:06:25.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-17T01:06:25.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:06:25.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.416+0000 7f92ae5ec700 1 Processor -- start 2023-10-17T01:06:25.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.416+0000 7f92ae5ec700 1 -- start start 2023-10-17T01:06:25.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.416+0000 7f92ae5ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:25.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.416+0000 7f92ae5ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a807a8a0 con 0x7f92a807be70 2023-10-17T01:06:25.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.417+0000 7f92a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:25.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.417+0000 7f92a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42348/0 (socket says 172.21.15.45:42348) 2023-10-17T01:06:25.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.417+0000 7f92a7fff700 1 -- 172.21.15.45:0/1681469213 learned_addr learned my addr 172.21.15.45:0/1681469213 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:25.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.417+0000 7f92a7fff700 1 -- 172.21.15.45:0/1681469213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a807a9e0 con 0x7f92a807be70 2023-10-17T01:06:25.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.418+0000 7f92a7fff700 1 --2- 172.21.15.45:0/1681469213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a807a2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f92980098a0 tx=0x7f9298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7700cd9d4f74ca57 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:25.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.418+0000 7f92a6ffd700 1 -- 172.21.15.45:0/1681469213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9298004160 con 0x7f92a807be70 2023-10-17T01:06:25.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.418+0000 7f92a6ffd700 1 -- 172.21.15.45:0/1681469213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92980042c0 con 0x7f92a807be70 2023-10-17T01:06:25.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.418+0000 7f92a6ffd700 1 -- 172.21.15.45:0/1681469213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92980119b0 con 0x7f92a807be70 2023-10-17T01:06:25.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 -- 172.21.15.45:0/1681469213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 msgr2=0x7f92a807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:25.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 --2- 172.21.15.45:0/1681469213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a807a2d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f92980098a0 tx=0x7f9298009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:25.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 -- 172.21.15.45:0/1681469213 shutdown_connections 2023-10-17T01:06:25.227 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 --2- 172.21.15.45:0/1681469213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a807a2d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:25.227 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 -- 172.21.15.45:0/1681469213 >> 172.21.15.45:0/1681469213 conn(0x7f92a8101a70 msgr2=0x7f92a8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:25.227 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 -- 172.21.15.45:0/1681469213 shutdown_connections 2023-10-17T01:06:25.227 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.419+0000 7f92ae5ec700 1 -- 172.21.15.45:0/1681469213 wait complete. 2023-10-17T01:06:25.228 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.420+0000 7f92ae5ec700 1 Processor -- start 2023-10-17T01:06:25.228 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.420+0000 7f92ae5ec700 1 -- start start 2023-10-17T01:06:25.228 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.420+0000 7f92ae5ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:25.228 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.420+0000 7f92ae5ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a819c830 con 0x7f92a807be70 2023-10-17T01:06:25.229 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.421+0000 7f92a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:25.229 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.421+0000 7f92a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42352/0 (socket says 172.21.15.45:42352) 2023-10-17T01:06:25.229 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.421+0000 7f92a7fff700 1 -- 172.21.15.45:0/3638457138 learned_addr learned my addr 172.21.15.45:0/3638457138 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:25.229 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.421+0000 7f92a7fff700 1 -- 172.21.15.45:0/3638457138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9298009550 con 0x7f92a807be70 2023-10-17T01:06:25.229 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.421+0000 7f92a7fff700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a819c2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9298011980 tx=0x7f9298011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:25.230 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.422+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9298003a60 con 0x7f92a807be70 2023-10-17T01:06:25.230 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.422+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9298003bc0 con 0x7f92a807be70 2023-10-17T01:06:25.230 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.422+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a819ca30 con 0x7f92a807be70 2023-10-17T01:06:25.230 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.422+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f929801a670 con 0x7f92a807be70 2023-10-17T01:06:25.230 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.422+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a819ced0 con 0x7f92a807be70 2023-10-17T01:06:25.231 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.423+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f929801a8e0 con 0x7f92a807be70 2023-10-17T01:06:25.231 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.423+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a810c8f0 con 0x7f92a807be70 2023-10-17T01:06:25.231 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.423+0000 7f92a57fa700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 0x7f929003aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:25.231 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.423+0000 7f92a77fe700 1 -- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 msgr2=0x7f929003aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:25.231 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.423+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f929804c830 con 0x7f92a807be70 2023-10-17T01:06:25.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.424+0000 7f92a77fe700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 0x7f929003aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:06:25.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.429+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f9298018db0 con 0x7f92a807be70 2023-10-17T01:06:25.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.625+0000 7f92a77fe700 1 -- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 msgr2=0x7f929003aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.625+0000 7f92a77fe700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 0x7f929003aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-17T01:06:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.713+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f92a810cb10 con 0x7f92a807be70 2023-10-17T01:06:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.714+0000 7f92a57fa700 1 -- 172.21.15.45:0/3638457138 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9298018350 con 0x7f92a807be70 2023-10-17T01:06:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 msgr2=0x7f929003aa60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:06:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 0x7f929003aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:25.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 msgr2=0x7f92a819c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:25.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a819c2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9298011980 tx=0x7f9298011a60 comp rx=0 tx=0).stop 2023-10-17T01:06:25.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 shutdown_connections 2023-10-17T01:06:25.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92900385a0 0x7f929003aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:25.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 --2- 172.21.15.45:0/3638457138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92a807be70 0x7f92a819c2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:25.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.717+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 >> 172.21.15.45:0/3638457138 conn(0x7f92a8101a70 msgr2=0x7f92a8102770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:25.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.718+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 shutdown_connections 2023-10-17T01:06:25.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:24.718+0000 7f92ae5ec700 1 -- 172.21.15.45:0/3638457138 wait complete. 2023-10-17T01:06:25.236 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2023-10-17T01:06:25.236 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2023-10-17T01:06:25.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:25 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3638457138' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-17T01:06:25.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:25.198+0000 7f30cdbdc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-17T01:06:26.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:25.786+0000 7f30cdbdc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-17T01:06:27.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:26.673+0000 7f30cdbdc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-17T01:06:27.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:26.783+0000 7f30cdbdc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-17T01:06:28.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:27.779+0000 7f30cdbdc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:06:28.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:27.908+0000 7f30cdbdc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-17T01:06:28.521 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:28.141+0000 7f30cdbdc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:06:28.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:28.520+0000 7f30cdbdc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-17T01:06:29.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:28.805+0000 7f30cdbdc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-17T01:06:29.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:29.008+0000 7f30cdbdc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-17T01:06:29.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:29.368+0000 7f30cdbdc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-17T01:06:30.782 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:30.338+0000 7f30cdbdc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-17T01:06:30.782 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:30.519+0000 7f30cdbdc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-17T01:06:31.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:30.780+0000 7f30cdbdc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-17T01:06:31.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:30.936+0000 7f30cdbdc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-17T01:06:31.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:31.225+0000 7f30cdbdc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-17T01:06:32.801 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:06:32.346+0000 7f30cdbdc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-17T01:06:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: Active manager daemon x restarted 2023-10-17T01:06:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: Activating manager daemon x 2023-10-17T01:06:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: osdmap e3: 0 total, 0 up, 0 in 2023-10-17T01:06:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: mgrmap e10: x(active, starting, since 0.00421435s) 2023-10-17T01:06:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:06:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-17T01:06:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-17T01:06:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-17T01:06:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-17T01:06:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: Manager daemon x is now available 2023-10-17T01:06:32.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:32.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-17T01:06:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-17T01:06:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:33 smithi045 ceph-mon[98474]: mgrmap e11: x(active, since 1.0076s) 2023-10-17T01:06:33.805 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-10-17T01:06:33.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-17T01:06:33.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-17T01:06:33.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-10-17T01:06:33.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.957+0000 7f9232059700 1 Processor -- start 2023-10-17T01:06:33.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.958+0000 7f9232059700 1 -- start start 2023-10-17T01:06:33.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.958+0000 7f9232059700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:33.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.958+0000 7f9232059700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c079e20 con 0x7f922c106fa0 2023-10-17T01:06:33.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.958+0000 7f922b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:33.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.958+0000 7f922b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42366/0 (socket says 172.21.15.45:42366) 2023-10-17T01:06:33.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.958+0000 7f922b7fe700 1 -- 172.21.15.45:0/977447257 learned_addr learned my addr 172.21.15.45:0/977447257 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:33.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.959+0000 7f922b7fe700 1 -- 172.21.15.45:0/977447257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c1098d0 con 0x7f922c106fa0 2023-10-17T01:06:33.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.959+0000 7f922b7fe700 1 --2- 172.21.15.45:0/977447257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c109390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f921c0098a0 tx=0x7f921c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20038d5112936f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:33.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.959+0000 7f922a7fc700 1 -- 172.21.15.45:0/977447257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f921c004160 con 0x7f922c106fa0 2023-10-17T01:06:33.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.959+0000 7f922a7fc700 1 -- 172.21.15.45:0/977447257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f921c0042c0 con 0x7f922c106fa0 2023-10-17T01:06:33.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.959+0000 7f922a7fc700 1 -- 172.21.15.45:0/977447257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f921c0119b0 con 0x7f922c106fa0 2023-10-17T01:06:33.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 -- 172.21.15.45:0/977447257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 msgr2=0x7f922c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:33.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 --2- 172.21.15.45:0/977447257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c109390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f921c0098a0 tx=0x7f921c009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:33.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 -- 172.21.15.45:0/977447257 shutdown_connections 2023-10-17T01:06:33.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 --2- 172.21.15.45:0/977447257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c109390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:33.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 -- 172.21.15.45:0/977447257 >> 172.21.15.45:0/977447257 conn(0x7f922c100990 msgr2=0x7f922c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:33.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 -- 172.21.15.45:0/977447257 shutdown_connections 2023-10-17T01:06:33.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.960+0000 7f9232059700 1 -- 172.21.15.45:0/977447257 wait complete. 2023-10-17T01:06:33.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.961+0000 7f9232059700 1 Processor -- start 2023-10-17T01:06:33.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f9232059700 1 -- start start 2023-10-17T01:06:33.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f9232059700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:33.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f9232059700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c19c7f0 con 0x7f922c106fa0 2023-10-17T01:06:33.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f922b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:33.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f922b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42382/0 (socket says 172.21.15.45:42382) 2023-10-17T01:06:33.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f922b7fe700 1 -- 172.21.15.45:0/3221913128 learned_addr learned my addr 172.21.15.45:0/3221913128 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:33.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.962+0000 7f922b7fe700 1 -- 172.21.15.45:0/3221913128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c009550 con 0x7f922c106fa0 2023-10-17T01:06:33.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.963+0000 7f922b7fe700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c19c2b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f921c011980 tx=0x7f921c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:33.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.963+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f921c003a60 con 0x7f922c106fa0 2023-10-17T01:06:33.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.963+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f921c003bc0 con 0x7f922c106fa0 2023-10-17T01:06:33.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.963+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c19c9f0 con 0x7f922c106fa0 2023-10-17T01:06:33.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.963+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f921c01a670 con 0x7f922c106fa0 2023-10-17T01:06:33.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.963+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c19ce90 con 0x7f922c106fa0 2023-10-17T01:06:33.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.964+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f921c024440 con 0x7f922c106fa0 2023-10-17T01:06:33.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.965+0000 7f9228ff9700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:33.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.965+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 --> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f921403b170 con 0x7f92140385a0 2023-10-17T01:06:33.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.965+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f921c04c5e0 con 0x7f922c106fa0 2023-10-17T01:06:33.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.965+0000 7f922affd700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:33.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:25.965+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:06:33.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:26.166+0000 7f922affd700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:33.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:26.166+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-17T01:06:33.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:26.567+0000 7f922affd700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:33.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:26.567+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-17T01:06:33.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:27.368+0000 7f922affd700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:33.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:27.368+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-17T01:06:33.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:28.969+0000 7f922affd700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:33.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:28.969+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-17T01:06:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:32.172+0000 7f922affd700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1775908551 2023-10-17T01:06:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:32.172+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-17T01:06:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:32.350+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) v1 ==== 45202+0+0 (secure 0 0 0) 0x7f921c024a00 con 0x7f922c106fa0 2023-10-17T01:06:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:32.350+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 msgr2=0x7f921403aa60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:06:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:32.350+0000 7f9228ff9700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:33.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.352+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) v1 ==== 45329+0+0 (secure 0 0 0) 0x7f921c04def0 con 0x7f922c106fa0 2023-10-17T01:06:33.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.353+0000 7f9228ff9700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f921403bfc0 0x7f921403e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:33.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.353+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f921403b170 con 0x7f921403bfc0 2023-10-17T01:06:33.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.353+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f921403bfc0 0x7f921403e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:33.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.354+0000 7f922affd700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f921403bfc0 0x7f921403e3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9220003a10 tx=0x7f9220009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.355+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f921403b170 con 0x7f921403bfc0 2023-10-17T01:06:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.362+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f922c19d5f0 con 0x7f921403bfc0 2023-10-17T01:06:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9228ff9700 1 -- 172.21.15.45:0/3221913128 <== mgr.14150 v2:172.21.15.45:6800/869456926 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f922c19d5f0 con 0x7f921403bfc0 2023-10-17T01:06:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f921403bfc0 msgr2=0x7f921403e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f921403bfc0 0x7f921403e3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9220003a10 tx=0x7f9220009270 comp rx=0 tx=0).stop 2023-10-17T01:06:33.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 msgr2=0x7f922c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:33.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c19c2b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f921c011980 tx=0x7f921c011a60 comp rx=0 tx=0).stop 2023-10-17T01:06:33.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 shutdown_connections 2023-10-17T01:06:33.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f921403bfc0 0x7f921403e3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:33.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c106fa0 0x7f922c19c2b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:33.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 --2- 172.21.15.45:0/3221913128 >> [v2:172.21.15.45:6800/1775908551,v1:172.21.15.45:6801/1775908551] conn(0x7f92140385a0 0x7f921403aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:33.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 >> 172.21.15.45:0/3221913128 conn(0x7f922c100990 msgr2=0x7f922c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:33.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 shutdown_connections 2023-10-17T01:06:33.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:33.363+0000 7f9232059700 1 -- 172.21.15.45:0/3221913128 wait complete. 2023-10-17T01:06:33.829 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2023-10-17T01:06:33.829 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2023-10-17T01:06:34.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:34 smithi045 ceph-mon[98474]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-17T01:06:34.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:34 smithi045 ceph-mon[98474]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-17T01:06:35.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-17T01:06:35.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.422+0000 7fb6729ff700 1 Processor -- start 2023-10-17T01:06:35.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.422+0000 7fb6729ff700 1 -- start start 2023-10-17T01:06:35.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.423+0000 7fb6729ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:35.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.423+0000 7fb6729ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c07a8a0 con 0x7fb66c07be70 2023-10-17T01:06:35.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.423+0000 7fb66bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:35.199 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.423+0000 7fb66bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33614/0 (socket says 172.21.15.45:33614) 2023-10-17T01:06:35.199 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.423+0000 7fb66bfff700 1 -- 172.21.15.45:0/3959326627 learned_addr learned my addr 172.21.15.45:0/3959326627 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:35.199 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.424+0000 7fb66bfff700 1 -- 172.21.15.45:0/3959326627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c07a9e0 con 0x7fb66c07be70 2023-10-17T01:06:35.199 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.424+0000 7fb66bfff700 1 --2- 172.21.15.45:0/3959326627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c07a2d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0098a0 tx=0x7fb65c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269639be86f9d772 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:35.199 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.424+0000 7fb66affd700 1 -- 172.21.15.45:0/3959326627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb65c004160 con 0x7fb66c07be70 2023-10-17T01:06:35.200 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.424+0000 7fb66affd700 1 -- 172.21.15.45:0/3959326627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c0042c0 con 0x7fb66c07be70 2023-10-17T01:06:35.200 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.424+0000 7fb66affd700 1 -- 172.21.15.45:0/3959326627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb65c0119b0 con 0x7fb66c07be70 2023-10-17T01:06:35.200 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 -- 172.21.15.45:0/3959326627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 msgr2=0x7fb66c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:35.200 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 --2- 172.21.15.45:0/3959326627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c07a2d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0098a0 tx=0x7fb65c009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:35.201 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 -- 172.21.15.45:0/3959326627 shutdown_connections 2023-10-17T01:06:35.201 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 --2- 172.21.15.45:0/3959326627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c07a2d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:35.201 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 -- 172.21.15.45:0/3959326627 >> 172.21.15.45:0/3959326627 conn(0x7fb66c101a70 msgr2=0x7fb66c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:35.201 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 -- 172.21.15.45:0/3959326627 shutdown_connections 2023-10-17T01:06:35.201 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.425+0000 7fb6729ff700 1 -- 172.21.15.45:0/3959326627 wait complete. 2023-10-17T01:06:35.202 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.426+0000 7fb6729ff700 1 Processor -- start 2023-10-17T01:06:35.202 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.427+0000 7fb6729ff700 1 -- start start 2023-10-17T01:06:35.202 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.427+0000 7fb6729ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:35.202 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.427+0000 7fb6729ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c19c830 con 0x7fb66c07be70 2023-10-17T01:06:35.202 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.427+0000 7fb66bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:35.202 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.427+0000 7fb66bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33622/0 (socket says 172.21.15.45:33622) 2023-10-17T01:06:35.203 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.427+0000 7fb66bfff700 1 -- 172.21.15.45:0/2916391460 learned_addr learned my addr 172.21.15.45:0/2916391460 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:35.203 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb66bfff700 1 -- 172.21.15.45:0/2916391460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c009550 con 0x7fb66c07be70 2023-10-17T01:06:35.203 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb66bfff700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c19c2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb65c011980 tx=0x7fb65c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:35.203 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb65c003a60 con 0x7fb66c07be70 2023-10-17T01:06:35.203 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c003bc0 con 0x7fb66c07be70 2023-10-17T01:06:35.204 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66c19ca30 con 0x7fb66c07be70 2023-10-17T01:06:35.204 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb65c01a670 con 0x7fb66c07be70 2023-10-17T01:06:35.204 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.428+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c19ced0 con 0x7fb66c07be70 2023-10-17T01:06:35.204 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.429+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fb65c024a00 con 0x7fb66c07be70 2023-10-17T01:06:35.204 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.429+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66c10c8f0 con 0x7fb66c07be70 2023-10-17T01:06:35.205 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.430+0000 7fb6697fa700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6540385a0 0x7fb65403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:35.205 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.430+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb65c04c5e0 con 0x7fb66c07be70 2023-10-17T01:06:35.205 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.430+0000 7fb66b7fe700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6540385a0 0x7fb65403aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:35.205 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.431+0000 7fb66b7fe700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6540385a0 0x7fb65403aa60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb660006fd0 tx=0x7fb660006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:35.205 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.435+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb65c024440 con 0x7fb66c07be70 2023-10-17T01:06:35.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.675+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb66c19fcb0 con 0x7fb6540385a0 2023-10-17T01:06:35.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.775+0000 7fb6697fa700 1 -- 172.21.15.45:0/2916391460 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb66c19fcb0 con 0x7fb6540385a0 2023-10-17T01:06:35.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6540385a0 msgr2=0x7fb65403aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:35.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6540385a0 0x7fb65403aa60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb660006fd0 tx=0x7fb660006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:35.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 msgr2=0x7fb66c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:35.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c19c2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb65c011980 tx=0x7fb65c011a60 comp rx=0 tx=0).stop 2023-10-17T01:06:35.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 shutdown_connections 2023-10-17T01:06:35.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6540385a0 0x7fb65403aa60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:35.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 --2- 172.21.15.45:0/2916391460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb66c07be70 0x7fb66c19c2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:35.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 >> 172.21.15.45:0/2916391460 conn(0x7fb66c101a70 msgr2=0x7fb66c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:35.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 shutdown_connections 2023-10-17T01:06:35.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:34.778+0000 7fb6729ff700 1 -- 172.21.15.45:0/2916391460 wait complete. 2023-10-17T01:06:35.208 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2023-10-17T01:06:35.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:33] ENGINE Bus STARTING 2023-10-17T01:06:35.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:33] ENGINE Serving on https://172.21.15.45:7150 2023-10-17T01:06:35.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:33] ENGINE Serving on http://172.21.15.45:8765 2023-10-17T01:06:35.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: [17/Oct/2023:01:06:33] ENGINE Bus STARTED 2023-10-17T01:06:35.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: mgrmap e12: x(active, since 2s) 2023-10-17T01:06:35.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:35.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:35.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:35 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:36 smithi045 ceph-mon[98474]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:36.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wT0fqZuA2CNhf3mrtLTgDOfoLRQDuApaW0wL.vzrk4Y0ffYGlv59y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1697504796, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-17T01:06:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.755+0000 7f60673e6700 1 Processor -- start 2023-10-17T01:06:36.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.755+0000 7f60673e6700 1 -- start start 2023-10-17T01:06:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.755+0000 7f60673e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f6060109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.755+0000 7f60673e6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6060079e20 con 0x7f6060106fa0 2023-10-17T01:06:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.756+0000 7f6065182700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f6060109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.756+0000 7f6065182700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f6060109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33626/0 (socket says 172.21.15.45:33626) 2023-10-17T01:06:36.785 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.756+0000 7f6065182700 1 -- 172.21.15.45:0/3268358000 learned_addr learned my addr 172.21.15.45:0/3268358000 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.756+0000 7f6065182700 1 -- 172.21.15.45:0/3268358000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60601098d0 con 0x7f6060106fa0 2023-10-17T01:06:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.756+0000 7f6065182700 1 --2- 172.21.15.45:0/3268358000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f6060109390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098a0 tx=0x7f605c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10794f20d9de9e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f6057fff700 1 -- 172.21.15.45:0/3268358000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f605c004160 con 0x7f6060106fa0 2023-10-17T01:06:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f6057fff700 1 -- 172.21.15.45:0/3268358000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c0042c0 con 0x7f6060106fa0 2023-10-17T01:06:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f6057fff700 1 -- 172.21.15.45:0/3268358000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f605c0119b0 con 0x7f6060106fa0 2023-10-17T01:06:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f60673e6700 1 -- 172.21.15.45:0/3268358000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 msgr2=0x7f6060109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f60673e6700 1 --2- 172.21.15.45:0/3268358000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f6060109390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098a0 tx=0x7f605c009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f60673e6700 1 -- 172.21.15.45:0/3268358000 shutdown_connections 2023-10-17T01:06:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f60673e6700 1 --2- 172.21.15.45:0/3268358000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f6060109390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:36.787 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.757+0000 7f60673e6700 1 -- 172.21.15.45:0/3268358000 >> 172.21.15.45:0/3268358000 conn(0x7f6060100990 msgr2=0x7f6060102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.758+0000 7f60673e6700 1 -- 172.21.15.45:0/3268358000 shutdown_connections 2023-10-17T01:06:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.758+0000 7f60673e6700 1 -- 172.21.15.45:0/3268358000 wait complete. 2023-10-17T01:06:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f60673e6700 1 Processor -- start 2023-10-17T01:06:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f60673e6700 1 -- start start 2023-10-17T01:06:36.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f60673e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f606019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f60673e6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606019c7f0 con 0x7f6060106fa0 2023-10-17T01:06:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f6065182700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f606019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f6065182700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f606019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33642/0 (socket says 172.21.15.45:33642) 2023-10-17T01:06:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.759+0000 7f6065182700 1 -- 172.21.15.45:0/798645833 learned_addr learned my addr 172.21.15.45:0/798645833 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.760+0000 7f6065182700 1 -- 172.21.15.45:0/798645833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c009550 con 0x7f6060106fa0 2023-10-17T01:06:36.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.760+0000 7f6065182700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f606019c2b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f605c011980 tx=0x7f605c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.760+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f605c003a60 con 0x7f6060106fa0 2023-10-17T01:06:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.760+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c003bc0 con 0x7f6060106fa0 2023-10-17T01:06:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.760+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f605c01a670 con 0x7f6060106fa0 2023-10-17T01:06:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.760+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606019c9f0 con 0x7f6060106fa0 2023-10-17T01:06:36.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.761+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606019ce90 con 0x7f6060106fa0 2023-10-17T01:06:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.761+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f605c01a8e0 con 0x7f6060106fa0 2023-10-17T01:06:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.762+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606010a660 con 0x7f6060106fa0 2023-10-17T01:06:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.762+0000 7f60567fc700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f604c0385f0 0x7f604c03aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.762+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f605c04c850 con 0x7f6060106fa0 2023-10-17T01:06:36.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.763+0000 7f6064981700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f604c0385f0 0x7f604c03aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.763+0000 7f6064981700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f604c0385f0 0x7f604c03aab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6050006fd0 tx=0x7f6050006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:35.769+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f605c0189a0 con 0x7f6060106fa0 2023-10-17T01:06:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.017+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f60600621a0 con 0x7f604c0385f0 2023-10-17T01:06:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.265+0000 7f60567fc700 1 -- 172.21.15.45:0/798645833 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f60600621a0 con 0x7f604c0385f0 2023-10-17T01:06:36.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.267+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f604c0385f0 msgr2=0x7f604c03aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.267+0000 7f60673e6700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f604c0385f0 0x7f604c03aab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6050006fd0 tx=0x7f6050006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.267+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 msgr2=0x7f606019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.267+0000 7f60673e6700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f606019c2b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f605c011980 tx=0x7f605c011a60 comp rx=0 tx=0).stop 2023-10-17T01:06:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.268+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 shutdown_connections 2023-10-17T01:06:36.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.268+0000 7f60673e6700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f604c0385f0 0x7f604c03aab0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.268+0000 7f60673e6700 1 --2- 172.21.15.45:0/798645833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6060106fa0 0x7f606019c2b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.268+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 >> 172.21.15.45:0/798645833 conn(0x7f6060100990 msgr2=0x7f60601016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.268+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 shutdown_connections 2023-10-17T01:06:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:36.268+0000 7f60673e6700 1 -- 172.21.15.45:0/798645833 wait complete. 2023-10-17T01:06:36.795 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2023-10-17T01:06:37.696 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:37 smithi045 ceph-mon[98474]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2023-10-17T01:06:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.328+0000 7f62b1124700 1 Processor -- start 2023-10-17T01:06:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.328+0000 7f62b1124700 1 -- start start 2023-10-17T01:06:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.328+0000 7f62b1124700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:38.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.328+0000 7f62b1124700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac07a8c0 con 0x7f62ac07be90 2023-10-17T01:06:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.329+0000 7f62aad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.329+0000 7f62aad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33646/0 (socket says 172.21.15.45:33646) 2023-10-17T01:06:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.329+0000 7f62aad9d700 1 -- 172.21.15.45:0/493366695 learned_addr learned my addr 172.21.15.45:0/493366695 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:38.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.329+0000 7f62aad9d700 1 -- 172.21.15.45:0/493366695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac07aa00 con 0x7f62ac07be90 2023-10-17T01:06:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.330+0000 7f62aad9d700 1 --2- 172.21.15.45:0/493366695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac07a2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098a0 tx=0x7f629c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13128cad8159c7de server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.330+0000 7f62a9d9b700 1 -- 172.21.15.45:0/493366695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f629c004160 con 0x7f62ac07be90 2023-10-17T01:06:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.330+0000 7f62a9d9b700 1 -- 172.21.15.45:0/493366695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c0042c0 con 0x7f62ac07be90 2023-10-17T01:06:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.330+0000 7f62a9d9b700 1 -- 172.21.15.45:0/493366695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f629c0119b0 con 0x7f62ac07be90 2023-10-17T01:06:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 -- 172.21.15.45:0/493366695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 msgr2=0x7f62ac07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 --2- 172.21.15.45:0/493366695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac07a2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098a0 tx=0x7f629c009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 -- 172.21.15.45:0/493366695 shutdown_connections 2023-10-17T01:06:38.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 --2- 172.21.15.45:0/493366695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac07a2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 -- 172.21.15.45:0/493366695 >> 172.21.15.45:0/493366695 conn(0x7f62ac101a30 msgr2=0x7f62ac103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 -- 172.21.15.45:0/493366695 shutdown_connections 2023-10-17T01:06:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.331+0000 7f62b1124700 1 -- 172.21.15.45:0/493366695 wait complete. 2023-10-17T01:06:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.332+0000 7f62b1124700 1 Processor -- start 2023-10-17T01:06:38.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.332+0000 7f62b1124700 1 -- start start 2023-10-17T01:06:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.332+0000 7f62b1124700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.332+0000 7f62b1124700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac19c8d0 con 0x7f62ac07be90 2023-10-17T01:06:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.332+0000 7f62aad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.333+0000 7f62aad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac19c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33656/0 (socket says 172.21.15.45:33656) 2023-10-17T01:06:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.333+0000 7f62aad9d700 1 -- 172.21.15.45:0/4599398 learned_addr learned my addr 172.21.15.45:0/4599398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.333+0000 7f62aad9d700 1 -- 172.21.15.45:0/4599398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009550 con 0x7f62ac07be90 2023-10-17T01:06:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.335+0000 7f62aad9d700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac19c390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f629c003e60 tx=0x7f629c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.335+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f629c003a60 con 0x7f62ac07be90 2023-10-17T01:06:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.335+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c003bc0 con 0x7f62ac07be90 2023-10-17T01:06:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.335+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f629c0196c0 con 0x7f62ac07be90 2023-10-17T01:06:38.390 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.335+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac19cad0 con 0x7f62ac07be90 2023-10-17T01:06:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.335+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac19cf70 con 0x7f62ac07be90 2023-10-17T01:06:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.336+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f629c019930 con 0x7f62ac07be90 2023-10-17T01:06:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.337+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac10c8b0 con 0x7f62ac07be90 2023-10-17T01:06:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.337+0000 7f6293fff700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6294038580 0x7f629403aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:38.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.337+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f629c04ca30 con 0x7f62ac07be90 2023-10-17T01:06:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.337+0000 7f62aa59c700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6294038580 0x7f629403aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.338+0000 7f62aa59c700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6294038580 0x7f629403aa40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62a0006fd0 tx=0x7f62a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.342+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f629c010bb0 con 0x7f62ac07be90 2023-10-17T01:06:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.579+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f62ac10caa0 con 0x7f62ac07be90 2023-10-17T01:06:38.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.579+0000 7f6293fff700 1 -- 172.21.15.45:0/4599398 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f629c010380 con 0x7f62ac07be90 2023-10-17T01:06:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6294038580 msgr2=0x7f629403aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6294038580 0x7f629403aa40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62a0006fd0 tx=0x7f62a0006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 msgr2=0x7f62ac19c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac19c390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f629c003e60 tx=0x7f629c011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 shutdown_connections 2023-10-17T01:06:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6294038580 0x7f629403aa40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 --2- 172.21.15.45:0/4599398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ac07be90 0x7f62ac19c390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 >> 172.21.15.45:0/4599398 conn(0x7f62ac101a30 msgr2=0x7f62ac103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 shutdown_connections 2023-10-17T01:06:38.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:37.582+0000 7f62b1124700 1 -- 172.21.15.45:0/4599398 wait complete. 2023-10-17T01:06:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-17T01:06:38.402 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-10-17T01:06:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-17T01:06:38.420 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-10-17T01:06:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-10-17T01:06:38.421 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-17T01:06:38.423 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2023-10-17T01:06:38.423 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2023-10-17T01:06:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2023-10-17T01:06:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: Password: zvazf5kjwd 2023-10-17T01:06:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config directory 2023-10-17T01:06:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2023-10-17T01:06:38.682 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:38 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4599398' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-17T01:06:39.612 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.943+0000 7f6f55764700 1 Processor -- start 2023-10-17T01:06:39.612 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.944+0000 7f6f55764700 1 -- start start 2023-10-17T01:06:39.612 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.944+0000 7f6f55764700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f501068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:39.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.944+0000 7f6f55764700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50106e80 con 0x7f6f50106490 2023-10-17T01:06:39.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.944+0000 7f6f4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f501068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:39.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.944+0000 7f6f4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f501068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52836/0 (socket says 172.21.15.45:52836) 2023-10-17T01:06:39.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.944+0000 7f6f4effd700 1 -- 172.21.15.45:0/93951655 learned_addr learned my addr 172.21.15.45:0/93951655 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:39.613 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.945+0000 7f6f4effd700 1 -- 172.21.15.45:0/93951655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50107690 con 0x7f6f50106490 2023-10-17T01:06:39.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.945+0000 7f6f4effd700 1 --2- 172.21.15.45:0/93951655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f501068b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f400098a0 tx=0x7f6f40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ee060ef7e82a01 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:39.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.945+0000 7f6f4dffb700 1 -- 172.21.15.45:0/93951655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f40004160 con 0x7f6f50106490 2023-10-17T01:06:39.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.945+0000 7f6f4dffb700 1 -- 172.21.15.45:0/93951655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f400042c0 con 0x7f6f50106490 2023-10-17T01:06:39.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.945+0000 7f6f4dffb700 1 -- 172.21.15.45:0/93951655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f400119b0 con 0x7f6f50106490 2023-10-17T01:06:39.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 -- 172.21.15.45:0/93951655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 msgr2=0x7f6f501068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:39.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 --2- 172.21.15.45:0/93951655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f501068b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f400098a0 tx=0x7f6f40009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:39.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 -- 172.21.15.45:0/93951655 shutdown_connections 2023-10-17T01:06:39.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 --2- 172.21.15.45:0/93951655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f501068b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:39.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 -- 172.21.15.45:0/93951655 >> 172.21.15.45:0/93951655 conn(0x7f6f50101a30 msgr2=0x7f6f50103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:39.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 -- 172.21.15.45:0/93951655 shutdown_connections 2023-10-17T01:06:39.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.946+0000 7f6f55764700 1 -- 172.21.15.45:0/93951655 wait complete. 2023-10-17T01:06:39.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.947+0000 7f6f55764700 1 Processor -- start 2023-10-17T01:06:39.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.947+0000 7f6f55764700 1 -- start start 2023-10-17T01:06:39.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.947+0000 7f6f55764700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f5019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:39.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.947+0000 7f6f55764700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5019a6d0 con 0x7f6f50106490 2023-10-17T01:06:39.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.947+0000 7f6f4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f5019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:39.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f5019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52850/0 (socket says 172.21.15.45:52850) 2023-10-17T01:06:39.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f4effd700 1 -- 172.21.15.45:0/3798909306 learned_addr learned my addr 172.21.15.45:0/3798909306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:39.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f4effd700 1 -- 172.21.15.45:0/3798909306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40009550 con 0x7f6f50106490 2023-10-17T01:06:39.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f4effd700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f5019a190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f40003e60 tx=0x7f6f40011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:39.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f40003a60 con 0x7f6f50106490 2023-10-17T01:06:39.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f40003bc0 con 0x7f6f50106490 2023-10-17T01:06:39.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f400196c0 con 0x7f6f50106490 2023-10-17T01:06:39.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.948+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5019a8d0 con 0x7f6f50106490 2023-10-17T01:06:39.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.949+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5019ad70 con 0x7f6f50106490 2023-10-17T01:06:39.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.950+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5010a8c0 con 0x7f6f50106490 2023-10-17T01:06:39.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.950+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f6f4001ab60 con 0x7f6f50106490 2023-10-17T01:06:39.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.950+0000 7f6f37fff700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6f38038580 0x7f6f3803aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:39.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.950+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6f4004c830 con 0x7f6f50106490 2023-10-17T01:06:39.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.950+0000 7f6f4e7fc700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6f38038580 0x7f6f3803aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:39.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.951+0000 7f6f4e7fc700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6f38038580 0x7f6f3803aa40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006fd0 tx=0x7f6f44006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:39.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:38.955+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f6f40019930 con 0x7f6f50106490 2023-10-17T01:06:39.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.164+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f6f5010aab0 con 0x7f6f50106490 2023-10-17T01:06:39.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.164+0000 7f6f37fff700 1 -- 172.21.15.45:0/3798909306 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6f40018350 con 0x7f6f50106490 2023-10-17T01:06:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6f38038580 msgr2=0x7f6f3803aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6f38038580 0x7f6f3803aa40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006fd0 tx=0x7f6f44006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 msgr2=0x7f6f5019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f5019a190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f40003e60 tx=0x7f6f40011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 shutdown_connections 2023-10-17T01:06:39.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6f38038580 0x7f6f3803aa40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:39.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 --2- 172.21.15.45:0/3798909306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f50106490 0x7f6f5019a190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:39.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 >> 172.21.15.45:0/3798909306 conn(0x7f6f50101a30 msgr2=0x7f6f50103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:39.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 shutdown_connections 2023-10-17T01:06:39.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:39.167+0000 7f6f55764700 1 -- 172.21.15.45:0/3798909306 wait complete. 2023-10-17T01:06:40.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:40 smithi045 ceph-mon[98474]: mgrmap e13: x(active, since 6s) 2023-10-17T01:06:40.997 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.225+0000 7f66cad25700 1 Processor -- start 2023-10-17T01:06:40.997 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.225+0000 7f66cad25700 1 -- start start 2023-10-17T01:06:40.997 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.225+0000 7f66cad25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c41068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:40.998 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.225+0000 7f66cad25700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4106e80 con 0x7f66c4106490 2023-10-17T01:06:40.998 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.226+0000 7f66c8ac1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c41068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:40.998 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.226+0000 7f66c8ac1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c41068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52854/0 (socket says 172.21.15.45:52854) 2023-10-17T01:06:40.998 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.226+0000 7f66c8ac1700 1 -- 172.21.15.45:0/2704215332 learned_addr learned my addr 172.21.15.45:0/2704215332 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:40.998 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.226+0000 7f66c8ac1700 1 -- 172.21.15.45:0/2704215332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4107690 con 0x7f66c4106490 2023-10-17T01:06:40.999 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.226+0000 7f66c8ac1700 1 --2- 172.21.15.45:0/2704215332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c41068b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66b80098a0 tx=0x7f66b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d546ed1cf316c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:40.999 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66c37fe700 1 -- 172.21.15.45:0/2704215332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66b8004160 con 0x7f66c4106490 2023-10-17T01:06:40.999 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66c37fe700 1 -- 172.21.15.45:0/2704215332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b80042c0 con 0x7f66c4106490 2023-10-17T01:06:40.999 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66c37fe700 1 -- 172.21.15.45:0/2704215332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66b80119b0 con 0x7f66c4106490 2023-10-17T01:06:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 -- 172.21.15.45:0/2704215332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 msgr2=0x7f66c41068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 --2- 172.21.15.45:0/2704215332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c41068b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66b80098a0 tx=0x7f66b8009bb0 comp rx=0 tx=0).stop 2023-10-17T01:06:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 -- 172.21.15.45:0/2704215332 shutdown_connections 2023-10-17T01:06:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 --2- 172.21.15.45:0/2704215332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c41068b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 -- 172.21.15.45:0/2704215332 >> 172.21.15.45:0/2704215332 conn(0x7f66c4101a30 msgr2=0x7f66c4103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 -- 172.21.15.45:0/2704215332 shutdown_connections 2023-10-17T01:06:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.227+0000 7f66cad25700 1 -- 172.21.15.45:0/2704215332 wait complete. 2023-10-17T01:06:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.229+0000 7f66cad25700 1 Processor -- start 2023-10-17T01:06:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.229+0000 7f66cad25700 1 -- start start 2023-10-17T01:06:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.229+0000 7f66cad25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.229+0000 7f66cad25700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c419c8d0 con 0x7f66c4106490 2023-10-17T01:06:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.230+0000 7f66c8ac1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.230+0000 7f66c8ac1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52866/0 (socket says 172.21.15.45:52866) 2023-10-17T01:06:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.230+0000 7f66c8ac1700 1 -- 172.21.15.45:0/4051289997 learned_addr learned my addr 172.21.15.45:0/4051289997 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.230+0000 7f66c8ac1700 1 -- 172.21.15.45:0/4051289997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8009550 con 0x7f66c4106490 2023-10-17T01:06:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.230+0000 7f66c8ac1700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c419c390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66b8003e60 tx=0x7f66b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.230+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66b8003a60 con 0x7f66c4106490 2023-10-17T01:06:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.231+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b8003bc0 con 0x7f66c4106490 2023-10-17T01:06:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.231+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66b80196c0 con 0x7f66c4106490 2023-10-17T01:06:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.231+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c419cad0 con 0x7f66c4106490 2023-10-17T01:06:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.231+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c419cf70 con 0x7f66c4106490 2023-10-17T01:06:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.232+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c410a8c0 con 0x7f66c4106490 2023-10-17T01:06:41.005 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.232+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f66b801ab60 con 0x7f66c4106490 2023-10-17T01:06:41.005 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.232+0000 7f66c1ffb700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f66b4038620 0x7f66b403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:41.005 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.232+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f66b804c950 con 0x7f66c4106490 2023-10-17T01:06:41.005 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.233+0000 7f66c3fff700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f66b4038620 0x7f66b403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:41.006 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.233+0000 7f66c3fff700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f66b4038620 0x7f66b403aae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f66b0006fd0 tx=0x7f66b0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:41.006 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.237+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f66b8018b00 con 0x7f66c4106490 2023-10-17T01:06:41.006 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.564+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f66c410aab0 con 0x7f66c4106490 2023-10-17T01:06:41.007 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.566+0000 7f66c1ffb700 1 -- 172.21.15.45:0/4051289997 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f66b8018b00 con 0x7f66c4106490 2023-10-17T01:06:41.007 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f66b4038620 msgr2=0x7f66b403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:41.007 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f66b4038620 0x7f66b403aae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f66b0006fd0 tx=0x7f66b0006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:41.007 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 msgr2=0x7f66c419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:41.007 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c419c390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66b8003e60 tx=0x7f66b8011a30 comp rx=0 tx=0).stop 2023-10-17T01:06:41.007 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 shutdown_connections 2023-10-17T01:06:41.008 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f66b4038620 0x7f66b403aae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:41.008 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 --2- 172.21.15.45:0/4051289997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66c4106490 0x7f66c419c390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:41.008 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 >> 172.21.15.45:0/4051289997 conn(0x7f66c4101a30 msgr2=0x7f66c41030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:41.008 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 shutdown_connections 2023-10-17T01:06:41.008 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2023-10-17T01:06:40.569+0000 7f66cad25700 1 -- 172.21.15.45:0/4051289997 wait complete. 2023-10-17T01:06:41.009 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-17T01:06:41.009 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-17T01:06:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:06:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.010 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2023-10-17T01:06:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-17T01:06:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.010 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-17T01:06:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2023-10-17T01:06:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.011 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2023-10-17T01:06:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-17T01:06:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:41.012 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2023-10-17T01:06:41.054 INFO:tasks.cephadm:Fetching config... 2023-10-17T01:06:41.055 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:06:41.055 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-17T01:06:41.099 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-17T01:06:41.099 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:06:41.100 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-17T01:06:41.184 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-17T01:06:41.185 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:06:41.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-17T01:06:41.297 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-17T01:06:41.298 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:06:41.298 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-17T01:06:41.385 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-17T01:06:41.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-17T01:06:41.527 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:06:41.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-17T01:06:41.606 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:06:41.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-17T01:06:41.664 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkTjP4ktotjRXaY5BtHcg9fs1b+A97NZKtCFE0w9AcaEdAK192lVgwu/CWFLHXdFRIJgg8+k07iihRQXbDwvjQKqyVVHHKvgKRAnWl+HiYKOB3Kj21e35Vyj/LPszEsHsEKzQOQEmoh+tmz8wQZze//fdhbpcK4MpLkmlODOlysRWvq2R571Yyse3uliaS6wmLZsT+p2Y4wfs+bUPtQ8SovPjy1D9dQbnSbKaPZW4csV0fQwNYH5TxlEotLu7Sq/k5sf0Rb5dtn1ONwXzt/Khs6iknH9a17eQJP1splbhI6Up9U4MRheJwRx8gCA9fzb+CFUdsg6dR6kXKkCShTb5Ijvnddbr3z2lZ6jbG7Aik3a6BIvDHYuaXTAeDxCsoO3ulwhRaqryovlGrSwPXJJ2sBK063vyDwo16dwQ4/gPiPSpbFi4QPxBTywkz+xCslxM2jKcNrAV/FlmXB09OFsrSre8kn+y9AiVclpxEZooEJLHoszDFejO/EKAW+cd+n4M= ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:06:41.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-17T01:06:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:41 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4051289997' entity='client.admin' 2023-10-17T01:06:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:06:44.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 -- 172.21.15.45:0/4288023513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 msgr2=0x7f6438103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:44.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 --2- 172.21.15.45:0/4288023513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f6438103720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6428009910 tx=0x7f6428009c20 comp rx=0 tx=0).stop 2023-10-17T01:06:44.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 -- 172.21.15.45:0/4288023513 shutdown_connections 2023-10-17T01:06:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 --2- 172.21.15.45:0/4288023513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f6438103720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 -- 172.21.15.45:0/4288023513 >> 172.21.15.45:0/4288023513 conn(0x7f64380facc0 msgr2=0x7f64380fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 -- 172.21.15.45:0/4288023513 shutdown_connections 2023-10-17T01:06:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.243+0000 7f643e1b9700 1 -- 172.21.15.45:0/4288023513 wait complete. 2023-10-17T01:06:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.245+0000 7f643e1b9700 1 Processor -- start 2023-10-17T01:06:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.245+0000 7f643e1b9700 1 -- start start 2023-10-17T01:06:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.245+0000 7f643e1b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f64381958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.245+0000 7f643e1b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438195e10 con 0x7f6438101330 2023-10-17T01:06:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.246+0000 7f64377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f64381958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.246+0000 7f64377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f64381958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52896/0 (socket says 172.21.15.45:52896) 2023-10-17T01:06:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.246+0000 7f64377fe700 1 -- 172.21.15.45:0/3980204790 learned_addr learned my addr 172.21.15.45:0/3980204790 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.246+0000 7f64377fe700 1 -- 172.21.15.45:0/3980204790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280095f0 con 0x7f6438101330 2023-10-17T01:06:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.246+0000 7f64377fe700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f64381958d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6428005f50 tx=0x7f64280057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.247+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f642801d070 con 0x7f6438101330 2023-10-17T01:06:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.247+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6428022470 con 0x7f6438101330 2023-10-17T01:06:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.247+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6438196010 con 0x7f6438101330 2023-10-17T01:06:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.247+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f642800f5f0 con 0x7f6438101330 2023-10-17T01:06:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.247+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64381964b0 con 0x7f6438101330 2023-10-17T01:06:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.248+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f642800f860 con 0x7f6438101330 2023-10-17T01:06:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.248+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643818f350 con 0x7f6438101330 2023-10-17T01:06:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.248+0000 7f6434ff9700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f64200385f0 0x7f642003aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.248+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f642804c8b0 con 0x7f6438101330 2023-10-17T01:06:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.249+0000 7f6436ffd700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f64200385f0 0x7f642003aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.249+0000 7f6436ffd700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f64200385f0 0x7f642003aab0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f642c006fd0 tx=0x7f642c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.256+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f642802a360 con 0x7f6438101330 2023-10-17T01:06:44.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.468+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f643802cc30 con 0x7f6438101330 2023-10-17T01:06:44.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.471+0000 7f6434ff9700 1 -- 172.21.15.45:0/3980204790 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6428027020 con 0x7f6438101330 2023-10-17T01:06:44.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f64200385f0 msgr2=0x7f642003aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f64200385f0 0x7f642003aab0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f642c006fd0 tx=0x7f642c006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 msgr2=0x7f64381958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f64381958d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6428005f50 tx=0x7f64280057c0 comp rx=0 tx=0).stop 2023-10-17T01:06:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 shutdown_connections 2023-10-17T01:06:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f64200385f0 0x7f642003aab0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 --2- 172.21.15.45:0/3980204790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438101330 0x7f64381958d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 >> 172.21.15.45:0/3980204790 conn(0x7f64380facc0 msgr2=0x7f64380fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 shutdown_connections 2023-10-17T01:06:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:44.474+0000 7f643e1b9700 1 -- 172.21.15.45:0/3980204790 wait complete. 2023-10-17T01:06:44.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:44.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:44.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:06:44.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:44.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:44.639 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:45.006 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-17T01:06:45.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-17T01:06:45.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:45 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3980204790' entity='client.admin' 2023-10-17T01:06:45.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:45.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:45.800 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:06:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 -- 172.21.15.45:0/308374955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 msgr2=0x7f00a0100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 --2- 172.21.15.45:0/308374955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0100c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f009c009910 tx=0x7f009c009c20 comp rx=0 tx=0).stop 2023-10-17T01:06:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 -- 172.21.15.45:0/308374955 shutdown_connections 2023-10-17T01:06:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 --2- 172.21.15.45:0/308374955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0100c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 -- 172.21.15.45:0/308374955 >> 172.21.15.45:0/308374955 conn(0x7f00a00fbde0 msgr2=0x7f00a00fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 -- 172.21.15.45:0/308374955 shutdown_connections 2023-10-17T01:06:47.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.288+0000 7f00a84ce700 1 -- 172.21.15.45:0/308374955 wait complete. 2023-10-17T01:06:47.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.289+0000 7f00a84ce700 1 Processor -- start 2023-10-17T01:06:47.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.289+0000 7f00a84ce700 1 -- start start 2023-10-17T01:06:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.289+0000 7f00a84ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00a84ce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a01980a0 con 0x7f00a0100860 2023-10-17T01:06:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00a626a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00a626a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52920/0 (socket says 172.21.15.45:52920) 2023-10-17T01:06:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00a626a700 1 -- 172.21.15.45:0/2019529318 learned_addr learned my addr 172.21.15.45:0/2019529318 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00a626a700 1 -- 172.21.15.45:0/2019529318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0095f0 con 0x7f00a0100860 2023-10-17T01:06:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00a626a700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f009c006010 tx=0x7f009c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.290+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f009c01d070 con 0x7f00a0100860 2023-10-17T01:06:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.291+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f009c0044e0 con 0x7f00a0100860 2023-10-17T01:06:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.291+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f009c0039b0 con 0x7f00a0100860 2023-10-17T01:06:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.291+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a01982a0 con 0x7f00a0100860 2023-10-17T01:06:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.291+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a01986e0 con 0x7f00a0100860 2023-10-17T01:06:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.292+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a01065b0 con 0x7f00a0100860 2023-10-17T01:06:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.292+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f009c00baf0 con 0x7f00a0100860 2023-10-17T01:06:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.292+0000 7f00977fe700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f008c0385f0 0x7f008c03aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.292+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f009c04c6d0 con 0x7f00a0100860 2023-10-17T01:06:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.292+0000 7f00a5a69700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f008c0385f0 0x7f008c03aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.293+0000 7f00a5a69700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f008c0385f0 0x7f008c03aab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0090006fd0 tx=0x7f0090006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.297+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f009c02a950 con 0x7f00a0100860 2023-10-17T01:06:47.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.515+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f00a0073730 con 0x7f008c0385f0 2023-10-17T01:06:47.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.519+0000 7f00977fe700 1 -- 172.21.15.45:0/2019529318 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00a0073730 con 0x7f008c0385f0 2023-10-17T01:06:47.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.521+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f008c0385f0 msgr2=0x7f008c03aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:47.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.521+0000 7f00a84ce700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f008c0385f0 0x7f008c03aab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0090006fd0 tx=0x7f0090006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:47.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.521+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 msgr2=0x7f00a0197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:47.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.521+0000 7f00a84ce700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0197b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f009c006010 tx=0x7f009c0057c0 comp rx=0 tx=0).stop 2023-10-17T01:06:47.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.521+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 shutdown_connections 2023-10-17T01:06:47.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.522+0000 7f00a84ce700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f008c0385f0 0x7f008c03aab0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:47.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.522+0000 7f00a84ce700 1 --2- 172.21.15.45:0/2019529318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0100860 0x7f00a0197b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:47.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.522+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 >> 172.21.15.45:0/2019529318 conn(0x7f00a00fbde0 msgr2=0x7f00a0073020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:47.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.522+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 shutdown_connections 2023-10-17T01:06:47.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:47.522+0000 7f00a84ce700 1 -- 172.21.15.45:0/2019529318 wait complete. 2023-10-17T01:06:47.983 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi060 2023-10-17T01:06:47.984 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:06:47.984 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/etc/ceph/ceph.conf 2023-10-17T01:06:48.006 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:06:48.006 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:06:48.066 INFO:tasks.cephadm:Adding host smithi060 to orchestrator... 2023-10-17T01:06:48.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi060 2023-10-17T01:06:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:48.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:48.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:06:48.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:06:48.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: Updating smithi045:/etc/ceph/ceph.conf 2023-10-17T01:06:48.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:48 smithi045 ceph-mon[98474]: Updating smithi045:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:06:48.833 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:06:49.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:49 smithi045 ceph-mon[98474]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:06:49.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:49 smithi045 ceph-mon[98474]: Updating smithi045:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-17T01:06:50.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 -- 172.21.15.45:0/2361845174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 msgr2=0x7f005c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 --2- 172.21.15.45:0/2361845174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c106bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0050009910 tx=0x7f0050009c20 comp rx=0 tx=0).stop 2023-10-17T01:06:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 -- 172.21.15.45:0/2361845174 shutdown_connections 2023-10-17T01:06:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 --2- 172.21.15.45:0/2361845174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c106bc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 -- 172.21.15.45:0/2361845174 >> 172.21.15.45:0/2361845174 conn(0x7f005c0fc310 msgr2=0x7f005c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 -- 172.21.15.45:0/2361845174 shutdown_connections 2023-10-17T01:06:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.333+0000 7f006316b700 1 -- 172.21.15.45:0/2361845174 wait complete. 2023-10-17T01:06:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.334+0000 7f006316b700 1 Processor -- start 2023-10-17T01:06:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f006316b700 1 -- start start 2023-10-17T01:06:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f006316b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f006316b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c1961f0 con 0x7f005c1067e0 2023-10-17T01:06:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f0060f07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f0060f07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51742/0 (socket says 172.21.15.45:51742) 2023-10-17T01:06:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f0060f07700 1 -- 172.21.15.45:0/3024168837 learned_addr learned my addr 172.21.15.45:0/3024168837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.335+0000 7f0060f07700 1 -- 172.21.15.45:0/3024168837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500095f0 con 0x7f005c1067e0 2023-10-17T01:06:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.336+0000 7f0060f07700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c195cb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0050006010 tx=0x7f0050004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.336+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f005001d070 con 0x7f005c1067e0 2023-10-17T01:06:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.336+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f005000bbb0 con 0x7f005c1067e0 2023-10-17T01:06:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.336+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0050003930 con 0x7f005c1067e0 2023-10-17T01:06:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.336+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c196450 con 0x7f005c1067e0 2023-10-17T01:06:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.336+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c1968d0 con 0x7f005c1067e0 2023-10-17T01:06:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.337+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f005000bd20 con 0x7f005c1067e0 2023-10-17T01:06:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.337+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c062380 con 0x7f005c1067e0 2023-10-17T01:06:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.338+0000 7f0059ffb700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f004c0385f0 0x7f004c03aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.338+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f005004c340 con 0x7f005c1067e0 2023-10-17T01:06:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.338+0000 7f005bfff700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f004c0385f0 0x7f004c03aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.338+0000 7f005bfff700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f004c0385f0 0x7f004c03aab0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0048006fd0 tx=0x7f0048006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:50.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.343+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0050029360 con 0x7f005c1067e0 2023-10-17T01:06:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:50.560+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi060", "target": ["mon-mgr", ""]}) v1 -- 0x7f005c1045b0 con 0x7f004c0385f0 2023-10-17T01:06:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:51.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:52.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:51 smithi045 ceph-mon[98474]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:06:52.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:51 smithi045 ceph-mon[98474]: Deploying cephadm binary to smithi060 2023-10-17T01:06:52.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.974+0000 7f0059ffb700 1 -- 172.21.15.45:0/3024168837 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f005c1045b0 con 0x7f004c0385f0 2023-10-17T01:06:52.975 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi060' with addr '172.21.15.60' 2023-10-17T01:06:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f004c0385f0 msgr2=0x7f004c03aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f004c0385f0 0x7f004c03aab0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0048006fd0 tx=0x7f0048006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 msgr2=0x7f005c195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c195cb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0050006010 tx=0x7f0050004ff0 comp rx=0 tx=0).stop 2023-10-17T01:06:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 shutdown_connections 2023-10-17T01:06:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f004c0385f0 0x7f004c03aab0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 --2- 172.21.15.45:0/3024168837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f005c1067e0 0x7f005c195cb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 >> 172.21.15.45:0/3024168837 conn(0x7f005c0fc310 msgr2=0x7f005c108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 shutdown_connections 2023-10-17T01:06:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:52.976+0000 7f006316b700 1 -- 172.21.15.45:0/3024168837 wait complete. 2023-10-17T01:06:53.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-17T01:06:53.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:53 smithi045 ceph-mon[98474]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:06:53.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:53 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:53.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:53 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:06:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:06:54.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:54 smithi045 ceph-mon[98474]: Added host smithi060 2023-10-17T01:06:54.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:54.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 -- 172.21.15.45:0/3557817481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 msgr2=0x7f5dd4106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 --2- 172.21.15.45:0/3557817481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4106bc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009910 tx=0x7f5dd0009c20 comp rx=0 tx=0).stop 2023-10-17T01:06:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 -- 172.21.15.45:0/3557817481 shutdown_connections 2023-10-17T01:06:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 --2- 172.21.15.45:0/3557817481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4106bc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 -- 172.21.15.45:0/3557817481 >> 172.21.15.45:0/3557817481 conn(0x7f5dd40fc310 msgr2=0x7f5dd40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 -- 172.21.15.45:0/3557817481 shutdown_connections 2023-10-17T01:06:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.665+0000 7f5ddb490700 1 -- 172.21.15.45:0/3557817481 wait complete. 2023-10-17T01:06:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.667+0000 7f5ddb490700 1 Processor -- start 2023-10-17T01:06:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.667+0000 7f5ddb490700 1 -- start start 2023-10-17T01:06:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.668+0000 7f5ddb490700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.668+0000 7f5ddb490700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4074fd0 con 0x7f5dd41067e0 2023-10-17T01:06:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.668+0000 7f5dd922c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.668+0000 7f5dd922c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51756/0 (socket says 172.21.15.45:51756) 2023-10-17T01:06:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.668+0000 7f5dd922c700 1 -- 172.21.15.45:0/1664966710 learned_addr learned my addr 172.21.15.45:0/1664966710 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.668+0000 7f5dd922c700 1 -- 172.21.15.45:0/1664966710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd00095f0 con 0x7f5dd41067e0 2023-10-17T01:06:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.669+0000 7f5dd922c700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4073050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0006010 tx=0x7f5dd0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.669+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dd001d070 con 0x7f5dd41067e0 2023-10-17T01:06:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.669+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd000bc10 con 0x7f5dd41067e0 2023-10-17T01:06:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.669+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd4073590 con 0x7f5dd41067e0 2023-10-17T01:06:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.669+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dd0003910 con 0x7f5dd41067e0 2023-10-17T01:06:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.669+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd4073a10 con 0x7f5dd41067e0 2023-10-17T01:06:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.670+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f5dd000bd80 con 0x7f5dd41067e0 2023-10-17T01:06:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.670+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd4104a90 con 0x7f5dd41067e0 2023-10-17T01:06:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.671+0000 7f5dca7fc700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5dc00385a0 0x7f5dc003aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.671+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5dd004c350 con 0x7f5dd41067e0 2023-10-17T01:06:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.671+0000 7f5dd8a2b700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5dc00385a0 0x7f5dc003aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.671+0000 7f5dd8a2b700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5dc00385a0 0x7f5dc003aa60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006fd0 tx=0x7f5dc4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:55.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.676+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f5dd0029360 con 0x7f5dd41067e0 2023-10-17T01:06:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:55 smithi045 ceph-mon[98474]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:06:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.894+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5dd4073d10 con 0x7f5dc00385a0 2023-10-17T01:06:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.895+0000 7f5dca7fc700 1 -- 172.21.15.45:0/1664966710 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f5dd4073d10 con 0x7f5dc00385a0 2023-10-17T01:06:55.897 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:06:55.897 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}] 2023-10-17T01:06:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.897+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5dc00385a0 msgr2=0x7f5dc003aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.897+0000 7f5ddb490700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5dc00385a0 0x7f5dc003aa60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006fd0 tx=0x7f5dc4006e40 comp rx=0 tx=0).stop 2023-10-17T01:06:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.897+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 msgr2=0x7f5dd4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.897+0000 7f5ddb490700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4073050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0006010 tx=0x7f5dd0004ff0 comp rx=0 tx=0).stop 2023-10-17T01:06:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.898+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 shutdown_connections 2023-10-17T01:06:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.898+0000 7f5ddb490700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5dc00385a0 0x7f5dc003aa60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.898+0000 7f5ddb490700 1 --2- 172.21.15.45:0/1664966710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd41067e0 0x7f5dd4073050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.898+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 >> 172.21.15.45:0/1664966710 conn(0x7f5dd40fc310 msgr2=0x7f5dd4108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.898+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 shutdown_connections 2023-10-17T01:06:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:55.898+0000 7f5ddb490700 1 -- 172.21.15.45:0/1664966710 wait complete. 2023-10-17T01:06:56.354 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2023-10-17T01:06:56.354 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:06:56.354 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2023-10-17T01:06:56.375 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:06:56.375 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:06:56.434 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2023-10-17T01:06:56.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi181 2023-10-17T01:06:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:06:57.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:57 smithi045 ceph-mon[98474]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:06:57.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:57 smithi045 ceph-mon[98474]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:06:58.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.660+0000 7f7c136d8700 1 -- 172.21.15.45:0/66769572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 msgr2=0x7f7c0c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:06:58.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.660+0000 7f7c136d8700 1 --2- 172.21.15.45:0/66769572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c106bc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009910 tx=0x7f7c08009c20 comp rx=0 tx=0).stop 2023-10-17T01:06:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.660+0000 7f7c136d8700 1 -- 172.21.15.45:0/66769572 shutdown_connections 2023-10-17T01:06:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.660+0000 7f7c136d8700 1 --2- 172.21.15.45:0/66769572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c106bc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:06:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.660+0000 7f7c136d8700 1 -- 172.21.15.45:0/66769572 >> 172.21.15.45:0/66769572 conn(0x7f7c0c075250 msgr2=0x7f7c0c075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:06:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.661+0000 7f7c136d8700 1 -- 172.21.15.45:0/66769572 shutdown_connections 2023-10-17T01:06:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.661+0000 7f7c136d8700 1 -- 172.21.15.45:0/66769572 wait complete. 2023-10-17T01:06:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.661+0000 7f7c136d8700 1 Processor -- start 2023-10-17T01:06:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.662+0000 7f7c136d8700 1 -- start start 2023-10-17T01:06:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.662+0000 7f7c136d8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.662+0000 7f7c136d8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c1983d0 con 0x7f7c0c1067e0 2023-10-17T01:06:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.662+0000 7f7c11474700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.662+0000 7f7c11474700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56662/0 (socket says 172.21.15.45:56662) 2023-10-17T01:06:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.662+0000 7f7c11474700 1 -- 172.21.15.45:0/3071443942 learned_addr learned my addr 172.21.15.45:0/3071443942 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:06:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c11474700 1 -- 172.21.15.45:0/3071443942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080095f0 con 0x7f7c0c1067e0 2023-10-17T01:06:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c11474700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c197e90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006010 tx=0x7f7c08004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0801d070 con 0x7f7c0c1067e0 2023-10-17T01:06:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c0800bbb0 con 0x7f7c0c1067e0 2023-10-17T01:06:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c08003930 con 0x7f7c0c1067e0 2023-10-17T01:06:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c198630 con 0x7f7c0c1067e0 2023-10-17T01:06:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.663+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c198ab0 con 0x7f7c0c1067e0 2023-10-17T01:06:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.664+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c062380 con 0x7f7c0c1067e0 2023-10-17T01:06:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.664+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f7c0800bd20 con 0x7f7c0c1067e0 2023-10-17T01:06:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.665+0000 7f7c027fc700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bf80385f0 0x7f7bf803aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:06:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.665+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c0804c330 con 0x7f7c0c1067e0 2023-10-17T01:06:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.665+0000 7f7c10c73700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bf80385f0 0x7f7bf803aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:06:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.666+0000 7f7c10c73700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bf80385f0 0x7f7bf803aab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:06:58.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.670+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7c08029770 con 0x7f7c0c1067e0 2023-10-17T01:06:58.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:06:58.886+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c0c0fefd0 con 0x7f7bf80385f0 2023-10-17T01:06:59.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:06:59.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:59.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:59.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:59.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:06:59.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:06:59.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:06:59.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:06:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:00.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: Updating smithi060:/etc/ceph/ceph.conf 2023-10-17T01:07:00.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:00.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:00.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:00.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-17T01:07:00.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: Deploying cephadm binary to smithi181 2023-10-17T01:07:00.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:00.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:00.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:01.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.293+0000 7f7c027fc700 1 -- 172.21.15.45:0/3071443942 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7c0c0fefd0 con 0x7f7bf80385f0 2023-10-17T01:07:01.295 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi181' with addr '172.21.15.181' 2023-10-17T01:07:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bf80385f0 msgr2=0x7f7bf803aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bf80385f0 0x7f7bf803aab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc006e40 comp rx=0 tx=0).stop 2023-10-17T01:07:01.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 msgr2=0x7f7c0c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c197e90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006010 tx=0x7f7c08004ff0 comp rx=0 tx=0).stop 2023-10-17T01:07:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 shutdown_connections 2023-10-17T01:07:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bf80385f0 0x7f7bf803aab0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 --2- 172.21.15.45:0/3071443942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1067e0 0x7f7c0c197e90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 >> 172.21.15.45:0/3071443942 conn(0x7f7c0c075250 msgr2=0x7f7c0c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:01.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 shutdown_connections 2023-10-17T01:07:01.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:01.296+0000 7f7c136d8700 1 -- 172.21.15.45:0/3071443942 wait complete. 2023-10-17T01:07:01.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:01 smithi045 ceph-mon[98474]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:01.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-17T01:07:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:07:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:02 smithi045 ceph-mon[98474]: Added host smithi181 2023-10-17T01:07:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:02.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:03.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:03.741 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:03 smithi045 ceph-mon[98474]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 -- 172.21.15.45:0/1570004671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 msgr2=0x7f622c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 --2- 172.21.15.45:0/1570004671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c106bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6228009910 tx=0x7f6228009c20 comp rx=0 tx=0).stop 2023-10-17T01:07:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 -- 172.21.15.45:0/1570004671 shutdown_connections 2023-10-17T01:07:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 --2- 172.21.15.45:0/1570004671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c106bc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 -- 172.21.15.45:0/1570004671 >> 172.21.15.45:0/1570004671 conn(0x7f622c0fc310 msgr2=0x7f622c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 -- 172.21.15.45:0/1570004671 shutdown_connections 2023-10-17T01:07:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.988+0000 7f6233c44700 1 -- 172.21.15.45:0/1570004671 wait complete. 2023-10-17T01:07:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.989+0000 7f6233c44700 1 Processor -- start 2023-10-17T01:07:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.989+0000 7f6233c44700 1 -- start start 2023-10-17T01:07:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.990+0000 7f6233c44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.990+0000 7f6233c44700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1983d0 con 0x7f622c1067e0 2023-10-17T01:07:03.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.990+0000 7f62319e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:03.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.990+0000 7f62319e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56676/0 (socket says 172.21.15.45:56676) 2023-10-17T01:07:03.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.990+0000 7f62319e0700 1 -- 172.21.15.45:0/978807915 learned_addr learned my addr 172.21.15.45:0/978807915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:07:03.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.990+0000 7f62319e0700 1 -- 172.21.15.45:0/978807915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62280095f0 con 0x7f622c1067e0 2023-10-17T01:07:03.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.991+0000 7f62319e0700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c197e90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6228006010 tx=0x7f6228004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:03.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.991+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f622801d070 con 0x7f622c1067e0 2023-10-17T01:07:03.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.991+0000 7f6233c44700 1 -- 172.21.15.45:0/978807915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c198630 con 0x7f622c1067e0 2023-10-17T01:07:03.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.991+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f622800bbb0 con 0x7f622c1067e0 2023-10-17T01:07:03.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.991+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6228003930 con 0x7f622c1067e0 2023-10-17T01:07:03.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.991+0000 7f6233c44700 1 -- 172.21.15.45:0/978807915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c198ab0 con 0x7f622c1067e0 2023-10-17T01:07:03.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.992+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f6228003b70 con 0x7f622c1067e0 2023-10-17T01:07:03.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.992+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c062380 con 0x7f622c1067e0 2023-10-17T01:07:03.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.993+0000 7f6222ffd700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f62180385a0 0x7f621803aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:03.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.993+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f622804c320 con 0x7f622c1067e0 2023-10-17T01:07:03.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.993+0000 7f62311df700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f62180385a0 0x7f621803aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:03.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.993+0000 7f62311df700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f62180385a0 0x7f621803aa60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f621c006fd0 tx=0x7f621c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:03.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:03.998+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f6228029540 con 0x7f622c1067e0 2023-10-17T01:07:04.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.215+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f622c1991f0 con 0x7f62180385a0 2023-10-17T01:07:04.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.216+0000 7f6222ffd700 1 -- 172.21.15.45:0/978807915 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f622c1991f0 con 0x7f62180385a0 2023-10-17T01:07:04.217 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:07:04.218 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2023-10-17T01:07:04.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.218+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f62180385a0 msgr2=0x7f621803aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:04.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.218+0000 7f6220ff9700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f62180385a0 0x7f621803aa60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f621c006fd0 tx=0x7f621c006e40 comp rx=0 tx=0).stop 2023-10-17T01:07:04.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.218+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 msgr2=0x7f622c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:04.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.218+0000 7f6220ff9700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c197e90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6228006010 tx=0x7f6228004ff0 comp rx=0 tx=0).stop 2023-10-17T01:07:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.219+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 shutdown_connections 2023-10-17T01:07:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.219+0000 7f6220ff9700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f62180385a0 0x7f621803aa60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.219+0000 7f6220ff9700 1 --2- 172.21.15.45:0/978807915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f622c1067e0 0x7f622c197e90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.219+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 >> 172.21.15.45:0/978807915 conn(0x7f622c0fc310 msgr2=0x7f622c108060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.219+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 shutdown_connections 2023-10-17T01:07:04.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:04.219+0000 7f6220ff9700 1 -- 172.21.15.45:0/978807915 wait complete. 2023-10-17T01:07:04.677 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-17T01:07:04.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd crush tunables default 2023-10-17T01:07:05.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:05.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:04 smithi045 ceph-mon[98474]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:07:05.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:04 smithi045 ceph-mon[98474]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:07:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 -- 172.21.15.45:0/4053208089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 msgr2=0x7f4c04106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 --2- 172.21.15.45:0/4053208089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04106bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009910 tx=0x7f4c00009c20 comp rx=0 tx=0).stop 2023-10-17T01:07:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 -- 172.21.15.45:0/4053208089 shutdown_connections 2023-10-17T01:07:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 --2- 172.21.15.45:0/4053208089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04106bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 -- 172.21.15.45:0/4053208089 >> 172.21.15.45:0/4053208089 conn(0x7f4c040fc310 msgr2=0x7f4c040fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 -- 172.21.15.45:0/4053208089 shutdown_connections 2023-10-17T01:07:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.917+0000 7f4c0c34d700 1 -- 172.21.15.45:0/4053208089 wait complete. 2023-10-17T01:07:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.918+0000 7f4c0c34d700 1 Processor -- start 2023-10-17T01:07:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.918+0000 7f4c0c34d700 1 -- start start 2023-10-17T01:07:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.919+0000 7f4c0c34d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.919+0000 7f4c0c34d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c041983e0 con 0x7f4c041067e0 2023-10-17T01:07:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.919+0000 7f4c0a0e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.919+0000 7f4c0a0e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56686/0 (socket says 172.21.15.45:56686) 2023-10-17T01:07:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.919+0000 7f4c0a0e9700 1 -- 172.21.15.45:0/1504744739 learned_addr learned my addr 172.21.15.45:0/1504744739 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:07:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.919+0000 7f4c0a0e9700 1 -- 172.21.15.45:0/1504744739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000095f0 con 0x7f4c041067e0 2023-10-17T01:07:06.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.920+0000 7f4c0a0e9700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04197ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4c00006010 tx=0x7f4c00004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:06.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.920+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c0001d070 con 0x7f4c041067e0 2023-10-17T01:07:06.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.920+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0000bbb0 con 0x7f4c041067e0 2023-10-17T01:07:06.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.920+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c04198640 con 0x7f4c041067e0 2023-10-17T01:07:06.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.920+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c00003930 con 0x7f4c041067e0 2023-10-17T01:07:06.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.920+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c04198ac0 con 0x7f4c041067e0 2023-10-17T01:07:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.921+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c04062380 con 0x7f4c041067e0 2023-10-17T01:07:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.922+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f4c0000bd20 con 0x7f4c041067e0 2023-10-17T01:07:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.922+0000 7f4bfb7fe700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4bf00385f0 0x7f4bf003aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.922+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4c0004c330 con 0x7f4c041067e0 2023-10-17T01:07:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.922+0000 7f4c098e8700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4bf00385f0 0x7f4bf003aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.923+0000 7f4c098e8700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4bf00385f0 0x7f4bf003aab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4006fd0 tx=0x7f4bf4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:06.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:06.929+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f4c00029770 con 0x7f4c041067e0 2023-10-17T01:07:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.191+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4c0419c990 con 0x7f4c041067e0 2023-10-17T01:07:07.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.568+0000 7f4bfb7fe700 1 -- 172.21.15.45:0/1504744739 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4c00026020 con 0x7f4c041067e0 2023-10-17T01:07:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4bf00385f0 msgr2=0x7f4bf003aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4bf00385f0 0x7f4bf003aab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4006fd0 tx=0x7f4bf4006e40 comp rx=0 tx=0).stop 2023-10-17T01:07:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 msgr2=0x7f4c04197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04197ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4c00006010 tx=0x7f4c00004ff0 comp rx=0 tx=0).stop 2023-10-17T01:07:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 shutdown_connections 2023-10-17T01:07:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4bf00385f0 0x7f4bf003aab0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 --2- 172.21.15.45:0/1504744739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c041067e0 0x7f4c04197ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 >> 172.21.15.45:0/1504744739 conn(0x7f4c040fc310 msgr2=0x7f4c04108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 shutdown_connections 2023-10-17T01:07:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:07.571+0000 7f4c0c34d700 1 -- 172.21.15.45:0/1504744739 wait complete. 2023-10-17T01:07:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2023-10-17T01:07:07.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:07.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:07.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:07.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:07.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:07.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:07:07.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:07.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:07.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:07 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1504744739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-17T01:07:08.099 INFO:tasks.cephadm:Adding mon.a on smithi045 2023-10-17T01:07:08.099 INFO:tasks.cephadm:Adding mon.b on smithi060 2023-10-17T01:07:08.099 INFO:tasks.cephadm:Adding mon.c on smithi181 2023-10-17T01:07:08.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c' 2023-10-17T01:07:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: Updating smithi181:/etc/ceph/ceph.conf 2023-10-17T01:07:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:08.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-17T01:07:08.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:08.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:08.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:08.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1504744739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-17T01:07:08.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:08 smithi045 ceph-mon[98474]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:09 smithi045 ceph-mon[98474]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 -- 172.21.15.181:0/2818707205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 msgr2=0x7f7cb4105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 --2- 172.21.15.181:0/2818707205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb4105b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009910 tx=0x7f7ca8009c20 comp rx=0 tx=0).stop 2023-10-17T01:07:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 -- 172.21.15.181:0/2818707205 shutdown_connections 2023-10-17T01:07:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 --2- 172.21.15.181:0/2818707205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb4105b20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 -- 172.21.15.181:0/2818707205 >> 172.21.15.181:0/2818707205 conn(0x7f7cb4075250 msgr2=0x7f7cb4075660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 -- 172.21.15.181:0/2818707205 shutdown_connections 2023-10-17T01:07:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.893+0000 7f7cbb085700 1 -- 172.21.15.181:0/2818707205 wait complete. 2023-10-17T01:07:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.894+0000 7f7cbb085700 1 Processor -- start 2023-10-17T01:07:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.894+0000 7f7cbb085700 1 -- start start 2023-10-17T01:07:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.894+0000 7f7cbb085700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.894+0000 7f7cbb085700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb419af00 con 0x7f7cb4105740 2023-10-17T01:07:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.894+0000 7f7cb8e21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.894+0000 7f7cb8e21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.181:37954/0 (socket says 172.21.15.181:37954) 2023-10-17T01:07:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cb8e21700 1 -- 172.21.15.181:0/3833398522 learned_addr learned my addr 172.21.15.181:0/3833398522 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:07:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cb8e21700 1 -- 172.21.15.181:0/3833398522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80095f0 con 0x7f7cb4105740 2023-10-17T01:07:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cb8e21700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb419a9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8006010 tx=0x7f7ca8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ca801d070 con 0x7f7cb4105740 2023-10-17T01:07:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb419b100 con 0x7f7cb4105740 2023-10-17T01:07:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca80044a0 con 0x7f7cb4105740 2023-10-17T01:07:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.895+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ca8003a70 con 0x7f7cb4105740 2023-10-17T01:07:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.896+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb419b520 con 0x7f7cb4105740 2023-10-17T01:07:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.896+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb4062380 con 0x7f7cb4105740 2023-10-17T01:07:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.896+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f7ca8003cb0 con 0x7f7cb4105740 2023-10-17T01:07:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.897+0000 7f7cb1ffb700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7ca40385a0 0x7f7ca403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.897+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7ca804c3a0 con 0x7f7cb4105740 2023-10-17T01:07:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.897+0000 7f7cb3fff700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7ca40385a0 0x7f7ca403aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.898+0000 7f7cb3fff700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7ca40385a0 0x7f7ca403aa60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:09.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:09.903+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7ca8005c00 con 0x7f7cb4105740 2023-10-17T01:07:10.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.099+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cb4103600 con 0x7f7ca40385a0 2023-10-17T01:07:10.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.107+0000 7f7cb1ffb700 1 -- 172.21.15.181:0/3833398522 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7cb4103600 con 0x7f7ca40385a0 2023-10-17T01:07:10.108 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2023-10-17T01:07:10.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.108+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7ca40385a0 msgr2=0x7f7ca403aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:10.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7ca40385a0 0x7f7ca403aa60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0006e40 comp rx=0 tx=0).stop 2023-10-17T01:07:10.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 msgr2=0x7f7cb419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:10.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb419a9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8006010 tx=0x7f7ca8004ff0 comp rx=0 tx=0).stop 2023-10-17T01:07:10.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 shutdown_connections 2023-10-17T01:07:10.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7ca40385a0 0x7f7ca403aa60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:10.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 --2- 172.21.15.181:0/3833398522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb4105740 0x7f7cb419a9c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:10.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 >> 172.21.15.181:0/3833398522 conn(0x7f7cb4075250 msgr2=0x7f7cb40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:10.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 shutdown_connections 2023-10-17T01:07:10.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:10.109+0000 7f7cbb085700 1 -- 172.21.15.181:0/3833398522 wait complete. 2023-10-17T01:07:10.463 DEBUG:teuthology.orchestra.run.smithi060:mon.b> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.b.service 2023-10-17T01:07:10.466 DEBUG:teuthology.orchestra.run.smithi181:mon.c> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.c.service 2023-10-17T01:07:10.468 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-17T01:07:10.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-17T01:07:10.497 INFO:journalctl@ceph.mon.c.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:07:10.503 INFO:journalctl@ceph.mon.b.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:07:11.424 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:11.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:11.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c;count:3 2023-10-17T01:07:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:11.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:11.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:11.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: Deploying daemon mon.c on smithi181 2023-10-17T01:07:11.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:11 smithi045 ceph-mon[98474]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:13.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.080+0000 7f4c75588700 1 -- 172.21.15.181:0/3442626898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 msgr2=0x7f4c70101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:13.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.080+0000 7f4c75588700 1 --2- 172.21.15.181:0/3442626898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c70101e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009910 tx=0x7f4c60009c20 comp rx=0 tx=0).stop 2023-10-17T01:07:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.081+0000 7f4c75588700 1 -- 172.21.15.181:0/3442626898 shutdown_connections 2023-10-17T01:07:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.081+0000 7f4c75588700 1 --2- 172.21.15.181:0/3442626898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c70101e00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.081+0000 7f4c75588700 1 -- 172.21.15.181:0/3442626898 >> 172.21.15.181:0/3442626898 conn(0x7f4c700fb1d0 msgr2=0x7f4c700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.081+0000 7f4c75588700 1 -- 172.21.15.181:0/3442626898 shutdown_connections 2023-10-17T01:07:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.081+0000 7f4c75588700 1 -- 172.21.15.181:0/3442626898 wait complete. 2023-10-17T01:07:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.081+0000 7f4c75588700 1 Processor -- start 2023-10-17T01:07:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.082+0000 7f4c75588700 1 -- start start 2023-10-17T01:07:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.082+0000 7f4c75588700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c700ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.082+0000 7f4c75588700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70101600 con 0x7f4c70101a20 2023-10-17T01:07:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.082+0000 7f4c6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c700ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.082+0000 7f4c6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c700ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.181:37960/0 (socket says 172.21.15.181:37960) 2023-10-17T01:07:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.082+0000 7f4c6effd700 1 -- 172.21.15.181:0/416100349 learned_addr learned my addr 172.21.15.181:0/416100349 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:07:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c6effd700 1 -- 172.21.15.181:0/416100349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600095f0 con 0x7f4c70101a20 2023-10-17T01:07:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c6effd700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c700ff680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c60005850 tx=0x7f4c60004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c6001d070 con 0x7f4c70101a20 2023-10-17T01:07:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700ffbc0 con 0x7f4c70101a20 2023-10-17T01:07:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c60004370 con 0x7f4c70101a20 2023-10-17T01:07:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c60003ab0 con 0x7f4c70101a20 2023-10-17T01:07:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.083+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c70100040 con 0x7f4c70101a20 2023-10-17T01:07:13.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.084+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c70107f30 con 0x7f4c70101a20 2023-10-17T01:07:13.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.084+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f4c6000f460 con 0x7f4c70101a20 2023-10-17T01:07:13.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.085+0000 7f4c57fff700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4c580385a0 0x7f4c5803aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:13.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.085+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4c6004c480 con 0x7f4c70101a20 2023-10-17T01:07:13.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.085+0000 7f4c6e7fc700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4c580385a0 0x7f4c5803aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:13.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.085+0000 7f4c6e7fc700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4c580385a0 0x7f4c5803aa60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c64006fd0 tx=0x7f4c64006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:13.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.089+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f4c6000fd20 con 0x7f4c70101a20 2023-10-17T01:07:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.374+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4c70062380 con 0x7f4c70101a20 2023-10-17T01:07:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.375+0000 7f4c57fff700 1 -- 172.21.15.181:0/416100349 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f4c60026070 con 0x7f4c70101a20 2023-10-17T01:07:13.377 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:07:13.377 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","modified":"2023-10-17T01:05:28.454342Z","created":"2023-10-17T01:05:28.454342Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-17T01:07:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4c580385a0 msgr2=0x7f4c5803aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4c580385a0 0x7f4c5803aa60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c64006fd0 tx=0x7f4c64006e40 comp rx=0 tx=0).stop 2023-10-17T01:07:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 msgr2=0x7f4c700ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c700ff680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c60005850 tx=0x7f4c60004ff0 comp rx=0 tx=0).stop 2023-10-17T01:07:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 shutdown_connections 2023-10-17T01:07:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4c580385a0 0x7f4c5803aa60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 --2- 172.21.15.181:0/416100349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c70101a20 0x7f4c700ff680 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 >> 172.21.15.181:0/416100349 conn(0x7f4c700fb1d0 msgr2=0x7f4c70105070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 shutdown_connections 2023-10-17T01:07:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:13.377+0000 7f4c75588700 1 -- 172.21.15.181:0/416100349 wait complete. 2023-10-17T01:07:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2023-10-17T01:07:13.495 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:13 smithi181 ceph-mon[134520]: expand_channel_meta expanded map: {default=debug} 2023-10-17T01:07:13.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:13 smithi181 systemd[1]: Started Ceph mon.c for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:07:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:13 smithi045 ceph-mon[98474]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:13 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/416100349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:14.778 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-17T01:07:14.779 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-17T01:07:15.345 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:07:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:07:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: mon.a calling monitor election 2023-10-17T01:07:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: mon.c calling monitor election 2023-10-17T01:07:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:18.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-17T01:07:18.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:18.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: fsmap 2023-10-17T01:07:18.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:18.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: mgrmap e13: x(active, since 46s) 2023-10-17T01:07:18.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: overall HEALTH_OK 2023-10-17T01:07:18.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:18.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:18.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:18.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:18.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:07:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: mon.a calling monitor election 2023-10-17T01:07:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: mon.c calling monitor election 2023-10-17T01:07:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:18.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:18.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-17T01:07:18.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:18.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: fsmap 2023-10-17T01:07:18.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:18.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: mgrmap e13: x(active, since 46s) 2023-10-17T01:07:18.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: overall HEALTH_OK 2023-10-17T01:07:18.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:18.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:18.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:18.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:18.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.538+0000 7f965226e700 1 -- 172.21.15.181:0/1318457351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96340035c0 msgr2=0x7f9634005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.538+0000 7f965226e700 1 --2- 172.21.15.181:0/1318457351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96340035c0 0x7f9634005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f964c076740 tx=0x7f963c00feb0 comp rx=0 tx=0).stop 2023-10-17T01:07:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.538+0000 7f965226e700 1 -- 172.21.15.181:0/1318457351 shutdown_connections 2023-10-17T01:07:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.538+0000 7f965226e700 1 --2- 172.21.15.181:0/1318457351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96340035c0 0x7f9634005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.538+0000 7f965226e700 1 -- 172.21.15.181:0/1318457351 >> 172.21.15.181:0/1318457351 conn(0x7f964c075300 msgr2=0x7f964c075710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.539+0000 7f965226e700 1 -- 172.21.15.181:0/1318457351 shutdown_connections 2023-10-17T01:07:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.539+0000 7f965226e700 1 -- 172.21.15.181:0/1318457351 wait complete. 2023-10-17T01:07:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.539+0000 7f965226e700 1 Processor -- start 2023-10-17T01:07:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.539+0000 7f965226e700 1 -- start start 2023-10-17T01:07:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f965226e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f965226e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c0ffec0 0x7f964c100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f965226e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c101900 con 0x7f964c0ffec0 2023-10-17T01:07:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f965226e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1a2020 con 0x7f96340035c0 2023-10-17T01:07:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45052/0 (socket says 172.21.15.181:45052) 2023-10-17T01:07:19.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 -- 172.21.15.181:0/2081197446 learned_addr learned my addr 172.21.15.181:0/2081197446 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:07:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 -- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 msgr2=0x7f964c0ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-17T01:07:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 -- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 msgr2=0x7f964c0ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-17T01:07:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-17T01:07:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.540+0000 7f964bfff700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:07:19.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.541+0000 7f964b7fe700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c0ffec0 0x7f964c100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:19.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.541+0000 7f964b7fe700 1 -- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 msgr2=0x7f964c0ff980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:07:19.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.541+0000 7f964b7fe700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:19.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.541+0000 7f964b7fe700 1 -- 172.21.15.181:0/2081197446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c00f900 con 0x7f964c0ffec0 2023-10-17T01:07:19.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.541+0000 7f964b7fe700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c0ffec0 0x7f964c100340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f964000b4d0 tx=0x7f964000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:19.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.544+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f9640011630 con 0x7f964c0ffec0 2023-10-17T01:07:19.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.544+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9640011c50 con 0x7f964c0ffec0 2023-10-17T01:07:19.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.544+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c1a2220 con 0x7f964c0ffec0 2023-10-17T01:07:19.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.544+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f9640010d90 con 0x7f964c0ffec0 2023-10-17T01:07:19.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.544+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c1a2740 con 0x7f964c0ffec0 2023-10-17T01:07:19.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.545+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f964001a400 con 0x7f964c0ffec0 2023-10-17T01:07:19.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.546+0000 7f96497fa700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f96380383d0 0x7f963803a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:19.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.546+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f964004bb50 con 0x7f964c0ffec0 2023-10-17T01:07:19.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.546+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c062380 con 0x7f964c0ffec0 2023-10-17T01:07:19.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.546+0000 7f964bfff700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f96380383d0 0x7f963803a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:19.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.547+0000 7f964bfff700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f96380383d0 0x7f963803a890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f963c011510 tx=0x7f963c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:19.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.551+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f96400178b0 con 0x7f964c0ffec0 2023-10-17T01:07:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:19 smithi045 ceph-mon[98474]: Deploying daemon mon.b on smithi060 2023-10-17T01:07:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:19 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:19.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.822+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f964c1a2ae0 con 0x7f964c0ffec0 2023-10-17T01:07:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.822+0000 7f96497fa700 1 -- 172.21.15.181:0/2081197446 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f96400172c0 con 0x7f964c0ffec0 2023-10-17T01:07:19.838 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:07:19.838 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","modified":"2023-10-17T01:07:13.454019Z","created":"2023-10-17T01:05:28.454342Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-17T01:07:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.824+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f96380383d0 msgr2=0x7f963803a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.824+0000 7f965226e700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f96380383d0 0x7f963803a890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f963c011510 tx=0x7f963c005960 comp rx=0 tx=0).stop 2023-10-17T01:07:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.824+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c0ffec0 msgr2=0x7f964c100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.824+0000 7f965226e700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c0ffec0 0x7f964c100340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f964000b4d0 tx=0x7f964000b890 comp rx=0 tx=0).stop 2023-10-17T01:07:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 shutdown_connections 2023-10-17T01:07:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f96380383d0 0x7f963803a890 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96340035c0 0x7f964c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 --2- 172.21.15.181:0/2081197446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c0ffec0 0x7f964c100340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 >> 172.21.15.181:0/2081197446 conn(0x7f964c075300 msgr2=0x7f964c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 shutdown_connections 2023-10-17T01:07:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:19.825+0000 7f965226e700 1 -- 172.21.15.181:0/2081197446 wait complete. 2023-10-17T01:07:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2023-10-17T01:07:19.852 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:19 smithi181 ceph-mon[134520]: Deploying daemon mon.b on smithi060 2023-10-17T01:07:19.852 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:19 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:20.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:20 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/2081197446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:20.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:20 smithi045 ceph-mon[98474]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:20.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:20 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/2081197446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:20.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:20 smithi181 ceph-mon[134520]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.209 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-17T01:07:21.210 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-17T01:07:21.536 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 systemd[1]: Starting Ceph mon.b for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:07:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:07:21.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 podman[134096]: 2023-10-17T01:07:21.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 bash[134096]: 86826790a891e6ca9beb4d93c14352cf708ea7f5dc550d3ceefdad823085b795 2023-10-17T01:07:21.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: set uid:gid to 167:167 (ceph:ceph) 2023-10-17T01:07:21.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: ceph version 18.2.0-668-gab9fe435 (ab9fe43522bc41de59746200b95f65d4fbcccffe) reef (stable), process ceph-mon, pid 7 2023-10-17T01:07:21.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pidfile_write: ignore empty --pid-file 2023-10-17T01:07:21.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: load: jerasure load: lrc 2023-10-17T01:07:21.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: RocksDB version: 7.9.2 2023-10-17T01:07:21.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Git sha 0 2023-10-17T01:07:21.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Compile date 2023-10-11 20:19:14 2023-10-17T01:07:21.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: DB SUMMARY 2023-10-17T01:07:21.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: DB Session ID: 6XMV8D5Z4L8DFK6TQ5V4 2023-10-17T01:07:21.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: CURRENT file: CURRENT 2023-10-17T01:07:21.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: IDENTITY file: IDENTITY 2023-10-17T01:07:21.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-17T01:07:21.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-17T01:07:21.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-17T01:07:21.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.error_if_exists: 0 2023-10-17T01:07:21.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.create_if_missing: 0 2023-10-17T01:07:21.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.paranoid_checks: 1 2023-10-17T01:07:21.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-17T01:07:21.834 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-17T01:07:21.834 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-17T01:07:21.834 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.env: 0x556662453720 2023-10-17T01:07:21.834 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.fs: PosixFileSystem 2023-10-17T01:07:21.834 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.info_log: 0x556664b89420 2023-10-17T01:07:21.835 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_file_opening_threads: 16 2023-10-17T01:07:21.835 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.statistics: (nil) 2023-10-17T01:07:21.835 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.use_fsync: 0 2023-10-17T01:07:21.835 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_log_file_size: 0 2023-10-17T01:07:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-17T01:07:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-17T01:07:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.keep_log_file_num: 1000 2023-10-17T01:07:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.recycle_log_file_num: 0 2023-10-17T01:07:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.allow_fallocate: 1 2023-10-17T01:07:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.allow_mmap_reads: 0 2023-10-17T01:07:21.837 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.allow_mmap_writes: 0 2023-10-17T01:07:21.837 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.use_direct_reads: 0 2023-10-17T01:07:21.837 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-17T01:07:21.837 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.create_missing_column_families: 0 2023-10-17T01:07:21.837 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.db_log_dir: 2023-10-17T01:07:21.838 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.wal_dir: 2023-10-17T01:07:21.838 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-17T01:07:21.838 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-17T01:07:21.839 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-17T01:07:21.841 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-17T01:07:21.841 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-17T01:07:21.842 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-17T01:07:21.842 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.advise_random_on_open: 1 2023-10-17T01:07:21.842 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.db_write_buffer_size: 0 2023-10-17T01:07:21.843 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.write_buffer_manager: 0x556663e1a5a0 2023-10-17T01:07:21.843 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-17T01:07:21.843 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-17T01:07:21.843 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-17T01:07:21.844 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.rate_limiter: (nil) 2023-10-17T01:07:21.844 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-17T01:07:21.844 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.wal_recovery_mode: 2 2023-10-17T01:07:21.844 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.enable_thread_tracking: 0 2023-10-17T01:07:21.844 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.enable_pipelined_write: 0 2023-10-17T01:07:21.845 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.unordered_write: 0 2023-10-17T01:07:21.845 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-17T01:07:21.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-17T01:07:21.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-17T01:07:21.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-17T01:07:21.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.row_cache: None 2023-10-17T01:07:21.847 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.wal_filter: None 2023-10-17T01:07:21.847 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-17T01:07:21.847 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.allow_ingest_behind: 0 2023-10-17T01:07:21.847 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.two_write_queues: 0 2023-10-17T01:07:21.847 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.manual_wal_flush: 0 2023-10-17T01:07:21.848 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.wal_compression: 0 2023-10-17T01:07:21.848 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.atomic_flush: 0 2023-10-17T01:07:21.848 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-17T01:07:21.848 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-17T01:07:21.848 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-17T01:07:21.848 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.log_readahead_size: 0 2023-10-17T01:07:21.849 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-17T01:07:21.849 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.best_efforts_recovery: 0 2023-10-17T01:07:21.849 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-17T01:07:21.849 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-17T01:07:21.849 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.allow_data_in_errors: 0 2023-10-17T01:07:21.850 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.db_host_id: __hostname__ 2023-10-17T01:07:21.850 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-17T01:07:21.850 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_background_jobs: 2 2023-10-17T01:07:21.850 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_background_compactions: -1 2023-10-17T01:07:21.850 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_subcompactions: 1 2023-10-17T01:07:21.851 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-17T01:07:21.851 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-17T01:07:21.851 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-17T01:07:21.851 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_total_wal_size: 0 2023-10-17T01:07:21.851 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-17T01:07:21.852 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-17T01:07:21.852 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-17T01:07:21.852 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-17T01:07:21.852 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_open_files: -1 2023-10-17T01:07:21.852 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bytes_per_sync: 0 2023-10-17T01:07:21.853 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-17T01:07:21.853 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-17T01:07:21.853 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_readahead_size: 0 2023-10-17T01:07:21.854 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_background_flushes: -1 2023-10-17T01:07:21.854 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Compression algorithms supported: 2023-10-17T01:07:21.854 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-17T01:07:21.854 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kZSTD supported: 0 2023-10-17T01:07:21.854 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kXpressCompression supported: 0 2023-10-17T01:07:21.855 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kLZ4HCCompression supported: 1 2023-10-17T01:07:21.855 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kZlibCompression supported: 1 2023-10-17T01:07:21.855 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kSnappyCompression supported: 1 2023-10-17T01:07:21.855 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kLZ4Compression supported: 1 2023-10-17T01:07:21.856 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: kBZip2Compression supported: 0 2023-10-17T01:07:21.856 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-17T01:07:21.856 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-17T01:07:21.857 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-17T01:07:21.857 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-17T01:07:21.857 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-17T01:07:21.858 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.merge_operator: 2023-10-17T01:07:21.858 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_filter: None 2023-10-17T01:07:21.859 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_filter_factory: None 2023-10-17T01:07:21.859 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.sst_partitioner_factory: None 2023-10-17T01:07:21.859 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-17T01:07:21.860 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-17T01:07:21.860 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556664b89520) 2023-10-17T01:07:21.860 INFO:journalctl@ceph.mon.b.smithi060.stdout: cache_index_and_filter_blocks: 1 2023-10-17T01:07:21.861 INFO:journalctl@ceph.mon.b.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-17T01:07:21.861 INFO:journalctl@ceph.mon.b.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-17T01:07:21.861 INFO:journalctl@ceph.mon.b.smithi060.stdout: pin_top_level_index_and_filter: 1 2023-10-17T01:07:21.862 INFO:journalctl@ceph.mon.b.smithi060.stdout: index_type: 0 2023-10-17T01:07:21.862 INFO:journalctl@ceph.mon.b.smithi060.stdout: data_block_index_type: 0 2023-10-17T01:07:21.862 INFO:journalctl@ceph.mon.b.smithi060.stdout: index_shortening: 1 2023-10-17T01:07:21.863 INFO:journalctl@ceph.mon.b.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-17T01:07:21.863 INFO:journalctl@ceph.mon.b.smithi060.stdout: checksum: 4 2023-10-17T01:07:21.863 INFO:journalctl@ceph.mon.b.smithi060.stdout: no_block_cache: 0 2023-10-17T01:07:21.864 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_cache: 0x556663e9b350 2023-10-17T01:07:21.864 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_cache_name: BinnedLRUCache 2023-10-17T01:07:21.865 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_cache_options: 2023-10-17T01:07:21.865 INFO:journalctl@ceph.mon.b.smithi060.stdout: capacity : 536870912 2023-10-17T01:07:21.866 INFO:journalctl@ceph.mon.b.smithi060.stdout: num_shard_bits : 4 2023-10-17T01:07:21.866 INFO:journalctl@ceph.mon.b.smithi060.stdout: strict_capacity_limit : 0 2023-10-17T01:07:21.866 INFO:journalctl@ceph.mon.b.smithi060.stdout: high_pri_pool_ratio: 0.000 2023-10-17T01:07:21.866 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_cache_compressed: (nil) 2023-10-17T01:07:21.866 INFO:journalctl@ceph.mon.b.smithi060.stdout: persistent_cache: (nil) 2023-10-17T01:07:21.867 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_size: 4096 2023-10-17T01:07:21.867 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_size_deviation: 10 2023-10-17T01:07:21.867 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_restart_interval: 16 2023-10-17T01:07:21.867 INFO:journalctl@ceph.mon.b.smithi060.stdout: index_block_restart_interval: 1 2023-10-17T01:07:21.867 INFO:journalctl@ceph.mon.b.smithi060.stdout: metadata_block_size: 4096 2023-10-17T01:07:21.868 INFO:journalctl@ceph.mon.b.smithi060.stdout: partition_filters: 0 2023-10-17T01:07:21.868 INFO:journalctl@ceph.mon.b.smithi060.stdout: use_delta_encoding: 1 2023-10-17T01:07:21.868 INFO:journalctl@ceph.mon.b.smithi060.stdout: filter_policy: bloomfilter 2023-10-17T01:07:21.868 INFO:journalctl@ceph.mon.b.smithi060.stdout: whole_key_filtering: 1 2023-10-17T01:07:21.868 INFO:journalctl@ceph.mon.b.smithi060.stdout: verify_compression: 0 2023-10-17T01:07:21.869 INFO:journalctl@ceph.mon.b.smithi060.stdout: read_amp_bytes_per_bit: 0 2023-10-17T01:07:21.869 INFO:journalctl@ceph.mon.b.smithi060.stdout: format_version: 5 2023-10-17T01:07:21.869 INFO:journalctl@ceph.mon.b.smithi060.stdout: enable_index_compression: 1 2023-10-17T01:07:21.869 INFO:journalctl@ceph.mon.b.smithi060.stdout: block_align: 0 2023-10-17T01:07:21.869 INFO:journalctl@ceph.mon.b.smithi060.stdout: max_auto_readahead_size: 262144 2023-10-17T01:07:21.869 INFO:journalctl@ceph.mon.b.smithi060.stdout: prepopulate_block_cache: 0 2023-10-17T01:07:21.870 INFO:journalctl@ceph.mon.b.smithi060.stdout: initial_auto_readahead_size: 8192 2023-10-17T01:07:21.870 INFO:journalctl@ceph.mon.b.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2023-10-17T01:07:21.870 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.write_buffer_size: 33554432 2023-10-17T01:07:21.870 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_write_buffer_number: 2 2023-10-17T01:07:21.870 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression: NoCompression 2023-10-17T01:07:21.871 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression: Disabled 2023-10-17T01:07:21.871 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.prefix_extractor: nullptr 2023-10-17T01:07:21.871 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-17T01:07:21.871 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.num_levels: 7 2023-10-17T01:07:21.871 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-17T01:07:21.872 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-17T01:07:21.872 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-17T01:07:21.872 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-17T01:07:21.872 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-17T01:07:21.872 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-17T01:07:21.874 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-17T01:07:21.874 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-17T01:07:21.875 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-17T01:07:21.875 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-17T01:07:21.875 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-17T01:07:21.876 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-17T01:07:21.876 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-17T01:07:21.876 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.level: 32767 2023-10-17T01:07:21.877 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.strategy: 0 2023-10-17T01:07:21.877 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-17T01:07:21.877 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-17T01:07:21.878 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-17T01:07:21.878 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-17T01:07:21.878 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.enabled: false 2023-10-17T01:07:21.879 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-17T01:07:21.879 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-17T01:07:21.879 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-17T01:07:21.880 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-17T01:07:21.880 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.target_file_size_base: 67108864 2023-10-17T01:07:21.880 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-17T01:07:21.880 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-17T01:07:21.881 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-17T01:07:21.881 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-17T01:07:21.881 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-17T01:07:21.882 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-17T01:07:21.882 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-17T01:07:21.882 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-17T01:07:21.883 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-17T01:07:21.883 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-17T01:07:21.883 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-17T01:07:21.883 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-17T01:07:21.884 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-17T01:07:21.884 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-17T01:07:21.884 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.arena_block_size: 1048576 2023-10-17T01:07:21.884 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-17T01:07:21.884 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-17T01:07:21.884 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.disable_auto_compactions: 0 2023-10-17T01:07:21.885 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-17T01:07:21.885 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-17T01:07:21.885 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-17T01:07:21.886 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-17T01:07:21.886 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-17T01:07:21.887 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-17T01:07:21.887 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-17T01:07:21.887 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-17T01:07:21.888 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-17T01:07:21.888 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-17T01:07:21.888 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.table_properties_collectors: 2023-10-17T01:07:21.889 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.inplace_update_support: 0 2023-10-17T01:07:21.889 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-17T01:07:21.889 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-17T01:07:21.890 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-17T01:07:21.890 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-17T01:07:21.890 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.bloom_locality: 0 2023-10-17T01:07:21.891 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.max_successive_merges: 0 2023-10-17T01:07:21.891 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-17T01:07:21.891 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.paranoid_file_checks: 0 2023-10-17T01:07:21.892 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.force_consistency_checks: 1 2023-10-17T01:07:21.892 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.report_bg_io_stats: 0 2023-10-17T01:07:21.892 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.ttl: 2592000 2023-10-17T01:07:21.893 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-17T01:07:21.893 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-17T01:07:21.893 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-17T01:07:21.894 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.enable_blob_files: false 2023-10-17T01:07:21.894 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.min_blob_size: 0 2023-10-17T01:07:21.895 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.blob_file_size: 268435456 2023-10-17T01:07:21.895 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-17T01:07:21.895 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-17T01:07:21.896 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-17T01:07:21.896 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-17T01:07:21.897 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-17T01:07:21.897 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.blob_file_starting_level: 0 2023-10-17T01:07:21.897 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-17T01:07:21.898 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-17T01:07:21.898 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-17T01:07:21.898 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 00341adb-8bbd-4744-adbc-8076a4c17c3c 2023-10-17T01:07:21.899 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697504841709926, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-17T01:07:21.899 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-17T01:07:21.899 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697504841710527, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697504841, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "00341adb-8bbd-4744-adbc-8076a4c17c3c", "db_session_id": "6XMV8D5Z4L8DFK6TQ5V4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-17T01:07:21.900 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697504841710626, "job": 1, "event": "recovery_finished"} 2023-10-17T01:07:21.900 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-17T01:07:21.900 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-17T01:07:21.900 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556663f38000 2023-10-17T01:07:21.900 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: DB pointer 0x556663f24000 2023-10-17T01:07:21.901 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-17T01:07:21.901 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-17T01:07:21.901 INFO:journalctl@ceph.mon.b.smithi060.stdout: ** DB Stats ** 2023-10-17T01:07:21.901 INFO:journalctl@ceph.mon.b.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-17T01:07:21.901 INFO:journalctl@ceph.mon.b.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-17T01:07:21.901 INFO:journalctl@ceph.mon.b.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-17T01:07:21.902 INFO:journalctl@ceph.mon.b.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-17T01:07:21.902 INFO:journalctl@ceph.mon.b.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-17T01:07:21.902 INFO:journalctl@ceph.mon.b.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-17T01:07:21.902 INFO:journalctl@ceph.mon.b.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-17T01:07:21.902 INFO:journalctl@ceph.mon.b.smithi060.stdout: 2023-10-17T01:07:21.903 INFO:journalctl@ceph.mon.b.smithi060.stdout: ** Compaction Stats [default] ** 2023-10-17T01:07:21.903 INFO:journalctl@ceph.mon.b.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-17T01:07:21.903 INFO:journalctl@ceph.mon.b.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-17T01:07:21.903 INFO:journalctl@ceph.mon.b.smithi060.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:07:21.903 INFO:journalctl@ceph.mon.b.smithi060.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:07:21.904 INFO:journalctl@ceph.mon.b.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:07:21.904 INFO:journalctl@ceph.mon.b.smithi060.stdout: 2023-10-17T01:07:21.904 INFO:journalctl@ceph.mon.b.smithi060.stdout: ** Compaction Stats [default] ** 2023-10-17T01:07:21.904 INFO:journalctl@ceph.mon.b.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-17T01:07:21.904 INFO:journalctl@ceph.mon.b.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-17T01:07:21.905 INFO:journalctl@ceph.mon.b.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-17T01:07:21.905 INFO:journalctl@ceph.mon.b.smithi060.stdout: 2023-10-17T01:07:21.905 INFO:journalctl@ceph.mon.b.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-17T01:07:21.905 INFO:journalctl@ceph.mon.b.smithi060.stdout: 2023-10-17T01:07:21.905 INFO:journalctl@ceph.mon.b.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-17T01:07:21.906 INFO:journalctl@ceph.mon.b.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-17T01:07:21.906 INFO:journalctl@ceph.mon.b.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-17T01:07:21.906 INFO:journalctl@ceph.mon.b.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-17T01:07:21.906 INFO:journalctl@ceph.mon.b.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-17T01:07:21.906 INFO:journalctl@ceph.mon.b.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-17T01:07:21.907 INFO:journalctl@ceph.mon.b.smithi060.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-17T01:07:21.907 INFO:journalctl@ceph.mon.b.smithi060.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-17T01:07:21.907 INFO:journalctl@ceph.mon.b.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-17T01:07:21.907 INFO:journalctl@ceph.mon.b.smithi060.stdout: Block cache BinnedLRUCache@0x556663e9b350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-17T01:07:21.907 INFO:journalctl@ceph.mon.b.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-17T01:07:21.908 INFO:journalctl@ceph.mon.b.smithi060.stdout: 2023-10-17T01:07:21.908 INFO:journalctl@ceph.mon.b.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-17T01:07:21.908 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-17T01:07:21.908 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: using public_addr v2:172.21.15.60:0/0 -> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] 2023-10-17T01:07:21.908 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: starting mon.b rank -1 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:07:21.909 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(???) e0 preinit fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:07:21.909 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 systemd[1]: Started Ceph mon.b for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:07:21.909 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).mds e1 new map 2023-10-17T01:07:21.909 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-17T01:07:21.909 INFO:journalctl@ceph.mon.b.smithi060.stdout: e1 2023-10-17T01:07:21.909 INFO:journalctl@ceph.mon.b.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-17T01:07:21.910 INFO:journalctl@ceph.mon.b.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-17T01:07:21.910 INFO:journalctl@ceph.mon.b.smithi060.stdout: legacy client fscid: -1 2023-10-17T01:07:21.910 INFO:journalctl@ceph.mon.b.smithi060.stdout: 2023-10-17T01:07:21.910 INFO:journalctl@ceph.mon.b.smithi060.stdout: No filesystems configured 2023-10-17T01:07:21.910 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-17T01:07:21.911 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-17T01:07:21.911 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-17T01:07:21.911 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-17T01:07:21.911 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-17T01:07:21.911 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-17T01:07:21.911 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-17T01:07:21.912 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-17T01:07:21.912 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-17T01:07:21.912 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-17T01:07:21.912 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Added host smithi060 2023-10-17T01:07:21.912 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.913 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.913 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.913 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.913 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:07:21.913 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.913 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.914 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.914 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.914 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.914 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.914 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:07:21.915 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:21.915 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:21.915 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi060:/etc/ceph/ceph.conf 2023-10-17T01:07:21.915 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:21.915 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:21.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:21.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-17T01:07:21.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Deploying cephadm binary to smithi181 2023-10-17T01:07:21.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Added host smithi181 2023-10-17T01:07:21.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:21.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.918 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.918 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.918 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.918 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:07:21.919 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.919 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.919 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.919 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.919 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.919 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.920 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:07:21.920 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:21.920 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:21.920 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1504744739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-17T01:07:21.920 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi181:/etc/ceph/ceph.conf 2023-10-17T01:07:21.921 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:21.921 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:21.921 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-17T01:07:21.921 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.921 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.922 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.922 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1504744739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-17T01:07:21.922 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:21.922 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.922 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:21.923 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c;count:3 2023-10-17T01:07:21.923 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.923 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:21.923 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:21.923 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:21.924 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.924 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:21.924 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:21.924 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Deploying daemon mon.c on smithi181 2023-10-17T01:07:21.925 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.925 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.925 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/416100349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:21.925 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:07:21.925 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.926 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.a calling monitor election 2023-10-17T01:07:21.926 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.926 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.926 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.926 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.c calling monitor election 2023-10-17T01:07:21.927 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.927 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.927 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.927 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.927 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.927 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-17T01:07:21.928 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: monmap e2: 2 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:21.928 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: fsmap 2023-10-17T01:07:21.928 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:21.928 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mgrmap e13: x(active, since 46s) 2023-10-17T01:07:21.928 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: overall HEALTH_OK 2023-10-17T01:07:21.929 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.929 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.929 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:21.929 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:21.929 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:21.930 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: Deploying daemon mon.b on smithi060 2023-10-17T01:07:21.930 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:21.930 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/2081197446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:21.930 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:21.930 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-17T01:07:21.930 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:21 smithi060 ceph-mon[134136]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-17T01:07:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: mon.c calling monitor election 2023-10-17T01:07:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:07:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: mon.a calling monitor election 2023-10-17T01:07:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:27.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:27.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:27.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:27.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-17T01:07:27.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:27.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: fsmap 2023-10-17T01:07:27.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:27.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: mgrmap e13: x(active, since 54s) 2023-10-17T01:07:27.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-17T01:07:27.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-17T01:07:27.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-17T01:07:27.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: mon.b (rank 2) addr [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] is down (out of quorum) 2023-10-17T01:07:27.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:27.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:27.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: mon.c calling monitor election 2023-10-17T01:07:27.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:07:27.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: mon.a calling monitor election 2023-10-17T01:07:27.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:27.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:27.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:27.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-17T01:07:27.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: fsmap 2023-10-17T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: mgrmap e13: x(active, since 54s) 2023-10-17T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-17T01:07:27.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-17T01:07:27.306 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-17T01:07:27.306 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: mon.b (rank 2) addr [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] is down (out of quorum) 2023-10-17T01:07:27.306 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:27.306 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.463+0000 7fbafd8a1700 1 -- 172.21.15.181:0/1854875186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 msgr2=0x7fbae0007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.463+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/1854875186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbae0007820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005fa0 tx=0x7fbaec00ff70 comp rx=0 tx=0).stop 2023-10-17T01:07:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.463+0000 7fbafd8a1700 1 -- 172.21.15.181:0/1854875186 shutdown_connections 2023-10-17T01:07:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.463+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/1854875186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbae0007820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.463+0000 7fbafd8a1700 1 -- 172.21.15.181:0/1854875186 >> 172.21.15.181:0/1854875186 conn(0x7fbaf80fb350 msgr2=0x7fbaf80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.464+0000 7fbafd8a1700 1 -- 172.21.15.181:0/1854875186 shutdown_connections 2023-10-17T01:07:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.464+0000 7fbafd8a1700 1 -- 172.21.15.181:0/1854875186 wait complete. 2023-10-17T01:07:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.464+0000 7fbafd8a1700 1 Processor -- start 2023-10-17T01:07:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.464+0000 7fbafd8a1700 1 -- start start 2023-10-17T01:07:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbafd8a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbaf8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbafd8a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbafd8a1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 0x7fbaf819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbafd8a1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8198cb0 con 0x7fbaf8105820 2023-10-17T01:07:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbafd8a1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81a4ce0 con 0x7fbaf819cf70 2023-10-17T01:07:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbafd8a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81a4e20 con 0x7fbaf81985c0 2023-10-17T01:07:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45080/0 (socket says 172.21.15.181:45080) 2023-10-17T01:07:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 -- 172.21.15.181:0/2180705776 learned_addr learned my addr 172.21.15.181:0/2180705776 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:07:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 msgr2=0x7fbaf819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-10-17T01:07:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 msgr2=0x7fbaf819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-17T01:07:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-17T01:07:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf67fc700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:07:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf6ffd700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbaf8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf77fe700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 0x7fbaf819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf77fe700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 msgr2=0x7fbaf819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-10-17T01:07:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf77fe700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 msgr2=0x7fbaf819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-17T01:07:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf77fe700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 0x7fbaf819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-17T01:07:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf77fe700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 0x7fbaf819d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-17T01:07:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf6ffd700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 msgr2=0x7fbaf819ca30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:07:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf6ffd700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf6ffd700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 msgr2=0x7fbaf819d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:07:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf6ffd700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 0x7fbaf819d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.465+0000 7fbaf6ffd700 1 -- 172.21.15.181:0/2180705776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec00f970 con 0x7fbaf8105820 2023-10-17T01:07:27.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.466+0000 7fbaf6ffd700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbaf8198080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005fa0 tx=0x7fbaec0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.466+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec012e40 con 0x7fbaf8105820 2023-10-17T01:07:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.466+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf81a50a0 con 0x7fbaf8105820 2023-10-17T01:07:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.466+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaec0042b0 con 0x7fbaf8105820 2023-10-17T01:07:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.466+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec01d6a0 con 0x7fbaf8105820 2023-10-17T01:07:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.466+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf81a55f0 con 0x7fbaf8105820 2023-10-17T01:07:27.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.467+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7fbaec024050 con 0x7fbaf8105820 2023-10-17T01:07:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.467+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8062380 con 0x7fbaf8105820 2023-10-17T01:07:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.467+0000 7fbafc89f700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbae4038a70 0x7fbae403af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.467+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbaec04df30 con 0x7fbaf8105820 2023-10-17T01:07:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.467+0000 7fbaf67fc700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbae4038a70 0x7fbae403af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.468+0000 7fbaf67fc700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbae4038a70 0x7fbae403af30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009180 tx=0x7fbae8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.473+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fbaec038530 con 0x7fbaf8105820 2023-10-17T01:07:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.748+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbaf81066c0 con 0x7fbaf8105820 2023-10-17T01:07:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.749+0000 7fbafc89f700 1 -- 172.21.15.181:0/2180705776 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7fbaec02b020 con 0x7fbaf8105820 2023-10-17T01:07:27.751 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-17T01:07:27.751 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","modified":"2023-10-17T01:07:21.738565Z","created":"2023-10-17T01:05:28.454342Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-17T01:07:27.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbae4038a70 msgr2=0x7fbae403af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbae4038a70 0x7fbae403af30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009180 tx=0x7fbae8008040 comp rx=0 tx=0).stop 2023-10-17T01:07:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 msgr2=0x7fbaf8198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbaf8198080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005fa0 tx=0x7fbaec0057d0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 shutdown_connections 2023-10-17T01:07:27.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbae4038a70 0x7fbae403af30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf819cf70 0x7fbaf819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf8105820 0x7fbaf8198080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 --2- 172.21.15.181:0/2180705776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf81985c0 0x7fbaf819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:27.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 >> 172.21.15.181:0/2180705776 conn(0x7fbaf80fb350 msgr2=0x7fbaf8101320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:27.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 shutdown_connections 2023-10-17T01:07:27.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:27.751+0000 7fbafd8a1700 1 -- 172.21.15.181:0/2180705776 wait complete. 2023-10-17T01:07:27.755 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 3 2023-10-17T01:07:28.136 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:28.136 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:28.136 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:28.136 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:28.137 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:28.137 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:27 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/2180705776' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:28.138 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-17T01:07:28.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-17T01:07:28.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:28.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:28.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:28.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:28.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:28.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:27 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/2180705776' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:28.903 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:07:29.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: mon.c calling monitor election 2023-10-17T01:07:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:07:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: mon.a calling monitor election 2023-10-17T01:07:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:07:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-17T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: fsmap 2023-10-17T01:07:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: mgrmap e13: x(active, since 54s) 2023-10-17T01:07:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-17T01:07:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-17T01:07:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-17T01:07:29.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: mon.b (rank 2) addr [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] is down (out of quorum) 2023-10-17T01:07:29.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.084 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:29.084 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.084 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:29.084 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:28 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/2180705776' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-17T01:07:29.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: mon.b calling monitor election 2023-10-17T01:07:29.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: mon.b calling monitor election 2023-10-17T01:07:29.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: mon.c calling monitor election 2023-10-17T01:07:29.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: mon.a calling monitor election 2023-10-17T01:07:29.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-17T01:07:29.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:29.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: fsmap 2023-10-17T01:07:29.540 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:29.540 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: mgrmap e13: x(active, since 56s) 2023-10-17T01:07:29.540 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-17T01:07:29.540 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: Cluster is now healthy 2023-10-17T01:07:29.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: overall HEALTH_OK 2023-10-17T01:07:29.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: mon.b calling monitor election 2023-10-17T01:07:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: mon.b calling monitor election 2023-10-17T01:07:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: mon.c calling monitor election 2023-10-17T01:07:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: mon.a calling monitor election 2023-10-17T01:07:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-17T01:07:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: fsmap 2023-10-17T01:07:29.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:29.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: mgrmap e13: x(active, since 56s) 2023-10-17T01:07:29.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-17T01:07:29.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: Cluster is now healthy 2023-10-17T01:07:29.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: overall HEALTH_OK 2023-10-17T01:07:29.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: mon.b calling monitor election 2023-10-17T01:07:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: mon.b calling monitor election 2023-10-17T01:07:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: mon.c calling monitor election 2023-10-17T01:07:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: mon.a calling monitor election 2023-10-17T01:07:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-17T01:07:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: monmap e3: 3 mons at {a=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],b=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} 2023-10-17T01:07:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: fsmap 2023-10-17T01:07:29.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: osdmap e4: 0 total, 0 up, 0 in 2023-10-17T01:07:29.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: mgrmap e13: x(active, since 56s) 2023-10-17T01:07:29.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-17T01:07:29.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: Cluster is now healthy 2023-10-17T01:07:29.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: overall HEALTH_OK 2023-10-17T01:07:29.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:29.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.431+0000 7f5b5949b700 1 -- 172.21.15.45:0/1685521331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 msgr2=0x7f5b54106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:30.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.431+0000 7f5b5949b700 1 --2- 172.21.15.45:0/1685521331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54106bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009910 tx=0x7f5b44009c20 comp rx=0 tx=0).stop 2023-10-17T01:07:30.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.432+0000 7f5b5949b700 1 -- 172.21.15.45:0/1685521331 shutdown_connections 2023-10-17T01:07:30.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.432+0000 7f5b5949b700 1 --2- 172.21.15.45:0/1685521331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54106bc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.432+0000 7f5b5949b700 1 -- 172.21.15.45:0/1685521331 >> 172.21.15.45:0/1685521331 conn(0x7f5b540fc310 msgr2=0x7f5b540fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:30.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.432+0000 7f5b5949b700 1 -- 172.21.15.45:0/1685521331 shutdown_connections 2023-10-17T01:07:30.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.432+0000 7f5b5949b700 1 -- 172.21.15.45:0/1685521331 wait complete. 2023-10-17T01:07:30.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 Processor -- start 2023-10-17T01:07:30.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 -- start start 2023-10-17T01:07:30.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b54198570 0x7f5b5419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b5419cf20 0x7f5b5419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54198c60 con 0x7f5b541067e0 2023-10-17T01:07:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a4c90 con 0x7f5b54198570 2023-10-17T01:07:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.433+0000 7f5b5949b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a4dd0 con 0x7f5b5419cf20 2023-10-17T01:07:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34766/0 (socket says 172.21.15.45:34766) 2023-10-17T01:07:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 -- 172.21.15.45:0/3910144726 learned_addr learned my addr 172.21.15.45:0/3910144726 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:07:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b537fe700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b5419cf20 0x7f5b5419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b527fc700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b54198570 0x7f5b5419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:30.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 -- 172.21.15.45:0/3910144726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b5419cf20 msgr2=0x7f5b5419d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:30.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b5419cf20 0x7f5b5419d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 -- 172.21.15.45:0/3910144726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b54198570 msgr2=0x7f5b5419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:30.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b54198570 0x7f5b5419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b52ffd700 1 -- 172.21.15.45:0/3910144726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440095f0 con 0x7f5b541067e0 2023-10-17T01:07:30.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.434+0000 7f5b527fc700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b54198570 0x7f5b5419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:07:30.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.435+0000 7f5b52ffd700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54198030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b44004c70 tx=0x7f5b44004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:30.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.435+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4401d070 con 0x7f5b541067e0 2023-10-17T01:07:30.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.435+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b44005000 con 0x7f5b541067e0 2023-10-17T01:07:30.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.435+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b541a50b0 con 0x7f5b541067e0 2023-10-17T01:07:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.435+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44022e30 con 0x7f5b541067e0 2023-10-17T01:07:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.435+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a55b0 con 0x7f5b541067e0 2023-10-17T01:07:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.436+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f5b4400f460 con 0x7f5b541067e0 2023-10-17T01:07:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.436+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b54062380 con 0x7f5b541067e0 2023-10-17T01:07:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.437+0000 7f5b3bfff700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5b3c038a70 0x7f5b3c03af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.437+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5b4404c4c0 con 0x7f5b541067e0 2023-10-17T01:07:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.437+0000 7f5b527fc700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5b3c038a70 0x7f5b3c03af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.438+0000 7f5b527fc700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5b3c038a70 0x7f5b3c03af30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009910 tx=0x7f5b48008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.442+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f5b44020960 con 0x7f5b541067e0 2023-10-17T01:07:30.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.674+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5b541a5890 con 0x7f5b541067e0 2023-10-17T01:07:30.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.675+0000 7f5b3bfff700 1 -- 172.21.15.45:0/3910144726 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f5b44027030 con 0x7f5b541067e0 2023-10-17T01:07:30.676 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:07:30.677 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2023-10-17T01:07:30.677 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:07:30.677 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2023-10-17T01:07:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.677+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5b3c038a70 msgr2=0x7f5b3c03af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.677+0000 7f5b5949b700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5b3c038a70 0x7f5b3c03af30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009910 tx=0x7f5b48008040 comp rx=0 tx=0).stop 2023-10-17T01:07:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.677+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 msgr2=0x7f5b54198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.677+0000 7f5b5949b700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54198030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b44004c70 tx=0x7f5b44004ca0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.677+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 shutdown_connections 2023-10-17T01:07:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5b3c038a70 0x7f5b3c03af30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b5419cf20 0x7f5b5419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b541067e0 0x7f5b54198030 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 --2- 172.21.15.45:0/3910144726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b54198570 0x7f5b5419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 >> 172.21.15.45:0/3910144726 conn(0x7f5b540fc310 msgr2=0x7f5b54073210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 shutdown_connections 2023-10-17T01:07:30.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:30.678+0000 7f5b5949b700 1 -- 172.21.15.45:0/3910144726 wait complete. 2023-10-17T01:07:30.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: Updating smithi045:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:30.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.710 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.710 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.710 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:30.710 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:30.710 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: Updating smithi045:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:30.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:30.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:30.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: Updating smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: Updating smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: Updating smithi045:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-17T01:07:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:07:30.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:30.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:30.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:30.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:31.402 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-17T01:07:31.403 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:07:31.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2023-10-17T01:07:31.477 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:07:31.478 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:31.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:31 smithi045 ceph-mon[98474]: Reconfiguring mon.a (unknown last config time)... 2023-10-17T01:07:31.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:31 smithi045 ceph-mon[98474]: Reconfiguring daemon mon.a on smithi045 2023-10-17T01:07:31.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:31 smithi045 ceph-mon[98474]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:31.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:31 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3910144726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:31.589 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:07:31.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2023-10-17T01:07:31.627 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:07:31.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:31.696 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:31 smithi060 ceph-mon[134136]: Reconfiguring mon.a (unknown last config time)... 2023-10-17T01:07:31.697 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:31 smithi060 ceph-mon[134136]: Reconfiguring daemon mon.a on smithi045 2023-10-17T01:07:31.697 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:31 smithi060 ceph-mon[134136]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:31.697 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:31 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3910144726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:31.698 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:07:31.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2023-10-17T01:07:31.730 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:31 smithi181 ceph-mon[134520]: Reconfiguring mon.a (unknown last config time)... 2023-10-17T01:07:31.730 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:31 smithi181 ceph-mon[134520]: Reconfiguring daemon mon.a on smithi045 2023-10-17T01:07:31.730 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:31 smithi181 ceph-mon[134520]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:31.730 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:31 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3910144726' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:31.735 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:07:31.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-17T01:07:31.805 INFO:tasks.cephadm:Adding mgr.x on smithi045 2023-10-17T01:07:31.805 INFO:tasks.cephadm:Adding mgr.y on smithi060 2023-10-17T01:07:31.806 INFO:tasks.cephadm:Adding mgr.z on smithi181 2023-10-17T01:07:31.806 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch apply mgr '3;smithi045=x;smithi060=y;smithi181=z' 2023-10-17T01:07:32.392 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:07:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: Reconfiguring mon.b (monmap changed)... 2023-10-17T01:07:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: Reconfiguring daemon mon.b on smithi060 2023-10-17T01:07:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:32 smithi060 ceph-mon[134136]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:32.867 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:32.867 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:32.867 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: Reconfiguring mon.b (monmap changed)... 2023-10-17T01:07:32.868 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:32.868 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:32.868 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:32.868 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: Reconfiguring daemon mon.b on smithi060 2023-10-17T01:07:32.868 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:32 smithi181 ceph-mon[134520]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:33.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: Reconfiguring mon.b (monmap changed)... 2023-10-17T01:07:33.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:33.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:33.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:33.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: Reconfiguring daemon mon.b on smithi060 2023-10-17T01:07:33.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:32 smithi045 ceph-mon[98474]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.643+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/1471466207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b41013f0 msgr2=0x7fc1b41037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.643+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/1471466207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b41013f0 0x7fc1b41037e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009910 tx=0x7fc1a4009c20 comp rx=0 tx=0).stop 2023-10-17T01:07:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.643+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/1471466207 shutdown_connections 2023-10-17T01:07:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.643+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/1471466207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b41013f0 0x7fc1b41037e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.643+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/1471466207 >> 172.21.15.181:0/1471466207 conn(0x7fc1b40fae20 msgr2=0x7fc1b40fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.644+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/1471466207 shutdown_connections 2023-10-17T01:07:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.644+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/1471466207 wait complete. 2023-10-17T01:07:33.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.644+0000 7fc1b9e2d700 1 Processor -- start 2023-10-17T01:07:33.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 -- start start 2023-10-17T01:07:33.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 0x7fc1b4195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1b41960b0 0x7fc1b419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b4196530 0x7fc1b419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b419de50 con 0x7fc1b4196530 2023-10-17T01:07:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b419dfc0 con 0x7fc1b41960b0 2023-10-17T01:07:33.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b9e2d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b419e180 con 0x7fc1b41013f0 2023-10-17T01:07:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 0x7fc1b4195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 0x7fc1b4195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45088/0 (socket says 172.21.15.181:45088) 2023-10-17T01:07:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 -- 172.21.15.181:0/3615789388 learned_addr learned my addr 172.21.15.181:0/3615789388 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:07:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b3fff700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b4196530 0x7fc1b419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:33.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b2ffd700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1b41960b0 0x7fc1b419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 -- 172.21.15.181:0/3615789388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1b41960b0 msgr2=0x7fc1b419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1b41960b0 0x7fc1b419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 -- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b4196530 msgr2=0x7fc1b419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b4196530 0x7fc1b419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 -- 172.21.15.181:0/3615789388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40095f0 con 0x7fc1b41013f0 2023-10-17T01:07:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.645+0000 7fc1b37fe700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 0x7fc1b4195b70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4004690 tx=0x7fc1a4004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.646+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a401d070 con 0x7fc1b41013f0 2023-10-17T01:07:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.646+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a400be20 con 0x7fc1b41013f0 2023-10-17T01:07:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.646+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b419e400 con 0x7fc1b41013f0 2023-10-17T01:07:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.646+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400f460 con 0x7fc1b41013f0 2023-10-17T01:07:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.646+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b419e970 con 0x7fc1b41013f0 2023-10-17T01:07:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.646+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7fc1a400f5c0 con 0x7fc1b41013f0 2023-10-17T01:07:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.647+0000 7fc1b0ff9700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc19c038a20 0x7fc19c03aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:33.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.647+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1a404d400 con 0x7fc1b41013f0 2023-10-17T01:07:33.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.647+0000 7fc1b2ffd700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc19c038a20 0x7fc19c03aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:33.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.647+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b418f420 con 0x7fc1b41013f0 2023-10-17T01:07:33.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.648+0000 7fc1b2ffd700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc19c038a20 0x7fc19c03aee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009850 tx=0x7fc1a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:33.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.652+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fc1a4027030 con 0x7fc1b41013f0 2023-10-17T01:07:33.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.865+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi060=y;smithi181=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b4061190 con 0x7fc19c038a20 2023-10-17T01:07:33.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.871+0000 7fc1b0ff9700 1 -- 172.21.15.181:0/3615789388 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc1b4061190 con 0x7fc19c038a20 2023-10-17T01:07:33.872 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mgr update... 2023-10-17T01:07:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc19c038a20 msgr2=0x7fc19c03aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc19c038a20 0x7fc19c03aee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009850 tx=0x7fc1a8008040 comp rx=0 tx=0).stop 2023-10-17T01:07:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 msgr2=0x7fc1b4195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:33.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 0x7fc1b4195b70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4004690 tx=0x7fc1a4004cb0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 shutdown_connections 2023-10-17T01:07:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc19c038a20 0x7fc19c03aee0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b4196530 0x7fc1b419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1b41013f0 0x7fc1b4195b70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 --2- 172.21.15.181:0/3615789388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1b41960b0 0x7fc1b419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:33.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.873+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 >> 172.21.15.181:0/3615789388 conn(0x7fc1b40fae20 msgr2=0x7fc1b40fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:33.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.874+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 shutdown_connections 2023-10-17T01:07:33.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:07:33.874+0000 7fc1b9e2d700 1 -- 172.21.15.181:0/3615789388 wait complete. 2023-10-17T01:07:34.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:34.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:34.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: Reconfiguring mon.c (monmap changed)... 2023-10-17T01:07:34.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:34.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:34.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:34.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:33 smithi045 ceph-mon[98474]: Reconfiguring daemon mon.c on smithi181 2023-10-17T01:07:34.058 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:34.058 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:34.058 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: Reconfiguring mon.c (monmap changed)... 2023-10-17T01:07:34.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:34.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:34.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:34.059 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:33 smithi181 ceph-mon[134520]: Reconfiguring daemon mon.c on smithi181 2023-10-17T01:07:34.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:34.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:34.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: Reconfiguring mon.c (monmap changed)... 2023-10-17T01:07:34.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-17T01:07:34.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-17T01:07:34.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:34.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:33 smithi060 ceph-mon[134136]: Reconfiguring daemon mon.c on smithi181 2023-10-17T01:07:34.340 DEBUG:teuthology.orchestra.run.smithi060:mgr.y> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.y.service 2023-10-17T01:07:34.343 DEBUG:teuthology.orchestra.run.smithi181:mgr.z> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.z.service 2023-10-17T01:07:34.345 INFO:tasks.cephadm:Deploying OSDs... 2023-10-17T01:07:34.346 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:07:34.346 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-10-17T01:07:34.374 INFO:journalctl@ceph.mgr.z.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:07:34.380 INFO:journalctl@ceph.mgr.y.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:07:34.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-17T01:07:34.397 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2023-10-17T01:07:34.447 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-17T01:07:34.448 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:34.448 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-17T01:07:34.448 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:34.448 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-17 01:06:44.547450377 +0000 2023-10-17T01:07:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-17 01:00:46.208963593 +0000 2023-10-17T01:07:34.449 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-17 01:00:46.208963593 +0000 2023-10-17T01:07:34.449 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-10-17 01:00:46.181963997 +0000 2023-10-17T01:07:34.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-17T01:07:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-17T01:07:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-17T01:07:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000179191 s, 2.9 MB/s 2023-10-17T01:07:34.522 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-17T01:07:34.568 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2023-10-17T01:07:34.613 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-17T01:07:34.614 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:34.614 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-17T01:07:34.614 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:34.615 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:34.615 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-17 01:06:44.648448856 +0000 2023-10-17T01:07:34.615 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-17 01:00:47.044951077 +0000 2023-10-17T01:07:34.615 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-17 01:00:47.044951077 +0000 2023-10-17T01:07:34.616 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-10-17 01:00:47.019951452 +0000 2023-10-17T01:07:34.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-17T01:07:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-17T01:07:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-17T01:07:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000194683 s, 2.6 MB/s 2023-10-17T01:07:34.723 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-17T01:07:34.769 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2023-10-17T01:07:34.853 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-17T01:07:34.854 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:34.854 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-17T01:07:34.854 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:34.854 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:34.855 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-17 01:06:44.728447650 +0000 2023-10-17T01:07:34.855 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-17 01:00:47.882938531 +0000 2023-10-17T01:07:34.855 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-17 01:00:47.882938531 +0000 2023-10-17T01:07:34.855 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-10-17 01:00:47.859938876 +0000 2023-10-17T01:07:34.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-17T01:07:34.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-17T01:07:34.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-17T01:07:34.961 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000192531 s, 2.7 MB/s 2023-10-17T01:07:34.963 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-17T01:07:35.009 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2023-10-17T01:07:35.095 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-17T01:07:35.095 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:35.095 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-17T01:07:35.095 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:35.096 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:35.096 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-17 01:06:44.806446474 +0000 2023-10-17T01:07:35.096 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-17 01:00:48.723925941 +0000 2023-10-17T01:07:35.096 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-17 01:00:48.723925941 +0000 2023-10-17T01:07:35.096 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2023-10-17 01:00:48.698926315 +0000 2023-10-17T01:07:35.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-17T01:07:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi060=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: Saving service mgr spec with placement smithi045=x;smithi060=y;smithi181=z;count:3 2023-10-17T01:07:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:35.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:35.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:35.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:35.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:35.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-17T01:07:35.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:35.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:35.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:34 smithi181 ceph-mon[134520]: Deploying daemon mgr.z on smithi181 2023-10-17T01:07:35.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi060=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:35.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: Saving service mgr spec with placement smithi045=x;smithi060=y;smithi181=z;count:3 2023-10-17T01:07:35.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.196 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.196 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.196 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:35.196 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:35.196 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:35.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:35.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:35.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-17T01:07:35.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:35.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:35.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:34 smithi045 ceph-mon[98474]: Deploying daemon mgr.z on smithi181 2023-10-17T01:07:35.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-17T01:07:35.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-17T01:07:35.202 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000178711 s, 2.9 MB/s 2023-10-17T01:07:35.204 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-17T01:07:35.250 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-10-17T01:07:35.250 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2023-10-17T01:07:35.272 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-17T01:07:35.272 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2023-10-17T01:07:35.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi060=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:35.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: Saving service mgr spec with placement smithi045=x;smithi060=y;smithi181=z;count:3 2023-10-17T01:07:35.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:35.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:35.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:35.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:35.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:35.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:35.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-17T01:07:35.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:35.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:35.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:34 smithi060 ceph-mon[134136]: Deploying daemon mgr.z on smithi181 2023-10-17T01:07:35.333 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-17T01:07:35.333 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:35.333 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 382662 Links: 1 2023-10-17T01:07:35.333 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:35.334 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:35.334 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-10-17 01:07:21.186356274 +0000 2023-10-17T01:07:35.334 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-10-17 00:57:36.701786399 +0000 2023-10-17T01:07:35.334 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-10-17 00:57:36.701786399 +0000 2023-10-17T01:07:35.334 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-10-17T01:07:35.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-17T01:07:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-10-17T01:07:35.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-10-17T01:07:35.404 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000129946 s, 3.9 MB/s 2023-10-17T01:07:35.405 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-17T01:07:35.465 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2023-10-17T01:07:35.524 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-17T01:07:35.525 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:35.525 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 394396 Links: 1 2023-10-17T01:07:35.525 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:35.525 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:35.526 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-10-17 01:07:21.186356274 +0000 2023-10-17T01:07:35.526 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-10-17 00:57:37.214781825 +0000 2023-10-17T01:07:35.526 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-10-17 00:57:37.214781825 +0000 2023-10-17T01:07:35.526 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-10-17T01:07:35.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-17T01:07:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-10-17T01:07:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-10-17T01:07:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000121515 s, 4.2 MB/s 2023-10-17T01:07:35.596 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-17T01:07:35.655 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2023-10-17T01:07:35.714 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-17T01:07:35.715 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:35.715 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 381850 Links: 1 2023-10-17T01:07:35.715 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:35.715 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:35.716 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-10-17 01:07:21.186356274 +0000 2023-10-17T01:07:35.716 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-10-17 00:57:37.716777349 +0000 2023-10-17T01:07:35.716 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-10-17 00:57:37.716777349 +0000 2023-10-17T01:07:35.716 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-10-17T01:07:35.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-17T01:07:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-10-17T01:07:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-10-17T01:07:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000118863 s, 4.3 MB/s 2023-10-17T01:07:35.785 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-17T01:07:35.844 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2023-10-17T01:07:35.904 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-17T01:07:35.904 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:35.904 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 390684 Links: 1 2023-10-17T01:07:35.904 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:35.905 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:35.905 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-10-17 01:07:21.186356274 +0000 2023-10-17T01:07:35.905 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-10-17 00:57:38.217772881 +0000 2023-10-17T01:07:35.905 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-10-17 00:57:38.217772881 +0000 2023-10-17T01:07:35.906 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-10-17T01:07:35.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-17T01:07:35.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-10-17T01:07:35.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-10-17T01:07:35.972 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.00010566 s, 4.8 MB/s 2023-10-17T01:07:35.973 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-17T01:07:36.033 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-17T01:07:36.033 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-10-17T01:07:36.052 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-17T01:07:36.052 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-10-17T01:07:36.109 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-17T01:07:36.109 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:36.109 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 382920 Links: 1 2023-10-17T01:07:36.110 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:36.110 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:36.110 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-17 01:07:35.793066839 +0000 2023-10-17T01:07:36.110 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-17 00:57:08.517945197 +0000 2023-10-17T01:07:36.110 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-17 00:57:08.517945197 +0000 2023-10-17T01:07:36.111 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-17T01:07:36.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-17T01:07:36.119 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:35 smithi181 systemd[1]: Starting Ceph mgr.z for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:07:36.148 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-17T01:07:36.148 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-17T01:07:36.148 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000251168 s, 2.0 MB/s 2023-10-17T01:07:36.149 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-17T01:07:36.211 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-10-17T01:07:36.268 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-17T01:07:36.269 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:36.269 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 396294 Links: 1 2023-10-17T01:07:36.269 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:36.269 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:36.270 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-17 01:07:35.794066821 +0000 2023-10-17T01:07:36.270 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-17 00:57:09.047935430 +0000 2023-10-17T01:07:36.270 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-17 00:57:09.047935430 +0000 2023-10-17T01:07:36.270 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-17T01:07:36.271 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-17T01:07:36.334 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-17T01:07:36.335 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-17T01:07:36.335 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000229637 s, 2.2 MB/s 2023-10-17T01:07:36.336 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-17T01:07:36.393 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-10-17T01:07:36.409 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 podman[136369]: 2023-10-17T01:07:36.410 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 bash[136369]: e48df65a8591c60b5cfec4b1f50d1996b9664b62ea6deabbd3108e656a1cd545 2023-10-17T01:07:36.410 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 ceph-mgr[136530]: -- 172.21.15.181:0/852417170 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ed3a0b1e0 con 0x555ed39e1000 2023-10-17T01:07:36.427 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-17T01:07:36.427 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:36.427 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 387342 Links: 1 2023-10-17T01:07:36.427 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:36.427 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:36.428 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-17 01:07:35.794066821 +0000 2023-10-17T01:07:36.428 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-17 00:57:09.555926069 +0000 2023-10-17T01:07:36.428 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-17 00:57:09.555926069 +0000 2023-10-17T01:07:36.428 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-17T01:07:36.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-17T01:07:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-17T01:07:36.495 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-17T01:07:36.496 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000220729 s, 2.3 MB/s 2023-10-17T01:07:36.497 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-17T01:07:36.554 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-10-17T01:07:36.614 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-17T01:07:36.615 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-17T01:07:36.615 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 393617 Links: 1 2023-10-17T01:07:36.615 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-17T01:07:36.615 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-17T01:07:36.615 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-17 01:07:35.794066821 +0000 2023-10-17T01:07:36.616 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-17 00:57:10.054916873 +0000 2023-10-17T01:07:36.616 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-17 00:57:10.054916873 +0000 2023-10-17T01:07:36.616 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-17T01:07:36.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-17T01:07:36.679 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 systemd[1]: Started Ceph mgr.z for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:07:36.679 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:36.494+0000 7f24e5adf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-17T01:07:36.679 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:36.576+0000 7f24e5adf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-17T01:07:36.680 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:36.668+0000 7f24e5adf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-17T01:07:36.681 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-17T01:07:36.681 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-17T01:07:36.682 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000199913 s, 2.6 MB/s 2023-10-17T01:07:36.683 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-17T01:07:36.741 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2023-10-17T01:07:36.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-17T01:07:36.936 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:36.768+0000 7f24e5adf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-17T01:07:37.297 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:36.935+0000 7f24e5adf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-17T01:07:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:07:37.609 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:37.297+0000 7f24e5adf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-17T01:07:37.609 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:37.426+0000 7f24e5adf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-17T01:07:37.609 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:37.514+0000 7f24e5adf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-17T01:07:37.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:37.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.611 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:37.611 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-17T01:07:37.611 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:37.611 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:37.754 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:37.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:37.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-17T01:07:37.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:37.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:37.793 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:37.793 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.793 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.794 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:37.794 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:37.794 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-17T01:07:37.794 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:37.795 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:37.907 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:37.608+0000 7f24e5adf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-17T01:07:38.078 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 systemd[1]: Starting Ceph mgr.y for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:07:38.407 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:38.027+0000 7f24e5adf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-17T01:07:38.556 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:38 smithi060 ceph-mon[134136]: Deploying daemon mgr.y on smithi060 2023-10-17T01:07:38.557 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 podman[135579]: 2023-10-17T01:07:38.557 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 bash[135579]: fb7dfbe4e42a9e2da3b90f2872021a44df2f4a6ba7b568f72357e935c457ca32 2023-10-17T01:07:38.688 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:38 smithi181 ceph-mon[134520]: Deploying daemon mgr.y on smithi060 2023-10-17T01:07:38.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:38 smithi045 ceph-mon[98474]: Deploying daemon mgr.y on smithi060 2023-10-17T01:07:38.828 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 ceph-mgr[135619]: -- 172.21.15.60:0/910481573 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e8fe9571e0 con 0x55e8fe92d000 2023-10-17T01:07:38.829 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 systemd[1]: Started Ceph mgr.y for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:07:38.829 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:38.704+0000 7fc5e6cb6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-17T01:07:38.829 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:38.800+0000 7fc5e6cb6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-17T01:07:39.157 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:38.688+0000 7f24e5adf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-17T01:07:39.157 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:38.770+0000 7f24e5adf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-17T01:07:39.238 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:38.909+0000 7fc5e6cb6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-17T01:07:39.238 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:39.023+0000 7fc5e6cb6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-17T01:07:39.238 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:39.236+0000 7fc5e6cb6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-17T01:07:39.545 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-17T01:07:39.545 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-17T01:07:39.545 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-10-17T01:07:39.546 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-10-17T01:07:39.546 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0315454 s, 332 MB/s 2023-10-17T01:07:39.546 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-10-17T01:07:39.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-mon[134136]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:39.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:39.674 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:39.580+0000 7f24e5adf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:07:39.675 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:39.673+0000 7f24e5adf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-17T01:07:39.675 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-mon[134520]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:39.675 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.675 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.676 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.676 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.676 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:39.688 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:39 smithi045 ceph-mon[98474]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:39.688 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:39 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:39 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:39 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:39 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:39.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:39 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:39.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2023-10-17T01:07:40.009 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:39.736+0000 7fc5e6cb6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-17T01:07:40.010 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:39 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:39.897+0000 7fc5e6cb6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-17T01:07:40.157 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:39.858+0000 7f24e5adf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:07:40.328 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:40.008+0000 7fc5e6cb6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-17T01:07:40.328 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:40.135+0000 7fc5e6cb6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-17T01:07:40.544 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:40.157+0000 7f24e5adf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-17T01:07:40.545 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:40.394+0000 7f24e5adf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-17T01:07:40.715 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:07:40.807 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:40.543+0000 7f24e5adf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-17T01:07:41.078 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:40.658+0000 7fc5e6cb6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-17T01:07:41.157 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:40.806+0000 7f24e5adf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-17T01:07:41.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:41.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:41.657 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:41.501+0000 7f24e5adf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-17T01:07:41.657 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:41.630+0000 7f24e5adf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-17T01:07:41.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:41.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:41.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:41.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:41.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:41.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:41.828 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:41.455+0000 7fc5e6cb6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-17T01:07:41.828 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:41.553+0000 7fc5e6cb6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-17T01:07:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:41.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:41.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:41.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-17T01:07:41.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-17T01:07:41.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:42.149 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:41.822+0000 7f24e5adf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-17T01:07:42.150 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:41.934+0000 7f24e5adf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-17T01:07:42.407 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:42.148+0000 7f24e5adf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-17T01:07:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 -- 172.21.15.45:0/1487104412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170101fe0 msgr2=0x7f0170102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 --2- 172.21.15.45:0/1487104412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170101fe0 0x7f0170102440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01640099c0 tx=0x7f0164009cd0 comp rx=0 tx=0).stop 2023-10-17T01:07:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 -- 172.21.15.45:0/1487104412 shutdown_connections 2023-10-17T01:07:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 --2- 172.21.15.45:0/1487104412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0170102980 0x7f0170109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 --2- 172.21.15.45:0/1487104412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170100de0 0x7f0170101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 --2- 172.21.15.45:0/1487104412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170101fe0 0x7f0170102440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 -- 172.21.15.45:0/1487104412 >> 172.21.15.45:0/1487104412 conn(0x7f01700fc340 msgr2=0x7f01700fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:07:42.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 -- 172.21.15.45:0/1487104412 shutdown_connections 2023-10-17T01:07:42.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.442+0000 7f017556d700 1 -- 172.21.15.45:0/1487104412 wait complete. 2023-10-17T01:07:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.443+0000 7f017556d700 1 Processor -- start 2023-10-17T01:07:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 -- start start 2023-10-17T01:07:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0170100de0 0x7f0170074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 0x7f0170073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170102980 0x7f0170073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0170073e00 con 0x7f0170102980 2023-10-17T01:07:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0170073f70 con 0x7f0170100de0 2023-10-17T01:07:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f017556d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01701a0b60 con 0x7f0170101fe0 2023-10-17T01:07:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f016e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 0x7f0170073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f016e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 0x7f0170073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50690/0 (socket says 172.21.15.45:50690) 2023-10-17T01:07:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f016e7fc700 1 -- 172.21.15.45:0/1312778476 learned_addr learned my addr 172.21.15.45:0/1312778476 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:07:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.444+0000 7f016effd700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0170100de0 0x7f0170074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f016e7fc700 1 -- 172.21.15.45:0/1312778476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0170100de0 msgr2=0x7f0170074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f016e7fc700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0170100de0 0x7f0170074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f016e7fc700 1 -- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170102980 msgr2=0x7f0170073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:07:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f016e7fc700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170102980 0x7f0170073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:07:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f016e7fc700 1 -- 172.21.15.45:0/1312778476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01640095f0 con 0x7f0170101fe0 2023-10-17T01:07:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f016e7fc700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 0x7f0170073140 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0164009fd0 tx=0x7f016400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016401d070 con 0x7f0170101fe0 2023-10-17T01:07:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0164005050 con 0x7f0170101fe0 2023-10-17T01:07:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01701a0de0 con 0x7f0170101fe0 2023-10-17T01:07:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164004ae0 con 0x7f0170101fe0 2023-10-17T01:07:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.445+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01701a11a0 con 0x7f0170101fe0 2023-10-17T01:07:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.446+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45478+0+0 (secure 0 0 0) 0x7f0164004c40 con 0x7f0170101fe0 2023-10-17T01:07:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.446+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017010c6a0 con 0x7f0170101fe0 2023-10-17T01:07:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.447+0000 7f0157fff700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f01580387d0 0x7f015803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:07:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.447+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f016404bee0 con 0x7f0170101fe0 2023-10-17T01:07:42.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.447+0000 7f016effd700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f01580387d0 0x7f015803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:07:42.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.448+0000 7f016effd700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f01580387d0 0x7f015803ac90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0170101e40 tx=0x7f0160009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:07:42.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.452+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0164029750 con 0x7f0170101fe0 2023-10-17T01:07:42.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:42.689+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0170061190 con 0x7f01580387d0 2023-10-17T01:07:42.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:42 smithi045 ceph-mon[98474]: Reconfiguring mgr.x (unknown last config time)... 2023-10-17T01:07:42.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:42 smithi045 ceph-mon[98474]: Reconfiguring daemon mgr.x on smithi045 2023-10-17T01:07:42.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:42 smithi060 ceph-mon[134136]: Reconfiguring mgr.x (unknown last config time)... 2023-10-17T01:07:42.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:42 smithi060 ceph-mon[134136]: Reconfiguring daemon mgr.x on smithi045 2023-10-17T01:07:42.828 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:42.526+0000 7fc5e6cb6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:07:42.829 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:42.638+0000 7fc5e6cb6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-17T01:07:42.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:42 smithi181 ceph-mon[134520]: Reconfiguring mgr.x (unknown last config time)... 2023-10-17T01:07:42.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:42 smithi181 ceph-mon[134520]: Reconfiguring daemon mgr.x on smithi045 2023-10-17T01:07:43.213 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:42.862+0000 7fc5e6cb6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-17T01:07:43.407 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z[136444]: 2023-10-17T01:07:43.008+0000 7f24e5adf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-17T01:07:43.466 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:43.212+0000 7fc5e6cb6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-17T01:07:43.467 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:43.467 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.467 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.468 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.468 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:43.468 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:43.468 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:43.468 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.469 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:07:43.469 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:07:43.469 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:43.469 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: Standby manager daemon z started 2023-10-17T01:07:43.469 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-17T01:07:43.470 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-17T01:07:43.470 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-17T01:07:43.470 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-17T01:07:43.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:43.580+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 14) v1 ==== 90477+0+0 (secure 0 0 0) 0x7f01640051c0 con 0x7f0170101fe0 2023-10-17T01:07:43.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:43.776 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:43.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:43.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:43.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:07:43.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:07:43.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:43.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: Standby manager daemon z started 2023-10-17T01:07:43.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-17T01:07:43.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-17T01:07:43.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-17T01:07:43.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:43 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-17T01:07:43.828 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:43.465+0000 7fc5e6cb6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-17T01:07:43.828 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:43.649+0000 7fc5e6cb6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-17T01:07:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:07:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:07:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:07:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:07:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:07:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:43.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: Standby manager daemon z started 2023-10-17T01:07:43.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-17T01:07:43.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-17T01:07:43.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-17T01:07:43.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:43 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.181:0/3397277286' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-17T01:07:44.328 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:43.978+0000 7fc5e6cb6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-17T01:07:44.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:44 smithi045 ceph-mon[98474]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:44.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:44 smithi045 ceph-mon[98474]: mgrmap e14: x(active, since 71s), standbys: z 2023-10-17T01:07:44.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-17T01:07:44.807 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:44 smithi060 ceph-mon[134136]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:44.808 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:44 smithi060 ceph-mon[134136]: mgrmap e14: x(active, since 71s), standbys: z 2023-10-17T01:07:44.808 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-17T01:07:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:44 smithi181 ceph-mon[134520]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:07:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:44 smithi181 ceph-mon[134520]: mgrmap e14: x(active, since 71s), standbys: z 2023-10-17T01:07:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-17T01:07:45.078 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:44.806+0000 7fc5e6cb6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-17T01:07:45.078 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:44.967+0000 7fc5e6cb6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-17T01:07:45.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:45 smithi060 ceph-mon[134136]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:45.578 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:45.214+0000 7fc5e6cb6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-17T01:07:45.579 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:45.351+0000 7fc5e6cb6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-17T01:07:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:45 smithi045 ceph-mon[98474]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:45 smithi181 ceph-mon[134520]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:46.078 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:45.626+0000 7fc5e6cb6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-17T01:07:46.674 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:46 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1317850118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]: dispatch 2023-10-17T01:07:46.675 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:46 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]: dispatch 2023-10-17T01:07:46.675 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:46 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]': finished 2023-10-17T01:07:46.675 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:46 smithi060 ceph-mon[134136]: osdmap e5: 1 total, 0 up, 1 in 2023-10-17T01:07:46.675 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:07:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:46 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1317850118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]: dispatch 2023-10-17T01:07:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:46 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]: dispatch 2023-10-17T01:07:46.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:46 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]': finished 2023-10-17T01:07:46.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:46 smithi045 ceph-mon[98474]: osdmap e5: 1 total, 0 up, 1 in 2023-10-17T01:07:46.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:07:46.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:46 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1317850118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]: dispatch 2023-10-17T01:07:46.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:46 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]: dispatch 2023-10-17T01:07:46.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:46 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84831425-d402-4501-8e75-de13106f9cf9"}]': finished 2023-10-17T01:07:46.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:46 smithi181 ceph-mon[134520]: osdmap e5: 1 total, 0 up, 1 in 2023-10-17T01:07:46.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:07:47.078 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:07:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y[135595]: 2023-10-17T01:07:46.673+0000 7fc5e6cb6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-17T01:07:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:07:47.430+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mon.1 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 15) v1 ==== 135476+0+0 (secure 0 0 0) 0x7f0164013250 con 0x7f0170101fe0 2023-10-17T01:07:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3326312515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:07:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: Standby manager daemon y started 2023-10-17T01:07:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-17T01:07:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-17T01:07:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-17T01:07:47.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:47 smithi045 ceph-mon[98474]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-17T01:07:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3326312515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:07:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: Standby manager daemon y started 2023-10-17T01:07:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-17T01:07:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-17T01:07:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-17T01:07:47.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:47 smithi060 ceph-mon[134136]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-17T01:07:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3326312515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:07:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: Standby manager daemon y started 2023-10-17T01:07:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-17T01:07:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-17T01:07:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-17T01:07:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:47 smithi181 ceph-mon[134520]: from='mgr.? 172.21.15.60:0/1687221497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-17T01:07:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:48 smithi045 ceph-mon[98474]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-10-17T01:07:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-17T01:07:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:48 smithi060 ceph-mon[134136]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-10-17T01:07:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-17T01:07:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:48 smithi181 ceph-mon[134520]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-10-17T01:07:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-17T01:07:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:49 smithi045 ceph-mon[98474]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:49 smithi060 ceph-mon[134136]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:49 smithi181 ceph-mon[134520]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:51.543 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:51 smithi045 ceph-mon[98474]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:51 smithi060 ceph-mon[134136]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:51.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:51 smithi181 ceph-mon[134520]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:53.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:53 smithi045 ceph-mon[98474]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:53.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:53 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-17T01:07:53.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:53 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:53 smithi060 ceph-mon[134136]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:53 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-17T01:07:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:53 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:53 smithi181 ceph-mon[134520]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:53 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-17T01:07:53.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:53 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:07:54.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:54 smithi045 ceph-mon[98474]: Deploying daemon osd.0 on smithi045 2023-10-17T01:07:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:54 smithi060 ceph-mon[134136]: Deploying daemon osd.0 on smithi045 2023-10-17T01:07:54.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:54 smithi181 ceph-mon[134520]: Deploying daemon osd.0 on smithi045 2023-10-17T01:07:55.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:55 smithi045 ceph-mon[98474]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:55 smithi060 ceph-mon[134136]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:55 smithi181 ceph-mon[134520]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:57 smithi045 ceph-mon[98474]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:57 smithi060 ceph-mon[134136]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:57 smithi181 ceph-mon[134520]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:59.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:07:59 smithi045 ceph-mon[98474]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:59.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:07:59 smithi060 ceph-mon[134136]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:07:59.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:07:59 smithi181 ceph-mon[134520]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:00.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:00 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:00.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:00 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:00.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:00 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:00.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:00.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:00.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:00.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:00 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:00.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:00 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:00.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:00 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:01.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:01 smithi045 ceph-mon[98474]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:01.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:01 smithi060 ceph-mon[134136]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:01.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:01 smithi181 ceph-mon[134520]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.708+0000 7f0157fff700 1 -- 172.21.15.45:0/1312778476 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0170061190 con 0x7f01580387d0 2023-10-17T01:08:02.710 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2023-10-17T01:08:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.711+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f01580387d0 msgr2=0x7f015803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.711+0000 7f017556d700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f01580387d0 0x7f015803ac90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0170101e40 tx=0x7f0160009260 comp rx=0 tx=0).stop 2023-10-17T01:08:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.711+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 msgr2=0x7f0170073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.711+0000 7f017556d700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 0x7f0170073140 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0164009fd0 tx=0x7f016400f690 comp rx=0 tx=0).stop 2023-10-17T01:08:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.711+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 shutdown_connections 2023-10-17T01:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.711+0000 7f017556d700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f01580387d0 0x7f015803ac90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.712+0000 7f017556d700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0170102980 0x7f0170073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.712+0000 7f017556d700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0170100de0 0x7f0170074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.712+0000 7f017556d700 1 --2- 172.21.15.45:0/1312778476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170101fe0 0x7f0170073140 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:02.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.712+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 >> 172.21.15.45:0/1312778476 conn(0x7f01700fc340 msgr2=0x7f01700fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:08:02.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.712+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 shutdown_connections 2023-10-17T01:08:02.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:02.712+0000 7f017556d700 1 -- 172.21.15.45:0/1312778476 wait complete. 2023-10-17T01:08:03.166 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.0.service 2023-10-17T01:08:03.171 INFO:tasks.cephadm:Deploying osd.1 on smithi060 with /dev/vg_nvme/lv_4... 2023-10-17T01:08:03.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-17T01:08:03.240 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.240 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.240 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:03.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:03.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:03.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:03.255 INFO:journalctl@ceph.osd.0.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:08:03.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:03.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:03.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:03.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:03.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:03.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:03.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:03.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:03.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:03.551 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:03 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0[116817]: 2023-10-17T01:08:03.260+0000 7fcf9b593640 -1 osd.0 0 log_to_monitors true 2023-10-17T01:08:03.746 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:08:04.175 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:03 smithi045 ceph-mon[98474]: from='osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-17T01:08:04.176 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:03 smithi045 ceph-mon[98474]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-17T01:08:04.243 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:03 smithi060 ceph-mon[134136]: from='osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-17T01:08:04.244 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:03 smithi060 ceph-mon[134136]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-17T01:08:04.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:03 smithi181 ceph-mon[134520]: from='osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-17T01:08:04.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:03 smithi181 ceph-mon[134520]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-17T01:08:05.237 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-17T01:08:05.237 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-17T01:08:05.237 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-10-17T01:08:05.237 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-10-17T01:08:05.238 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00816465 s, 1.3 GB/s 2023-10-17T01:08:05.238 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-10-17T01:08:05.272 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:05 smithi060 ceph-mon[134136]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-17T01:08:05.272 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:05 smithi060 ceph-mon[134136]: osdmap e6: 1 total, 0 up, 1 in 2023-10-17T01:08:05.273 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:05 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:05.273 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:05 smithi060 ceph-mon[134136]: from='osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:08:05.273 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:05 smithi060 ceph-mon[134136]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:08:05.273 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:05 smithi060 ceph-mon[134136]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:05.301 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0[116817]: 2023-10-17T01:08:05.009+0000 7fcf90c0a700 -1 osd.0 0 waiting for initial osdmap 2023-10-17T01:08:05.301 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0[116817]: 2023-10-17T01:08:05.013+0000 7fcf8b1fc700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:08:05.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-mon[98474]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-17T01:08:05.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-mon[98474]: osdmap e6: 1 total, 0 up, 1 in 2023-10-17T01:08:05.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:05.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-mon[98474]: from='osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:08:05.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-mon[98474]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:08:05.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:05 smithi045 ceph-mon[98474]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:05.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:05 smithi181 ceph-mon[134520]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-17T01:08:05.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:05 smithi181 ceph-mon[134520]: osdmap e6: 1 total, 0 up, 1 in 2023-10-17T01:08:05.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:05 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:05.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:05 smithi181 ceph-mon[134520]: from='osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:08:05.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:05 smithi181 ceph-mon[134520]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:08:05.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:05 smithi181 ceph-mon[134520]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:05.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2023-10-17T01:08:06.153 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:08:06.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:06 smithi045 ceph-mon[98474]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:08:06.295 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:06 smithi045 ceph-mon[98474]: osdmap e7: 1 total, 0 up, 1 in 2023-10-17T01:08:06.295 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:06.295 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:06.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:06 smithi060 ceph-mon[134136]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:08:06.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:06 smithi060 ceph-mon[134136]: osdmap e7: 1 total, 0 up, 1 in 2023-10-17T01:08:06.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:06.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:06.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:06 smithi181 ceph-mon[134520]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:08:06.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:06 smithi181 ceph-mon[134520]: osdmap e7: 1 total, 0 up, 1 in 2023-10-17T01:08:06.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:06.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:08:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:08:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] boot 2023-10-17T01:08:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: osdmap e8: 1 total, 1 up, 1 in 2023-10-17T01:08:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:07.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:07.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:08:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:08:07.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] boot 2023-10-17T01:08:07.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: osdmap e8: 1 total, 1 up, 1 in 2023-10-17T01:08:07.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:07.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:07.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:07.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:07.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 -- 172.21.15.60:0/877042554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090101870 msgr2=0x7fe090103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 --2- 172.21.15.60:0/877042554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090101870 0x7fe090103c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe080009990 tx=0x7fe080009ca0 comp rx=0 tx=0).stop 2023-10-17T01:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 -- 172.21.15.60:0/877042554 shutdown_connections 2023-10-17T01:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 --2- 172.21.15.60:0/877042554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe090106ad0 0x7fe090108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 --2- 172.21.15.60:0/877042554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090101870 0x7fe090103c60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 --2- 172.21.15.60:0/877042554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0901041a0 0x7fe090106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:07.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 -- 172.21.15.60:0/877042554 >> 172.21.15.60:0/877042554 conn(0x7fe0900fb260 msgr2=0x7fe0900fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:08:07.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 -- 172.21.15.60:0/877042554 shutdown_connections 2023-10-17T01:08:07.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.367+0000 7fe0986cb700 1 -- 172.21.15.60:0/877042554 wait complete. 2023-10-17T01:08:07.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 Processor -- start 2023-10-17T01:08:07.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 -- start start 2023-10-17T01:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe090101870 0x7fe0901011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0901041a0 0x7fe0900ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 0x7fe0900ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901004e0 con 0x7fe0901041a0 2023-10-17T01:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe090100650 con 0x7fe090106ad0 2023-10-17T01:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe0986cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901a0b30 con 0x7fe090101870 2023-10-17T01:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe096c68700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 0x7fe0900ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe096c68700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 0x7fe0900ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37354/0 (socket says 172.21.15.60:37354) 2023-10-17T01:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe096c68700 1 -- 172.21.15.60:0/81328660 learned_addr learned my addr 172.21.15.60:0/81328660 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-10-17T01:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe096467700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe090101870 0x7fe0901011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.368+0000 7fe095c66700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0901041a0 0x7fe0900ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe096c68700 1 -- 172.21.15.60:0/81328660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe090101870 msgr2=0x7fe0901011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe096c68700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe090101870 0x7fe0901011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe096c68700 1 -- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0901041a0 msgr2=0x7fe0900ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:07.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe096c68700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0901041a0 0x7fe0900ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:07.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe096c68700 1 -- 172.21.15.60:0/81328660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0800095f0 con 0x7fe090106ad0 2023-10-17T01:08:07.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe096c68700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 0x7fe0900ffd60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0880094f0 tx=0x7fe08800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:08:07.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088004db0 con 0x7fe090106ad0 2023-10-17T01:08:07.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08800f040 con 0x7fe090106ad0 2023-10-17T01:08:07.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0901a0e10 con 0x7fe090106ad0 2023-10-17T01:08:07.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088005980 con 0x7fe090106ad0 2023-10-17T01:08:07.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.369+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0901a1230 con 0x7fe090106ad0 2023-10-17T01:08:07.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.370+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0900fcde0 con 0x7fe090106ad0 2023-10-17T01:08:07.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.371+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 15) v1 ==== 135476+0+0 (secure 0 0 0) 0x7fe088005000 con 0x7fe090106ad0 2023-10-17T01:08:07.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.372+0000 7fe0877fe700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe07c0a0a90 0x7fe07c0a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:07.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.372+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fe0880ca750 con 0x7fe090106ad0 2023-10-17T01:08:07.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.372+0000 7fe096467700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe07c0a0a90 0x7fe07c0a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:07.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.373+0000 7fe096467700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe07c0a0a90 0x7fe07c0a2f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe08000b5c0 tx=0x7fe080005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:08:07.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.375+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe088095b50 con 0x7fe090106ad0 2023-10-17T01:08:07.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:08:07.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:08:07.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: osd.0 [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] boot 2023-10-17T01:08:07.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: osdmap e8: 1 total, 1 up, 1 in 2023-10-17T01:08:07.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-17T01:08:07.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-17T01:08:07.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:07.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:07.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:07.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:07.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:07.582+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe090061190 con 0x7fe07c0a0a90 2023-10-17T01:08:08.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:08 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi045 2023-10-17T01:08:08.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:08 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi045 to 17276M 2023-10-17T01:08:08.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:08 smithi045 ceph-mon[98474]: osdmap e9: 1 total, 1 up, 1 in 2023-10-17T01:08:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:08.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:08.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:08 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi045 2023-10-17T01:08:08.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:08 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi045 to 17276M 2023-10-17T01:08:08.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:08 smithi060 ceph-mon[134136]: osdmap e9: 1 total, 1 up, 1 in 2023-10-17T01:08:08.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:08.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:08.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:08.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:08 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi045 2023-10-17T01:08:08.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:08 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi045 to 17276M 2023-10-17T01:08:08.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:08 smithi181 ceph-mon[134520]: osdmap e9: 1 total, 1 up, 1 in 2023-10-17T01:08:08.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:08.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:08.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:09.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:09 smithi060 ceph-mon[134136]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:09.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:09 smithi060 ceph-mon[134136]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:09.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:09 smithi181 ceph-mon[134520]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:09.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:09 smithi181 ceph-mon[134520]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:09 smithi045 ceph-mon[98474]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:09.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:09 smithi045 ceph-mon[98474]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:10.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:10 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/1519816497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]: dispatch 2023-10-17T01:08:10.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:10 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]: dispatch 2023-10-17T01:08:10.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:10 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]': finished 2023-10-17T01:08:10.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:10 smithi060 ceph-mon[134136]: osdmap e10: 2 total, 1 up, 2 in 2023-10-17T01:08:10.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:10.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:10 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/1519816497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]: dispatch 2023-10-17T01:08:10.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:10 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]: dispatch 2023-10-17T01:08:10.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:10 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]': finished 2023-10-17T01:08:10.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:10 smithi181 ceph-mon[134520]: osdmap e10: 2 total, 1 up, 2 in 2023-10-17T01:08:10.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:10 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/1519816497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]: dispatch 2023-10-17T01:08:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:10 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]: dispatch 2023-10-17T01:08:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:10 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a91b91e7-247a-46b5-b229-bdadf8126588"}]': finished 2023-10-17T01:08:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:10 smithi045 ceph-mon[98474]: osdmap e10: 2 total, 1 up, 2 in 2023-10-17T01:08:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:11.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:11 smithi060 ceph-mon[134136]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:11.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:11 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/2550155583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:11.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:11 smithi181 ceph-mon[134520]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:11.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:11 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/2550155583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:11 smithi045 ceph-mon[98474]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:11 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/2550155583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:13.718 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:13 smithi060 ceph-mon[134136]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:13 smithi045 ceph-mon[98474]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:13 smithi181 ceph-mon[134520]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:15.686 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:15 smithi060 ceph-mon[134136]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:15.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:15 smithi045 ceph-mon[98474]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:15 smithi181 ceph-mon[134520]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:17 smithi045 ceph-mon[98474]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-17T01:08:17.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:17 smithi060 ceph-mon[134136]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-17T01:08:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:17 smithi181 ceph-mon[134520]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-17T01:08:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:18.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:18 smithi060 ceph-mon[134136]: Deploying daemon osd.1 on smithi060 2023-10-17T01:08:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:18 smithi045 ceph-mon[98474]: Deploying daemon osd.1 on smithi060 2023-10-17T01:08:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:18 smithi181 ceph-mon[134520]: Deploying daemon osd.1 on smithi060 2023-10-17T01:08:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:19 smithi045 ceph-mon[98474]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:19 smithi060 ceph-mon[134136]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:19 smithi181 ceph-mon[134520]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:21 smithi045 ceph-mon[98474]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:21.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:21.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:21.814 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:21 smithi060 ceph-mon[134136]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:21.814 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:21.814 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:21.815 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:21 smithi181 ceph-mon[134520]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:21.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:23.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.554+0000 7fe0877fe700 1 -- 172.21.15.60:0/81328660 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe090061190 con 0x7fe07c0a0a90 2023-10-17T01:08:23.556 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 1 on host 'smithi060' 2023-10-17T01:08:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe07c0a0a90 msgr2=0x7fe07c0a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe07c0a0a90 0x7fe07c0a2f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe08000b5c0 tx=0x7fe080005fd0 comp rx=0 tx=0).stop 2023-10-17T01:08:23.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 msgr2=0x7fe0900ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:23.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 0x7fe0900ffd60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0880094f0 tx=0x7fe08800d3b0 comp rx=0 tx=0).stop 2023-10-17T01:08:23.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 shutdown_connections 2023-10-17T01:08:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe07c0a0a90 0x7fe07c0a2f50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe090106ad0 0x7fe0900ffd60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe090101870 0x7fe0901011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:23.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 --2- 172.21.15.60:0/81328660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0901041a0 0x7fe0900ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:23.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 >> 172.21.15.60:0/81328660 conn(0x7fe0900fb260 msgr2=0x7fe0901024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:08:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 shutdown_connections 2023-10-17T01:08:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:08:23.556+0000 7fe0986cb700 1 -- 172.21.15.60:0/81328660 wait complete. 2023-10-17T01:08:23.699 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:23 smithi060 ceph-mon[134136]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:23.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:23.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:23.701 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:23 smithi045 ceph-mon[98474]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:23 smithi181 ceph-mon[134520]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:23.950 DEBUG:teuthology.orchestra.run.smithi060:osd.1> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.1.service 2023-10-17T01:08:23.952 INFO:tasks.cephadm:Deploying osd.2 on smithi181 with /dev/vg_nvme/lv_4... 2023-10-17T01:08:23.953 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-17T01:08:23.984 INFO:journalctl@ceph.osd.1.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:08:24.516 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:08:24.782 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:24.783 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:24.783 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:24.783 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:24 smithi060 ceph-mon[134136]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:24.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:24 smithi045 ceph-mon[98474]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:24.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:24.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:24.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:24.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:24 smithi181 ceph-mon[134520]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:25.078 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1[139250]: 2023-10-17T01:08:24.989+0000 7fc90e9ab640 -1 osd.1 0 log_to_monitors true 2023-10-17T01:08:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:25 smithi060 ceph-mon[134136]: from='osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-17T01:08:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:25 smithi060 ceph-mon[134136]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-17T01:08:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:25 smithi181 ceph-mon[134520]: from='osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-17T01:08:25.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:25 smithi181 ceph-mon[134520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-17T01:08:25.981 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-17T01:08:25.981 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-17T01:08:25.981 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-10-17T01:08:25.981 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-10-17T01:08:25.982 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111024 s, 944 MB/s 2023-10-17T01:08:25.982 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-10-17T01:08:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:25 smithi045 ceph-mon[98474]: from='osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-17T01:08:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:25 smithi045 ceph-mon[98474]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-17T01:08:26.360 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2023-10-17T01:08:26.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:26 smithi181 ceph-mon[134520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-17T01:08:26.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:26 smithi181 ceph-mon[134520]: osdmap e11: 2 total, 1 up, 2 in 2023-10-17T01:08:26.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:26.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:26 smithi181 ceph-mon[134520]: from='osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:08:26.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:26 smithi181 ceph-mon[134520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:08:26.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:26 smithi181 ceph-mon[134520]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:26.828 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1[139250]: 2023-10-17T01:08:26.566+0000 7fc904022700 -1 osd.1 0 waiting for initial osdmap 2023-10-17T01:08:26.828 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1[139250]: 2023-10-17T01:08:26.569+0000 7fc8ffe17700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:08:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-mon[134136]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-17T01:08:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-mon[134136]: osdmap e11: 2 total, 1 up, 2 in 2023-10-17T01:08:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-mon[134136]: from='osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:08:26.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-mon[134136]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:08:26.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:26 smithi060 ceph-mon[134136]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:26.912 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:08:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:26 smithi045 ceph-mon[98474]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-17T01:08:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:26 smithi045 ceph-mon[98474]: osdmap e11: 2 total, 1 up, 2 in 2023-10-17T01:08:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:27.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:26 smithi045 ceph-mon[98474]: from='osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:08:27.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:26 smithi045 ceph-mon[98474]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:08:27.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:26 smithi045 ceph-mon[98474]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:08:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: osdmap e12: 2 total, 1 up, 2 in 2023-10-17T01:08:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi060 2023-10-17T01:08:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi060 to 17173M 2023-10-17T01:08:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:27.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:08:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: osdmap e12: 2 total, 1 up, 2 in 2023-10-17T01:08:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi060 2023-10-17T01:08:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi060 to 17173M 2023-10-17T01:08:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:27.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:27.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:27.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:28.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:08:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: osdmap e12: 2 total, 1 up, 2 in 2023-10-17T01:08:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi060 2023-10-17T01:08:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi060 to 17173M 2023-10-17T01:08:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:28.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:28.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:28.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 -- 172.21.15.181:0/4044848943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06501018e0 msgr2=0x7f0650108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 --2- 172.21.15.181:0/4044848943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06501018e0 0x7f0650108170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f064800b0a0 tx=0x7f064800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:08:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 -- 172.21.15.181:0/4044848943 shutdown_connections 2023-10-17T01:08:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 --2- 172.21.15.181:0/4044848943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06501018e0 0x7f0650108170 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 --2- 172.21.15.181:0/4044848943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0650100200 0x7f0650100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:28.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 --2- 172.21.15.181:0/4044848943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0650100f20 0x7f06501013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 -- 172.21.15.181:0/4044848943 >> 172.21.15.181:0/4044848943 conn(0x7f06500faa00 msgr2=0x7f06500fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:08:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 -- 172.21.15.181:0/4044848943 shutdown_connections 2023-10-17T01:08:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.112+0000 7f06571ca700 1 -- 172.21.15.181:0/4044848943 wait complete. 2023-10-17T01:08:28.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 Processor -- start 2023-10-17T01:08:28.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 -- start start 2023-10-17T01:08:28.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0650100200 0x7f06500ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 0x7f065019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06501018e0 0x7f065019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065019e4c0 con 0x7f0650100200 2023-10-17T01:08:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065019e630 con 0x7f06501018e0 2023-10-17T01:08:28.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065019e7f0 con 0x7f0650100f20 2023-10-17T01:08:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 0x7f065019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 0x7f065019b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45120/0 (socket says 172.21.15.181:45120) 2023-10-17T01:08:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 -- 172.21.15.181:0/893978139 learned_addr learned my addr 172.21.15.181:0/893978139 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:08:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f0655767700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06501018e0 0x7f065019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:28.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 -- 172.21.15.181:0/893978139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06501018e0 msgr2=0x7f065019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06501018e0 0x7f065019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 -- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0650100200 msgr2=0x7f06500ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0650100200 0x7f06500ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 -- 172.21.15.181:0/893978139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648009d00 con 0x7f0650100f20 2023-10-17T01:08:28.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f0655767700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06501018e0 0x7f065019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:08:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064ffff700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 0x7f065019b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c990 tx=0x7f063c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:08:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c007b00 con 0x7f0650100f20 2023-10-17T01:08:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063c007c60 con 0x7f0650100f20 2023-10-17T01:08:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.113+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0650195580 con 0x7f0650100f20 2023-10-17T01:08:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.114+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c018660 con 0x7f0650100f20 2023-10-17T01:08:28.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.114+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650195b50 con 0x7f0650100f20 2023-10-17T01:08:28.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.115+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065010b4e0 con 0x7f0650100f20 2023-10-17T01:08:28.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.115+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 135476+0+0 (secure 0 0 0) 0x7f063c00f450 con 0x7f0650100f20 2023-10-17T01:08:28.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.116+0000 7f064dffb700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06400a0b10 0x7f06400a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:28.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.116+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f063c0ca8a0 con 0x7f0650100f20 2023-10-17T01:08:28.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.116+0000 7f0654f66700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06400a0b10 0x7f06400a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:28.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.117+0000 7f0654f66700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06400a0b10 0x7f06400a2fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0650068380 tx=0x7f0644006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:08:28.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.120+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f063c095b30 con 0x7f0650100f20 2023-10-17T01:08:28.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:28.316+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0650106d40 con 0x7f06400a0b10 2023-10-17T01:08:28.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:08:28.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:08:28.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] boot 2023-10-17T01:08:28.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: osdmap e13: 2 total, 2 up, 2 in 2023-10-17T01:08:28.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:28.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:28.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:28.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:28.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:28.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:28 smithi060 ceph-mon[134136]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:08:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:08:28.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] boot 2023-10-17T01:08:28.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: osdmap e13: 2 total, 2 up, 2 in 2023-10-17T01:08:28.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:28.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:28.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:28.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:28.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:28.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:28 smithi181 ceph-mon[134520]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:29.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:08:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:08:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: osd.1 [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] boot 2023-10-17T01:08:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: osdmap e13: 2 total, 2 up, 2 in 2023-10-17T01:08:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-17T01:08:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:28 smithi045 ceph-mon[98474]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-17T01:08:29.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:29 smithi181 ceph-mon[134520]: osdmap e14: 2 total, 2 up, 2 in 2023-10-17T01:08:30.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:29 smithi045 ceph-mon[98474]: osdmap e14: 2 total, 2 up, 2 in 2023-10-17T01:08:30.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:29 smithi060 ceph-mon[134136]: osdmap e14: 2 total, 2 up, 2 in 2023-10-17T01:08:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:30 smithi181 ceph-mon[134520]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:31.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:30 smithi045 ceph-mon[98474]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:31.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:30 smithi060 ceph-mon[134136]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:31 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/3608324413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]: dispatch 2023-10-17T01:08:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:31 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]: dispatch 2023-10-17T01:08:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:31 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]': finished 2023-10-17T01:08:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:31 smithi181 ceph-mon[134520]: osdmap e15: 3 total, 2 up, 3 in 2023-10-17T01:08:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:31 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/3883919713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:32.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:31 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/3608324413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]: dispatch 2023-10-17T01:08:32.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:31 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]: dispatch 2023-10-17T01:08:32.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:31 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]': finished 2023-10-17T01:08:32.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:31 smithi045 ceph-mon[98474]: osdmap e15: 3 total, 2 up, 3 in 2023-10-17T01:08:32.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:32.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:31 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/3883919713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:32.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:31 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/3608324413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]: dispatch 2023-10-17T01:08:32.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:31 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]: dispatch 2023-10-17T01:08:32.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:31 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b9dd7f-cf88-49c9-bd44-36a439272919"}]': finished 2023-10-17T01:08:32.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:31 smithi060 ceph-mon[134136]: osdmap e15: 3 total, 2 up, 3 in 2023-10-17T01:08:32.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:32.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:31 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/3883919713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:32 smithi181 ceph-mon[134520]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:32 smithi045 ceph-mon[98474]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:33.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:32 smithi060 ceph-mon[134136]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:35.752 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:35 smithi181 ceph-mon[134520]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:35 smithi045 ceph-mon[98474]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:35 smithi060 ceph-mon[134136]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:37.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:37 smithi181 ceph-mon[134520]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:37.687 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-17T01:08:37.688 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:37 smithi045 ceph-mon[98474]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-17T01:08:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:37 smithi060 ceph-mon[134136]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-17T01:08:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:38.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:38 smithi045 ceph-mon[98474]: Deploying daemon osd.2 on smithi181 2023-10-17T01:08:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:38 smithi060 ceph-mon[134136]: Deploying daemon osd.2 on smithi181 2023-10-17T01:08:38.886 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:38 smithi181 ceph-mon[134520]: Deploying daemon osd.2 on smithi181 2023-10-17T01:08:39.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:39 smithi181 ceph-mon[134520]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:39 smithi045 ceph-mon[98474]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:39 smithi060 ceph-mon[134136]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:41.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:41 smithi181 ceph-mon[134520]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:41.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:41.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:41.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:41 smithi045 ceph-mon[98474]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:41.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:41.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:41 smithi060 ceph-mon[134136]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:41.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:43 smithi045 ceph-mon[98474]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:43 smithi060 ceph-mon[134136]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:43.863 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:43 smithi181 ceph-mon[134520]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:43.880 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 2 on host 'smithi181' 2023-10-17T01:08:43.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.879+0000 7f064dffb700 1 -- 172.21.15.181:0/893978139 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0650106d40 con 0x7f06400a0b10 2023-10-17T01:08:43.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06400a0b10 msgr2=0x7f06400a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06400a0b10 0x7f06400a2fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0650068380 tx=0x7f0644006d20 comp rx=0 tx=0).stop 2023-10-17T01:08:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 msgr2=0x7f065019b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 0x7f065019b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c990 tx=0x7f063c00cd50 comp rx=0 tx=0).stop 2023-10-17T01:08:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 shutdown_connections 2023-10-17T01:08:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06400a0b10 0x7f06400a2fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06501018e0 0x7f065019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0650100200 0x7f06500ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 --2- 172.21.15.181:0/893978139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0650100f20 0x7f065019b690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 >> 172.21.15.181:0/893978139 conn(0x7f06500faa00 msgr2=0x7f0650104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:08:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 shutdown_connections 2023-10-17T01:08:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:08:43.881+0000 7f06571ca700 1 -- 172.21.15.181:0/893978139 wait complete. 2023-10-17T01:08:44.305 DEBUG:teuthology.orchestra.run.smithi181:osd.2> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.2.service 2023-10-17T01:08:44.308 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2023-10-17T01:08:44.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-17T01:08:44.341 INFO:journalctl@ceph.osd.2.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:08:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:44.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:44.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:44.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:44 smithi045 ceph-mon[98474]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:44.836 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.837 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.837 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:44.837 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:44.837 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.838 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.838 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:44.838 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:44.838 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:44 smithi181 ceph-mon[134520]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:08:45.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:45.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:45.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:45.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:45.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:45.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:45.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:08:45.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:45.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:44 smithi060 ceph-mon[134136]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:45.562 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2[140308]: 2023-10-17T01:08:45.337+0000 7f70f6e38640 -1 osd.2 0 log_to_monitors true 2023-10-17T01:08:46.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:45 smithi181 ceph-mon[134520]: from='osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-17T01:08:46.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:45 smithi181 ceph-mon[134520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-17T01:08:46.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:45 smithi045 ceph-mon[98474]: from='osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-17T01:08:46.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:45 smithi045 ceph-mon[98474]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-17T01:08:46.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:45 smithi060 ceph-mon[134136]: from='osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-17T01:08:46.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:45 smithi060 ceph-mon[134136]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-17T01:08:46.965 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-17T01:08:46.966 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-17T01:08:46.966 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-10-17T01:08:46.966 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-10-17T01:08:46.967 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0311622 s, 336 MB/s 2023-10-17T01:08:46.967 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-10-17T01:08:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-17T01:08:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: osdmap e16: 3 total, 2 up, 3 in 2023-10-17T01:08:47.144 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:47.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:08:47.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:08:47.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:47.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:47.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:47.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:47.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.157 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2[140308]: 2023-10-17T01:08:46.897+0000 7f70ec4af700 -1 osd.2 0 waiting for initial osdmap 2023-10-17T01:08:47.157 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2[140308]: 2023-10-17T01:08:46.903+0000 7f70e62a0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:08:47.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-17T01:08:47.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: osdmap e16: 3 total, 2 up, 3 in 2023-10-17T01:08:47.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:47.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:08:47.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:08:47.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:47.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:47.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:47.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:47.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-17T01:08:47.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: osdmap e16: 3 total, 2 up, 3 in 2023-10-17T01:08:47.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:47.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:08:47.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:08:47.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-17T01:08:47.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:08:47.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:47.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:08:47.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:47.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2023-10-17T01:08:48.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:47 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi181 2023-10-17T01:08:48.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:47 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi181 to 17168M 2023-10-17T01:08:48.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:47 smithi181 ceph-mon[134520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:08:48.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:47 smithi181 ceph-mon[134520]: osdmap e17: 3 total, 2 up, 3 in 2023-10-17T01:08:48.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:48.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:08:48.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:47 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi181 2023-10-17T01:08:48.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:47 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi181 to 17168M 2023-10-17T01:08:48.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:47 smithi045 ceph-mon[98474]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:08:48.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:47 smithi045 ceph-mon[98474]: osdmap e17: 3 total, 2 up, 3 in 2023-10-17T01:08:48.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:48.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:48.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:47 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi181 2023-10-17T01:08:48.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:47 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi181 to 17168M 2023-10-17T01:08:48.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:47 smithi060 ceph-mon[134136]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:08:48.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:47 smithi060 ceph-mon[134136]: osdmap e17: 3 total, 2 up, 3 in 2023-10-17T01:08:48.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:48.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:49.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:48 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:08:49.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:48 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:08:49.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:48 smithi045 ceph-mon[98474]: osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] boot 2023-10-17T01:08:49.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:48 smithi045 ceph-mon[98474]: osdmap e18: 3 total, 3 up, 3 in 2023-10-17T01:08:49.021 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:49.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:48 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:08:49.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:48 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:08:49.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:48 smithi060 ceph-mon[134136]: osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] boot 2023-10-17T01:08:49.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:48 smithi060 ceph-mon[134136]: osdmap e18: 3 total, 3 up, 3 in 2023-10-17T01:08:49.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:49.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:48 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:08:49.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:48 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:08:49.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:48 smithi181 ceph-mon[134520]: osd.2 [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] boot 2023-10-17T01:08:49.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:48 smithi181 ceph-mon[134520]: osdmap e18: 3 total, 3 up, 3 in 2023-10-17T01:08:49.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-17T01:08:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 -- 172.21.15.45:0/3696650594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08101fe0 msgr2=0x7f6c08102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/3696650594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08101fe0 0x7f6c08102440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0099c0 tx=0x7f6bfc009cd0 comp rx=0 tx=0).stop 2023-10-17T01:08:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 -- 172.21.15.45:0/3696650594 shutdown_connections 2023-10-17T01:08:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/3696650594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c08102980 0x7f6c08109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/3696650594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c08100de0 0x7f6c08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/3696650594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08101fe0 0x7f6c08102440 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 -- 172.21.15.45:0/3696650594 >> 172.21.15.45:0/3696650594 conn(0x7f6c080fc340 msgr2=0x7f6c080fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:08:49.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 -- 172.21.15.45:0/3696650594 shutdown_connections 2023-10-17T01:08:49.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.668+0000 7f6c0d72d700 1 -- 172.21.15.45:0/3696650594 wait complete. 2023-10-17T01:08:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.669+0000 7f6c0d72d700 1 Processor -- start 2023-10-17T01:08:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 -- start start 2023-10-17T01:08:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c08100de0 0x7f6c08196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c08101fe0 0x7f6c08197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 0x7f6c0819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:49.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0819e6d0 con 0x7f6c08102980 2023-10-17T01:08:49.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0819e810 con 0x7f6c08100de0 2023-10-17T01:08:49.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c0d72d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0819e9a0 con 0x7f6c08101fe0 2023-10-17T01:08:49.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 0x7f6c0819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:49.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 0x7f6c0819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40038/0 (socket says 172.21.15.45:40038) 2023-10-17T01:08:49.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c077fe700 1 -- 172.21.15.45:0/165331309 learned_addr learned my addr 172.21.15.45:0/165331309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:08:49.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c06ffd700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c08100de0 0x7f6c08196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:49.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.670+0000 7f6c067fc700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c08101fe0 0x7f6c08197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:49.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c077fe700 1 -- 172.21.15.45:0/165331309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c08101fe0 msgr2=0x7f6c08197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c077fe700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c08101fe0 0x7f6c08197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c077fe700 1 -- 172.21.15.45:0/165331309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c08100de0 msgr2=0x7f6c08196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:08:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c077fe700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c08100de0 0x7f6c08196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:08:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c077fe700 1 -- 172.21.15.45:0/165331309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0095f0 con 0x7f6c08102980 2023-10-17T01:08:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c077fe700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 0x7f6c0819e100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000ef60 tx=0x7f6c0000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:08:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00018070 con 0x7f6c08102980 2023-10-17T01:08:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c0000f040 con 0x7f6c08102980 2023-10-17T01:08:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000ccc0 con 0x7f6c08102980 2023-10-17T01:08:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0819ec20 con 0x7f6c08102980 2023-10-17T01:08:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.671+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0819f0d0 con 0x7f6c08102980 2023-10-17T01:08:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.672+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0810c630 con 0x7f6c08102980 2023-10-17T01:08:49.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.675+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) v1 ==== 135476+0+0 (secure 0 0 0) 0x7f6c00003a00 con 0x7f6c08102980 2023-10-17T01:08:49.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.676+0000 7f6beffff700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6bf00a0b10 0x7f6bf00a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:08:49.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.676+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f6c000caff0 con 0x7f6c08102980 2023-10-17T01:08:49.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.676+0000 7f6c06ffd700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6bf00a0b10 0x7f6bf00a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:08:49.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.677+0000 7f6c06ffd700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6bf00a0b10 0x7f6bf00a2fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c08101e40 tx=0x7f6bf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:08:49.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.679+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f6c00095ed0 con 0x7f6c08102980 2023-10-17T01:08:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:49.899+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c08061190 con 0x7f6bf00a0b10 2023-10-17T01:08:50.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:50.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:08:50.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-17T01:08:50.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: osdmap e19: 3 total, 3 up, 3 in 2023-10-17T01:08:50.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:08:50.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:50.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:50.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:50.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:50.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:08:50.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-17T01:08:50.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: osdmap e19: 3 total, 3 up, 3 in 2023-10-17T01:08:50.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:08:50.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:50.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:50.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:50.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:08:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-17T01:08:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: osdmap e19: 3 total, 3 up, 3 in 2023-10-17T01:08:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:08:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:08:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:08:50.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:08:51.293 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122445]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-17T01:08:51.294 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:51.294 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122445]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:51.295 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122445]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:51.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:51 smithi045 ceph-mon[98474]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:51.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-17T01:08:51.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:51 smithi045 ceph-mon[98474]: osdmap e20: 3 total, 3 up, 3 in 2023-10-17T01:08:51.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:51 smithi045 ceph-mon[98474]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:51.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:51 smithi060 ceph-mon[134136]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:51.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-17T01:08:51.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:51 smithi060 ceph-mon[134136]: osdmap e20: 3 total, 3 up, 3 in 2023-10-17T01:08:51.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:51 smithi060 ceph-mon[134136]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:51.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:51 smithi181 ceph-mon[134520]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:08:51.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-17T01:08:51.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:51 smithi181 ceph-mon[134520]: osdmap e20: 3 total, 3 up, 3 in 2023-10-17T01:08:51.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:51 smithi181 ceph-mon[134520]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:52.051 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122471]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-17T01:08:52.051 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122471]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:52.051 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122471]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:52.052 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:08:51 smithi045 sudo[122471]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:52.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:52 smithi045 ceph-mon[98474]: osdmap e21: 3 total, 3 up, 3 in 2023-10-17T01:08:52.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:52 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:52.578 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144733]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-17T01:08:52.578 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:52.578 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144733]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:52.579 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144733]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:52.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:52 smithi060 ceph-mon[134136]: osdmap e21: 3 total, 3 up, 3 in 2023-10-17T01:08:52.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:52 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:52.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:52 smithi181 ceph-mon[134520]: osdmap e21: 3 total, 3 up, 3 in 2023-10-17T01:08:52.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:52 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:53.078 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144736]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-17T01:08:53.078 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144736]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:53.078 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144736]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:53.079 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:08:52 smithi060 sudo[144736]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:53.218 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:52 smithi181 sudo[144952]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-17T01:08:53.219 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:52 smithi181 sudo[144952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:53.219 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:52 smithi181 sudo[144952]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:53.219 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:52 smithi181 sudo[144952]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:53.657 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:53 smithi181 sudo[144955]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-17T01:08:53.657 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:53 smithi181 sudo[144955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:53.658 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:53 smithi181 sudo[144955]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:53.658 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:08:53 smithi181 sudo[144955]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:54.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:53 smithi045 sudo[122593]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-17T01:08:54.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:53 smithi045 sudo[122593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:54.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:53 smithi045 sudo[122593]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:54.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:53 smithi045 sudo[122593]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:08:54.352+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f6c00003cb0 con 0x7f6c08102980 2023-10-17T01:08:54.601 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:54.601 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:54.601 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2911785606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "806e62ab-4fa6-4c3f-80b7-eb25f498baab"}]: dispatch 2023-10-17T01:08:54.601 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2911785606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "806e62ab-4fa6-4c3f-80b7-eb25f498baab"}]': finished 2023-10-17T01:08:54.602 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: osdmap e22: 4 total, 3 up, 4 in 2023-10-17T01:08:54.602 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:08:54.602 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:54.602 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 ceph-mon[134520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:54.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144958]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-17T01:08:54.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:54.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144958]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:54.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144958]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:54.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 sudo[122645]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-17T01:08:54.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 sudo[122645]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:54.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 sudo[122645]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:54.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 sudo[122645]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:54.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:54.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:54.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2911785606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "806e62ab-4fa6-4c3f-80b7-eb25f498baab"}]: dispatch 2023-10-17T01:08:54.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2911785606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "806e62ab-4fa6-4c3f-80b7-eb25f498baab"}]': finished 2023-10-17T01:08:54.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: osdmap e22: 4 total, 3 up, 4 in 2023-10-17T01:08:54.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:08:54.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:54.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:54 smithi045 ceph-mon[98474]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2911785606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "806e62ab-4fa6-4c3f-80b7-eb25f498baab"}]: dispatch 2023-10-17T01:08:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2911785606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "806e62ab-4fa6-4c3f-80b7-eb25f498baab"}]': finished 2023-10-17T01:08:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: osdmap e22: 4 total, 3 up, 4 in 2023-10-17T01:08:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:08:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:08:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:54 smithi060 ceph-mon[134136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:54.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144961]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-17T01:08:54.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144961]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:54.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144961]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:54.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:54 smithi181 sudo[144961]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:55.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144739]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-17T01:08:55.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144739]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:55.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144739]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:55.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144739]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3587169707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:56.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:56.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-17T01:08:56.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:56.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:56.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144742]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-17T01:08:56.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-17T01:08:56.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144742]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-17T01:08:56.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 sudo[144742]: pam_unix(sudo:session): session closed for user root 2023-10-17T01:08:56.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3587169707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:56.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:56.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:56.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-17T01:08:56.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:56.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:56.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3587169707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:08:56.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:56.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:56.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-17T01:08:56.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:56.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:56.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:56.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-17T01:08:56.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-17T01:08:56.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-17T01:08:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:56 smithi045 ceph-mon[98474]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:56 smithi045 ceph-mon[98474]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:56 smithi045 ceph-mon[98474]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:56 smithi060 ceph-mon[134136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:56 smithi060 ceph-mon[134136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:56 smithi060 ceph-mon[134136]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:57.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:56 smithi181 ceph-mon[134520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-17T01:08:57.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:56 smithi181 ceph-mon[134520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-17T01:08:57.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:56 smithi181 ceph-mon[134520]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:59.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:08:59 smithi045 ceph-mon[98474]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:08:59 smithi060 ceph-mon[134136]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:08:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:08:59 smithi181 ceph-mon[134520]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:09:01.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:01 smithi045 ceph-mon[98474]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-17T01:09:01.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-17T01:09:01.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:01 smithi060 ceph-mon[134136]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-17T01:09:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-17T01:09:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:01 smithi181 ceph-mon[134520]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-17T01:09:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-17T01:09:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:02 smithi045 ceph-mon[98474]: Deploying daemon osd.3 on smithi045 2023-10-17T01:09:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:02 smithi060 ceph-mon[134136]: Deploying daemon osd.3 on smithi045 2023-10-17T01:09:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:02 smithi181 ceph-mon[134520]: Deploying daemon osd.3 on smithi045 2023-10-17T01:09:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:03 smithi045 ceph-mon[98474]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-17T01:09:03.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:03 smithi060 ceph-mon[134136]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-17T01:09:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:03 smithi181 ceph-mon[134520]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-17T01:09:05.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:05 smithi045 ceph-mon[98474]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-17T01:09:05.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:05 smithi181 ceph-mon[134520]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-17T01:09:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:05 smithi060 ceph-mon[134136]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-17T01:09:07.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:07 smithi045 ceph-mon[98474]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:07.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:07.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:07.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:07 smithi060 ceph-mon[134136]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:07 smithi181 ceph-mon[134520]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:09 smithi045 ceph-mon[98474]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:09 smithi060 ceph-mon[134136]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:09 smithi181 ceph-mon[134520]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.462+0000 7f6beffff700 1 -- 172.21.15.45:0/165331309 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c08061190 con 0x7f6bf00a0b10 2023-10-17T01:09:10.464 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2023-10-17T01:09:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6bf00a0b10 msgr2=0x7f6bf00a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6bf00a0b10 0x7f6bf00a2fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c08101e40 tx=0x7f6bf8009260 comp rx=0 tx=0).stop 2023-10-17T01:09:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 msgr2=0x7f6c0819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 0x7f6c0819e100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000ef60 tx=0x7f6c0000ef90 comp rx=0 tx=0).stop 2023-10-17T01:09:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 shutdown_connections 2023-10-17T01:09:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6bf00a0b10 0x7f6bf00a2fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c08102980 0x7f6c0819e100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c08100de0 0x7f6c08196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 --2- 172.21.15.45:0/165331309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c08101fe0 0x7f6c08197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 >> 172.21.15.45:0/165331309 conn(0x7f6c080fc340 msgr2=0x7f6c080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:09:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 shutdown_connections 2023-10-17T01:09:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:10.464+0000 7f6c0d72d700 1 -- 172.21.15.45:0/165331309 wait complete. 2023-10-17T01:09:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:10.921 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.3.service 2023-10-17T01:09:10.926 INFO:tasks.cephadm:Deploying osd.4 on smithi060 with /dev/vg_nvme/lv_3... 2023-10-17T01:09:10.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-17T01:09:10.959 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.960 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.960 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:10.960 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:10.960 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.960 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-17T01:09:10.961 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.961 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:10.961 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:10.999 INFO:journalctl@ceph.osd.3.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:09:11.499 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:09:11.868 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:11 smithi060 ceph-mon[134136]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-17T01:09:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:11 smithi181 ceph-mon[134520]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-17T01:09:11.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:11 smithi045 ceph-mon[98474]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-17T01:09:12.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:12 smithi181 ceph-mon[134520]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-17T01:09:12.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:12 smithi181 ceph-mon[134520]: osdmap e23: 4 total, 3 up, 4 in 2023-10-17T01:09:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:12 smithi181 ceph-mon[134520]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:09:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:12 smithi181 ceph-mon[134520]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:09:12.995 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-17T01:09:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-17T01:09:12.996 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-10-17T01:09:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-10-17T01:09:12.996 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00879879 s, 1.2 GB/s 2023-10-17T01:09:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-10-17T01:09:13.031 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:12 smithi060 ceph-mon[134136]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-17T01:09:13.031 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:12 smithi060 ceph-mon[134136]: osdmap e23: 4 total, 3 up, 4 in 2023-10-17T01:09:13.032 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:13.032 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:12 smithi060 ceph-mon[134136]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:09:13.032 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:12 smithi060 ceph-mon[134136]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:09:13.051 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:09:12.623+0000 7fd3340e0700 -1 osd.3 0 waiting for initial osdmap 2023-10-17T01:09:13.051 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:09:12.630+0000 7fd32f6d4700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:09:13.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-mon[98474]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-17T01:09:13.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-mon[98474]: osdmap e23: 4 total, 3 up, 4 in 2023-10-17T01:09:13.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:13.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-mon[98474]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:09:13.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:12 smithi045 ceph-mon[98474]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-17T01:09:13.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2023-10-17T01:09:13.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:13 smithi045 ceph-mon[98474]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:09:13.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:13 smithi045 ceph-mon[98474]: osdmap e24: 4 total, 3 up, 4 in 2023-10-17T01:09:13.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:13 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:13.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:13 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:13 smithi181 ceph-mon[134520]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:09:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:13 smithi181 ceph-mon[134520]: osdmap e24: 4 total, 3 up, 4 in 2023-10-17T01:09:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:13 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:13 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:13.957 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:09:14.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:13 smithi060 ceph-mon[134136]: from='osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:09:14.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:13 smithi060 ceph-mon[134136]: osdmap e24: 4 total, 3 up, 4 in 2023-10-17T01:09:14.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:13 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:14.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:13 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:09:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:09:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] boot 2023-10-17T01:09:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: osdmap e25: 4 total, 4 up, 4 in 2023-10-17T01:09:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: osdmap e26: 4 total, 4 up, 4 in 2023-10-17T01:09:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi045 2023-10-17T01:09:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi045 to 8638M 2023-10-17T01:09:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:09:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:09:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] boot 2023-10-17T01:09:14.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: osdmap e25: 4 total, 4 up, 4 in 2023-10-17T01:09:14.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:14.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: osdmap e26: 4 total, 4 up, 4 in 2023-10-17T01:09:14.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:14.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi045 2023-10-17T01:09:14.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:14.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:14.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi045 to 8638M 2023-10-17T01:09:14.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:14.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:14.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:14.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:15.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:09:15.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:09:15.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: osd.3 [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] boot 2023-10-17T01:09:15.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: osdmap e25: 4 total, 4 up, 4 in 2023-10-17T01:09:15.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-17T01:09:15.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: osdmap e26: 4 total, 4 up, 4 in 2023-10-17T01:09:15.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:15.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi045 2023-10-17T01:09:15.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:15.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:15.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:15.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:15.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi045 to 8638M 2023-10-17T01:09:15.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:15.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:15.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:15.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 -- 172.21.15.60:0/541181867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0101880 msgr2=0x7fd1c0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 --2- 172.21.15.60:0/541181867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0101880 0x7fd1c0108130 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800b0d0 tx=0x7fd1b800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:09:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 -- 172.21.15.60:0/541181867 shutdown_connections 2023-10-17T01:09:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 --2- 172.21.15.60:0/541181867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0101880 0x7fd1c0108130 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 --2- 172.21.15.60:0/541181867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c00ffce0 0x7fd1c0100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 --2- 172.21.15.60:0/541181867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 -- 172.21.15.60:0/541181867 >> 172.21.15.60:0/541181867 conn(0x7fd1c00fb260 msgr2=0x7fd1c00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:09:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 -- 172.21.15.60:0/541181867 shutdown_connections 2023-10-17T01:09:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.268+0000 7fd1c61af700 1 -- 172.21.15.60:0/541181867 wait complete. 2023-10-17T01:09:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 Processor -- start 2023-10-17T01:09:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 -- start start 2023-10-17T01:09:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c00ffce0 0x7fd1c0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c0101880 0x7fd1c019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c019d660 con 0x7fd1c0101880 2023-10-17T01:09:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c019d7a0 con 0x7fd1c0100ee0 2023-10-17T01:09:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1c61af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c019d930 con 0x7fd1c00ffce0 2023-10-17T01:09:15.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:15.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37434/0 (socket says 172.21.15.60:37434) 2023-10-17T01:09:15.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1beffd700 1 -- 172.21.15.60:0/3059286737 learned_addr learned my addr 172.21.15.60:0/3059286737 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-10-17T01:09:15.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.269+0000 7fd1bf7fe700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c00ffce0 0x7fd1c0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:15.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1beffd700 1 -- 172.21.15.60:0/3059286737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c00ffce0 msgr2=0x7fd1c0195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:15.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1beffd700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c00ffce0 0x7fd1c0195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1beffd700 1 -- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c0101880 msgr2=0x7fd1c019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1beffd700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c0101880 0x7fd1c019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1beffd700 1 -- 172.21.15.60:0/3059286737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009d00 con 0x7fd1c0100ee0 2023-10-17T01:09:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1beffd700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0196010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b530 tx=0x7fd1b400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:09:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4011830 con 0x7fd1c0100ee0 2023-10-17T01:09:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1b4011e50 con 0x7fd1c0100ee0 2023-10-17T01:09:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c019dc10 con 0x7fd1c0100ee0 2023-10-17T01:09:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400f2e0 con 0x7fd1c0100ee0 2023-10-17T01:09:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.270+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c019e190 con 0x7fd1c0100ee0 2023-10-17T01:09:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.271+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c010b560 con 0x7fd1c0100ee0 2023-10-17T01:09:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.272+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd1b4010380 con 0x7fd1c0100ee0 2023-10-17T01:09:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.272+0000 7fd1bcff9700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd1a80a0b20 0x7fd1a80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.273+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fd1b40caad0 con 0x7fd1c0100ee0 2023-10-17T01:09:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.273+0000 7fd1bf7fe700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd1a80a0b20 0x7fd1a80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.274+0000 7fd1bf7fe700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd1a80a0b20 0x7fd1a80a2fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0100d40 tx=0x7fd1b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:09:15.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.277+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd1b4095660 con 0x7fd1c0100ee0 2023-10-17T01:09:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:15.490+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1c0061190 con 0x7fd1a80a0b20 2023-10-17T01:09:16.576 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:16 smithi060 ceph-mon[134136]: osdmap e27: 4 total, 4 up, 4 in 2023-10-17T01:09:16.576 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:16.576 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:16.576 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:16.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:16 smithi181 ceph-mon[134520]: osdmap e27: 4 total, 4 up, 4 in 2023-10-17T01:09:16.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:16.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:16.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:16.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:16 smithi045 ceph-mon[98474]: osdmap e27: 4 total, 4 up, 4 in 2023-10-17T01:09:16.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:17.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:17 smithi181 ceph-mon[134520]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:17.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:17 smithi181 ceph-mon[134520]: osdmap e28: 4 total, 4 up, 4 in 2023-10-17T01:09:17.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:17 smithi181 ceph-mon[134520]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:17 smithi045 ceph-mon[98474]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:17 smithi045 ceph-mon[98474]: osdmap e28: 4 total, 4 up, 4 in 2023-10-17T01:09:17.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:17 smithi045 ceph-mon[98474]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:17 smithi060 ceph-mon[134136]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:17 smithi060 ceph-mon[134136]: osdmap e28: 4 total, 4 up, 4 in 2023-10-17T01:09:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:17 smithi060 ceph-mon[134136]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:18.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:18 smithi181 ceph-mon[134520]: osdmap e29: 4 total, 4 up, 4 in 2023-10-17T01:09:18.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:18 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/3347886533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]: dispatch 2023-10-17T01:09:18.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:18 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]: dispatch 2023-10-17T01:09:18.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:18 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]': finished 2023-10-17T01:09:18.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:18 smithi181 ceph-mon[134520]: osdmap e30: 5 total, 4 up, 5 in 2023-10-17T01:09:18.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:18 smithi045 ceph-mon[98474]: osdmap e29: 4 total, 4 up, 4 in 2023-10-17T01:09:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:18 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/3347886533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]: dispatch 2023-10-17T01:09:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:18 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]: dispatch 2023-10-17T01:09:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:18 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]': finished 2023-10-17T01:09:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:18 smithi045 ceph-mon[98474]: osdmap e30: 5 total, 4 up, 5 in 2023-10-17T01:09:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:18.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:18 smithi060 ceph-mon[134136]: osdmap e29: 4 total, 4 up, 4 in 2023-10-17T01:09:18.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:18 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/3347886533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]: dispatch 2023-10-17T01:09:18.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:18 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]: dispatch 2023-10-17T01:09:18.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:18 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3232fc8e-ae84-448d-9665-adc7f64f6c99"}]': finished 2023-10-17T01:09:18.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:18 smithi060 ceph-mon[134136]: osdmap e30: 5 total, 4 up, 5 in 2023-10-17T01:09:18.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:19.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:19 smithi181 ceph-mon[134520]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:19.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:19 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/4086446910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:09:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:19 smithi045 ceph-mon[98474]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:19 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/4086446910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:09:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:19 smithi060 ceph-mon[134136]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:19 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/4086446910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:09:21.717 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:21 smithi060 ceph-mon[134136]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-10-17T01:09:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:21 smithi045 ceph-mon[98474]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-10-17T01:09:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:21 smithi181 ceph-mon[134520]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-10-17T01:09:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:23 smithi045 ceph-mon[98474]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:09:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:23 smithi060 ceph-mon[134136]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:09:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:23 smithi181 ceph-mon[134520]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:09:24.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-17T01:09:24.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-17T01:09:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-17T01:09:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:25.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:25 smithi060 ceph-mon[134136]: Deploying daemon osd.4 on smithi060 2023-10-17T01:09:25.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:25 smithi060 ceph-mon[134136]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:09:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:25 smithi045 ceph-mon[98474]: Deploying daemon osd.4 on smithi060 2023-10-17T01:09:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:25 smithi045 ceph-mon[98474]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:09:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:25 smithi181 ceph-mon[134520]: Deploying daemon osd.4 on smithi060 2023-10-17T01:09:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:25 smithi181 ceph-mon[134520]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:09:27.774 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:27 smithi060 ceph-mon[134136]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-17T01:09:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:27 smithi045 ceph-mon[98474]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-17T01:09:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:27 smithi181 ceph-mon[134520]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-17T01:09:29.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:29 smithi060 ceph-mon[134136]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-10-17T01:09:29.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:29.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:29.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:29 smithi045 ceph-mon[98474]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-10-17T01:09:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:29 smithi181 ceph-mon[134520]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-10-17T01:09:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:31.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.004+0000 7fd1bcff9700 1 -- 172.21.15.60:0/3059286737 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd1c0061190 con 0x7fd1a80a0b20 2023-10-17T01:09:31.006 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 4 on host 'smithi060' 2023-10-17T01:09:31.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd1a80a0b20 msgr2=0x7fd1a80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:31.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd1a80a0b20 0x7fd1a80a2fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0100d40 tx=0x7fd1b000b410 comp rx=0 tx=0).stop 2023-10-17T01:09:31.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 msgr2=0x7fd1c0196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:31.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0196010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b530 tx=0x7fd1b400b840 comp rx=0 tx=0).stop 2023-10-17T01:09:31.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 shutdown_connections 2023-10-17T01:09:31.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd1a80a0b20 0x7fd1a80a2fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:31.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c0101880 0x7fd1c019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:31.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c00ffce0 0x7fd1c0195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:31.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 --2- 172.21.15.60:0/3059286737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c0100ee0 0x7fd1c0196010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:31.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 >> 172.21.15.60:0/3059286737 conn(0x7fd1c00fb260 msgr2=0x7fd1c00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:09:31.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 shutdown_connections 2023-10-17T01:09:31.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:09:31.006+0000 7fd1c61af700 1 -- 172.21.15.60:0/3059286737 wait complete. 2023-10-17T01:09:31.456 DEBUG:teuthology.orchestra.run.smithi060:osd.4> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.4.service 2023-10-17T01:09:31.458 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_3... 2023-10-17T01:09:31.458 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-17T01:09:31.488 INFO:journalctl@ceph.osd.4.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:09:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:09:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:09:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:31.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:31.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:31.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:09:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:31.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:32.021 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:09:32.532 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:32 smithi181 ceph-mon[134520]: from='osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-17T01:09:32.533 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:32 smithi181 ceph-mon[134520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-17T01:09:32.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:32 smithi060 ceph-mon[134136]: from='osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-17T01:09:32.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:32 smithi060 ceph-mon[134136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-17T01:09:32.579 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:09:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:09:32.438+0000 7f459966e640 -1 osd.4 0 log_to_monitors true 2023-10-17T01:09:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:32 smithi045 ceph-mon[98474]: from='osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-17T01:09:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:32 smithi045 ceph-mon[98474]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-17T01:09:33.487 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-17T01:09:33.487 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-17T01:09:33.487 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-10-17T01:09:33.488 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-10-17T01:09:33.488 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128456 s, 816 MB/s 2023-10-17T01:09:33.488 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-10-17T01:09:33.723 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:09:33.468+0000 7f458ece5700 -1 osd.4 0 waiting for initial osdmap 2023-10-17T01:09:33.724 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:09:33.477+0000 7f4587ad4700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:09:33.724 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-mon[134136]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:33.724 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-mon[134136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-17T01:09:33.724 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-mon[134136]: osdmap e31: 5 total, 4 up, 5 in 2023-10-17T01:09:33.725 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:33.725 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-mon[134136]: from='osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:09:33.725 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:33 smithi060 ceph-mon[134136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:09:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:33 smithi045 ceph-mon[98474]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:33 smithi045 ceph-mon[98474]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-17T01:09:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:33 smithi045 ceph-mon[98474]: osdmap e31: 5 total, 4 up, 5 in 2023-10-17T01:09:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:33 smithi045 ceph-mon[98474]: from='osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:09:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:33 smithi045 ceph-mon[98474]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:09:33.870 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:33 smithi181 ceph-mon[134520]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:33.871 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:33 smithi181 ceph-mon[134520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-17T01:09:33.871 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:33 smithi181 ceph-mon[134520]: osdmap e31: 5 total, 4 up, 5 in 2023-10-17T01:09:33.871 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:33.872 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:33 smithi181 ceph-mon[134520]: from='osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:09:33.872 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:33 smithi181 ceph-mon[134520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:09:33.873 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2023-10-17T01:09:34.424 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:09:34.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:09:34.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: osdmap e32: 5 total, 4 up, 5 in 2023-10-17T01:09:34.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:34.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:34.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:34.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:34.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:34.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:34.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:34 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:09:34.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: osdmap e32: 5 total, 4 up, 5 in 2023-10-17T01:09:34.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:34.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:34.808 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.808 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.808 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:34.808 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:34.809 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.809 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:34.809 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:34.809 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:34 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:09:34.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: osdmap e32: 5 total, 4 up, 5 in 2023-10-17T01:09:34.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:34.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:34.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:34.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:34.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:34.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:34.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:34.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:34 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:35.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 -- 172.21.15.181:0/1284252686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780100ee0 msgr2=0x7fd780101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:35.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 --2- 172.21.15.181:0/1284252686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780100ee0 0x7fd780101340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd770009990 tx=0x7fd770009ca0 comp rx=0 tx=0).stop 2023-10-17T01:09:35.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 -- 172.21.15.181:0/1284252686 shutdown_connections 2023-10-17T01:09:35.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 --2- 172.21.15.181:0/1284252686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd780101880 0x7fd780108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:35.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 --2- 172.21.15.181:0/1284252686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7800ffce0 0x7fd780100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:35.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 --2- 172.21.15.181:0/1284252686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780100ee0 0x7fd780101340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:35.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 -- 172.21.15.181:0/1284252686 >> 172.21.15.181:0/1284252686 conn(0x7fd7800fb260 msgr2=0x7fd7800fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:09:35.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 -- 172.21.15.181:0/1284252686 shutdown_connections 2023-10-17T01:09:35.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.587+0000 7fd784bbd700 1 -- 172.21.15.181:0/1284252686 wait complete. 2023-10-17T01:09:35.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 Processor -- start 2023-10-17T01:09:35.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 -- start start 2023-10-17T01:09:35.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7800ffce0 0x7fd780069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:35.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd780100ee0 0x7fd78019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:35.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 0x7fd7801a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:35.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801a0800 con 0x7fd780100ee0 2023-10-17T01:09:35.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801a0970 con 0x7fd7800ffce0 2023-10-17T01:09:35.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.588+0000 7fd784bbd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801a0b30 con 0x7fd780101880 2023-10-17T01:09:35.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 0x7fd7801a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:35.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7800ffce0 0x7fd780069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:35.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 0x7fd7801a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45202/0 (socket says 172.21.15.181:45202) 2023-10-17T01:09:35.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 -- 172.21.15.181:0/2686044650 learned_addr learned my addr 172.21.15.181:0/2686044650 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:09:35.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77dd9b700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd780100ee0 0x7fd78019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:35.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 -- 172.21.15.181:0/2686044650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7800ffce0 msgr2=0x7fd780069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:35.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7800ffce0 0x7fd780069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:35.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 -- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd780100ee0 msgr2=0x7fd78019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:35.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd780100ee0 0x7fd78019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:35.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 -- 172.21.15.181:0/2686044650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700095f0 con 0x7fd780101880 2023-10-17T01:09:35.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd77ed9d700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 0x7fd7801a0150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd778007ee0 tx=0x7fd77800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:09:35.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778017070 con 0x7fd780101880 2023-10-17T01:09:35.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7801a0e10 con 0x7fd780101880 2023-10-17T01:09:35.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd77800f040 con 0x7fd780101880 2023-10-17T01:09:35.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778005580 con 0x7fd780101880 2023-10-17T01:09:35.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.589+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7801a1290 con 0x7fd780101880 2023-10-17T01:09:35.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.590+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7801072d0 con 0x7fd780101880 2023-10-17T01:09:35.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.591+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd77800db20 con 0x7fd780101880 2023-10-17T01:09:35.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.592+0000 7fd76f7fe700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd7680a0b20 0x7fd7680a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:35.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.592+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fd7780ca860 con 0x7fd780101880 2023-10-17T01:09:35.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.592+0000 7fd77e59c700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd7680a0b20 0x7fd7680a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:35.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.593+0000 7fd77e59c700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd7680a0b20 0x7fd7680a2fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd780100d40 tx=0x7fd77400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:09:35.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.596+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd778095260 con 0x7fd780101880 2023-10-17T01:09:35.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:09:35.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:09:35.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi060 2023-10-17T01:09:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi060 to 8586M 2023-10-17T01:09:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] boot 2023-10-17T01:09:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: osdmap e33: 5 total, 5 up, 5 in 2023-10-17T01:09:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:35 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:35.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:35.792+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd780061190 con 0x7fd7680a0b20 2023-10-17T01:09:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:09:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:09:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi060 2023-10-17T01:09:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi060 to 8586M 2023-10-17T01:09:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] boot 2023-10-17T01:09:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: osdmap e33: 5 total, 5 up, 5 in 2023-10-17T01:09:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:35 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:35.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:09:35.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:09:35.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi060 2023-10-17T01:09:35.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi060 to 8586M 2023-10-17T01:09:35.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-17T01:09:35.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: osd.4 [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] boot 2023-10-17T01:09:35.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: osdmap e33: 5 total, 5 up, 5 in 2023-10-17T01:09:35.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:35 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-17T01:09:36.766 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:36 smithi181 ceph-mon[134520]: osdmap e34: 5 total, 5 up, 5 in 2023-10-17T01:09:36.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:36.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:36.767 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:36 smithi045 ceph-mon[98474]: osdmap e34: 5 total, 5 up, 5 in 2023-10-17T01:09:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:36.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:36.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:36 smithi060 ceph-mon[134136]: osdmap e34: 5 total, 5 up, 5 in 2023-10-17T01:09:36.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:36.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:36.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:37.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:37 smithi181 ceph-mon[134520]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:37.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:37 smithi181 ceph-mon[134520]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:37 smithi045 ceph-mon[98474]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:37 smithi045 ceph-mon[98474]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:37 smithi060 ceph-mon[134136]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:37 smithi060 ceph-mon[134136]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:38.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:38 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/574123090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]: dispatch 2023-10-17T01:09:38.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:38 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]: dispatch 2023-10-17T01:09:38.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:38 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]': finished 2023-10-17T01:09:38.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:38 smithi045 ceph-mon[98474]: osdmap e35: 6 total, 5 up, 6 in 2023-10-17T01:09:38.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:38 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/574123090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]: dispatch 2023-10-17T01:09:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:38 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]: dispatch 2023-10-17T01:09:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:38 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]': finished 2023-10-17T01:09:38.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:38 smithi060 ceph-mon[134136]: osdmap e35: 6 total, 5 up, 6 in 2023-10-17T01:09:38.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:38.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:38 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/574123090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]: dispatch 2023-10-17T01:09:38.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:38 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]: dispatch 2023-10-17T01:09:38.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:38 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28b10aa9-ecab-4539-9fa5-e753609c3bb2"}]': finished 2023-10-17T01:09:38.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:38 smithi181 ceph-mon[134520]: osdmap e35: 6 total, 5 up, 6 in 2023-10-17T01:09:38.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:39 smithi045 ceph-mon[98474]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:39 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/2209034335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:09:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:39 smithi060 ceph-mon[134136]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:39 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/2209034335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:09:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:39 smithi181 ceph-mon[134520]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:39 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/2209034335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:09:40.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:40 smithi045 ceph-mon[98474]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:40.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:40 smithi060 ceph-mon[134136]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:40.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:40 smithi181 ceph-mon[134520]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:43.653 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:43 smithi181 ceph-mon[134520]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:43 smithi045 ceph-mon[98474]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:43 smithi060 ceph-mon[134136]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:45.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:45 smithi181 ceph-mon[134520]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:45.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-17T01:09:45.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:45 smithi045 ceph-mon[98474]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-17T01:09:45.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:45 smithi060 ceph-mon[134136]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-17T01:09:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:46.518 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:46 smithi181 ceph-mon[134520]: Deploying daemon osd.5 on smithi181 2023-10-17T01:09:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:46 smithi045 ceph-mon[98474]: Deploying daemon osd.5 on smithi181 2023-10-17T01:09:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:46 smithi060 ceph-mon[134136]: Deploying daemon osd.5 on smithi181 2023-10-17T01:09:47.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:47 smithi181 ceph-mon[134520]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:47 smithi045 ceph-mon[98474]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:47 smithi060 ceph-mon[134136]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:49.770 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:49 smithi181 ceph-mon[134520]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:49.770 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:49.771 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:49.771 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:49 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:49.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:49 smithi045 ceph-mon[98474]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:49.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:49.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:49.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:49 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:49 smithi060 ceph-mon[134136]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:49.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:49.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:49 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:51.456 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2023-10-17T01:09:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.454+0000 7fd76f7fe700 1 -- 172.21.15.181:0/2686044650 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd780061190 con 0x7fd7680a0b20 2023-10-17T01:09:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd7680a0b20 msgr2=0x7fd7680a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd7680a0b20 0x7fd7680a2fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd780100d40 tx=0x7fd77400b410 comp rx=0 tx=0).stop 2023-10-17T01:09:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 msgr2=0x7fd7801a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 0x7fd7801a0150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd778007ee0 tx=0x7fd77800d3b0 comp rx=0 tx=0).stop 2023-10-17T01:09:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 shutdown_connections 2023-10-17T01:09:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd7680a0b20 0x7fd7680a2fe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd780101880 0x7fd7801a0150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7800ffce0 0x7fd780069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 --2- 172.21.15.181:0/2686044650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd780100ee0 0x7fd78019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 >> 172.21.15.181:0/2686044650 conn(0x7fd7800fb260 msgr2=0x7fd780104410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:09:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 shutdown_connections 2023-10-17T01:09:51.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:09:51.457+0000 7fd784bbd700 1 -- 172.21.15.181:0/2686044650 wait complete. 2023-10-17T01:09:51.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:51.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:51.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:51.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:51.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:51.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:51.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:51.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:51.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:51.905 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.5.service 2023-10-17T01:09:51.907 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2023-10-17T01:09:51.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-17T01:09:51.953 INFO:journalctl@ceph.osd.5.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:09:52.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:52 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:09:52.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:52 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:52.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:52 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:09:53.143 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:09:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:09:52.698+0000 7f1cb3dcc640 -1 osd.5 0 log_to_monitors true 2023-10-17T01:09:53.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:53 smithi045 ceph-mon[98474]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:53.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:53 smithi045 ceph-mon[98474]: from='osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-17T01:09:53.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:53 smithi045 ceph-mon[98474]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-17T01:09:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:53 smithi060 ceph-mon[134136]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:53 smithi060 ceph-mon[134136]: from='osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-17T01:09:53.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:53 smithi060 ceph-mon[134136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-17T01:09:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:53 smithi181 ceph-mon[134520]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:53 smithi181 ceph-mon[134520]: from='osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-17T01:09:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:53 smithi181 ceph-mon[134520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-17T01:09:54.575 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-17T01:09:54.575 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-17T01:09:54.576 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-10-17T01:09:54.576 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-10-17T01:09:54.576 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0578969 s, 181 MB/s 2023-10-17T01:09:54.576 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-10-17T01:09:54.657 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:09:54.467+0000 7f1ca9443700 -1 osd.5 0 waiting for initial osdmap 2023-10-17T01:09:54.657 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:09:54.476+0000 7f1ca5238700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:09:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-mon[134520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-17T01:09:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-mon[134520]: osdmap e36: 6 total, 5 up, 6 in 2023-10-17T01:09:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-mon[134520]: from='osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:09:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:54 smithi181 ceph-mon[134520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:09:54.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:54 smithi045 ceph-mon[98474]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-17T01:09:54.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:54 smithi045 ceph-mon[98474]: osdmap e36: 6 total, 5 up, 6 in 2023-10-17T01:09:54.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:54.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:54 smithi045 ceph-mon[98474]: from='osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:09:54.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:54 smithi045 ceph-mon[98474]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:09:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:54 smithi060 ceph-mon[134136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-17T01:09:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:54 smithi060 ceph-mon[134136]: osdmap e36: 6 total, 5 up, 6 in 2023-10-17T01:09:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:54 smithi060 ceph-mon[134136]: from='osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:09:54.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:54 smithi060 ceph-mon[134136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:09:55.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2023-10-17T01:09:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:09:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi181 2023-10-17T01:09:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:09:55.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: osdmap e37: 6 total, 5 up, 6 in 2023-10-17T01:09:55.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:55.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:55.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi181 to 8584M 2023-10-17T01:09:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi181 2023-10-17T01:09:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:09:55.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: osdmap e37: 6 total, 5 up, 6 in 2023-10-17T01:09:55.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:55.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:55.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:55.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:55.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi181 to 8584M 2023-10-17T01:09:55.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:55.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:55.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-17T01:09:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi181 2023-10-17T01:09:55.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:09:55.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: osdmap e37: 6 total, 5 up, 6 in 2023-10-17T01:09:55.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:55.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:55.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:55.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:09:55.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi181 to 8584M 2023-10-17T01:09:55.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:55.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:55.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:09:55.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:09:56.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:56 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:09:56.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:56 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:09:56.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:56 smithi045 ceph-mon[98474]: osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] boot 2023-10-17T01:09:56.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:56 smithi045 ceph-mon[98474]: osdmap e38: 6 total, 6 up, 6 in 2023-10-17T01:09:56.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:56.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:56 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:09:56.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:56 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:09:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:56 smithi060 ceph-mon[134136]: osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] boot 2023-10-17T01:09:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:56 smithi060 ceph-mon[134136]: osdmap e38: 6 total, 6 up, 6 in 2023-10-17T01:09:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:56.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:56 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:09:56.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:56 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:09:56.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:56 smithi181 ceph-mon[134520]: osd.5 [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] boot 2023-10-17T01:09:56.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:56 smithi181 ceph-mon[134520]: osdmap e38: 6 total, 6 up, 6 in 2023-10-17T01:09:56.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-17T01:09:57.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 -- 172.21.15.45:0/3094221792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 msgr2=0x7f18a0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:57.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3094221792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0102440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18940099c0 tx=0x7f1894009cd0 comp rx=0 tx=0).stop 2023-10-17T01:09:57.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 -- 172.21.15.45:0/3094221792 shutdown_connections 2023-10-17T01:09:57.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3094221792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a0102980 0x7f18a0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:57.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3094221792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18a0100de0 0x7f18a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:57.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3094221792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0102440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:57.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 -- 172.21.15.45:0/3094221792 >> 172.21.15.45:0/3094221792 conn(0x7f18a00fc340 msgr2=0x7f18a00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:09:57.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 -- 172.21.15.45:0/3094221792 shutdown_connections 2023-10-17T01:09:57.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.242+0000 7f18a5c33700 1 -- 172.21.15.45:0/3094221792 wait complete. 2023-10-17T01:09:57.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.243+0000 7f18a5c33700 1 Processor -- start 2023-10-17T01:09:57.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.243+0000 7f18a5c33700 1 -- start start 2023-10-17T01:09:57.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f18a5c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a0100de0 0x7f18a0196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:57.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f18a5c33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:57.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f18a5c33700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18a0102980 0x7f18a019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:57.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f18a5c33700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a019e6e0 con 0x7f18a0101fe0 2023-10-17T01:09:57.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f18a5c33700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a019e820 con 0x7f18a0102980 2023-10-17T01:09:57.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f18a5c33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a019e9b0 con 0x7f18a0100de0 2023-10-17T01:09:57.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:57.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a0100de0 0x7f18a0196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42690/0 (socket says 172.21.15.45:42690) 2023-10-17T01:09:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 -- 172.21.15.45:0/3812630753 learned_addr learned my addr 172.21.15.45:0/3812630753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:09:57.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189ffff700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18a0102980 0x7f18a019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:57.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 -- 172.21.15.45:0/3812630753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a0100de0 msgr2=0x7f18a0196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:57.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a0100de0 0x7f18a0196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:57.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 -- 172.21.15.45:0/3812630753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18a0102980 msgr2=0x7f18a019e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:09:57.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.244+0000 7f189effd700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18a0102980 0x7f18a019e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:09:57.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f189effd700 1 -- 172.21.15.45:0/3812630753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940095f0 con 0x7f18a0101fe0 2023-10-17T01:09:57.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f189effd700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0197090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18940095c0 tx=0x7f189400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:09:57.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189401d070 con 0x7f18a0101fe0 2023-10-17T01:09:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a019ec30 con 0x7f18a0101fe0 2023-10-17T01:09:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1894005050 con 0x7f18a0101fe0 2023-10-17T01:09:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894004b40 con 0x7f18a0101fe0 2023-10-17T01:09:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.245+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a019f050 con 0x7f18a0101fe0 2023-10-17T01:09:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.246+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a0108480 con 0x7f18a0101fe0 2023-10-17T01:09:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.248+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f18940051c0 con 0x7f18a0101fe0 2023-10-17T01:09:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.249+0000 7f189cff9700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f18880a0b20 0x7f18880a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:09:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.250+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f18940cc5f0 con 0x7f18a0101fe0 2023-10-17T01:09:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.250+0000 7f189f7fe700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f18880a0b20 0x7f18880a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:09:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.250+0000 7f189f7fe700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f18880a0b20 0x7f18880a2fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18a0101e40 tx=0x7f1890009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:09:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.252+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1894096e10 con 0x7f18a0101fe0 2023-10-17T01:09:57.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:09:57.473+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a0061190 con 0x7f18880a0b20 2023-10-17T01:09:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:57 smithi045 ceph-mon[98474]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:09:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:57 smithi045 ceph-mon[98474]: osdmap e39: 6 total, 6 up, 6 in 2023-10-17T01:09:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:57 smithi060 ceph-mon[134136]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:09:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:57 smithi060 ceph-mon[134136]: osdmap e39: 6 total, 6 up, 6 in 2023-10-17T01:09:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:57 smithi181 ceph-mon[134520]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:09:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:57 smithi181 ceph-mon[134520]: osdmap e39: 6 total, 6 up, 6 in 2023-10-17T01:09:58.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:58 smithi045 ceph-mon[98474]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:58 smithi045 ceph-mon[98474]: osdmap e40: 6 total, 6 up, 6 in 2023-10-17T01:09:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:58 smithi060 ceph-mon[134136]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:58 smithi060 ceph-mon[134136]: osdmap e40: 6 total, 6 up, 6 in 2023-10-17T01:09:58.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:58.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:58.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:58 smithi181 ceph-mon[134520]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:09:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:58 smithi181 ceph-mon[134520]: osdmap e40: 6 total, 6 up, 6 in 2023-10-17T01:09:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:09:58.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:09:58.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:09:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:59 smithi045 ceph-mon[98474]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:09:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:09:59 smithi045 ceph-mon[98474]: osdmap e41: 6 total, 6 up, 6 in 2023-10-17T01:09:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:59 smithi060 ceph-mon[134136]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:09:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:09:59 smithi060 ceph-mon[134136]: osdmap e41: 6 total, 6 up, 6 in 2023-10-17T01:09:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:59 smithi181 ceph-mon[134520]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:09:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:09:59 smithi181 ceph-mon[134520]: osdmap e41: 6 total, 6 up, 6 in 2023-10-17T01:10:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:00 smithi045 ceph-mon[98474]: osdmap e42: 6 total, 6 up, 6 in 2023-10-17T01:10:00.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:00 smithi045 ceph-mon[98474]: overall HEALTH_OK 2023-10-17T01:10:00.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:00 smithi045 ceph-mon[98474]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-17T01:10:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:00 smithi060 ceph-mon[134136]: osdmap e42: 6 total, 6 up, 6 in 2023-10-17T01:10:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:00 smithi060 ceph-mon[134136]: overall HEALTH_OK 2023-10-17T01:10:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:00 smithi060 ceph-mon[134136]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-17T01:10:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:00 smithi181 ceph-mon[134520]: osdmap e42: 6 total, 6 up, 6 in 2023-10-17T01:10:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:00 smithi181 ceph-mon[134520]: overall HEALTH_OK 2023-10-17T01:10:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:00 smithi181 ceph-mon[134520]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-17T01:10:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:01 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/451475633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]: dispatch 2023-10-17T01:10:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:01 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]: dispatch 2023-10-17T01:10:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:01 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]': finished 2023-10-17T01:10:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:01 smithi045 ceph-mon[98474]: osdmap e43: 7 total, 6 up, 7 in 2023-10-17T01:10:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:01 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/973461141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:01 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/451475633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]: dispatch 2023-10-17T01:10:01.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:01 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]: dispatch 2023-10-17T01:10:01.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:01 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]': finished 2023-10-17T01:10:01.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:01 smithi060 ceph-mon[134136]: osdmap e43: 7 total, 6 up, 7 in 2023-10-17T01:10:01.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:01.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:01 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/973461141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:01 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/451475633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]: dispatch 2023-10-17T01:10:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:01 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]: dispatch 2023-10-17T01:10:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:01 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb616844-fabd-44f0-9347-9868616fa738"}]': finished 2023-10-17T01:10:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:01 smithi181 ceph-mon[134520]: osdmap e43: 7 total, 6 up, 7 in 2023-10-17T01:10:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:01 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/973461141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:02.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:02 smithi045 ceph-mon[98474]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-17T01:10:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:02 smithi060 ceph-mon[134136]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-17T01:10:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:02 smithi181 ceph-mon[134520]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-17T01:10:05.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:05 smithi181 ceph-mon[134520]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:10:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:05 smithi045 ceph-mon[98474]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:10:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:05 smithi060 ceph-mon[134136]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:10:07.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:07 smithi045 ceph-mon[98474]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-17T01:10:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:07 smithi060 ceph-mon[134136]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-17T01:10:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:07 smithi181 ceph-mon[134520]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-17T01:10:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-17T01:10:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-17T01:10:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-17T01:10:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:09.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:09 smithi045 ceph-mon[98474]: Deploying daemon osd.6 on smithi045 2023-10-17T01:10:09.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:09 smithi045 ceph-mon[98474]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:09 smithi060 ceph-mon[134136]: Deploying daemon osd.6 on smithi045 2023-10-17T01:10:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:09 smithi060 ceph-mon[134136]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:09 smithi181 ceph-mon[134520]: Deploying daemon osd.6 on smithi045 2023-10-17T01:10:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:09 smithi181 ceph-mon[134520]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:11.723 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:11 smithi045 ceph-mon[98474]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:11 smithi060 ceph-mon[134136]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:11 smithi181 ceph-mon[134520]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:13.495 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:13 smithi045 ceph-mon[98474]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:13 smithi060 ceph-mon[134136]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:13 smithi181 ceph-mon[134520]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:15.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:15 smithi181 ceph-mon[134520]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:15.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:15 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:15.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:15 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:15.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:15 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:15.764 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:15 smithi045 ceph-mon[98474]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:15.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:15 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:15.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:15 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:15.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:15 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:15 smithi060 ceph-mon[134136]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:15 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:15.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:15 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:15.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:15 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:17.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:17 smithi045 ceph-mon[98474]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:17 smithi060 ceph-mon[134136]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:17 smithi181 ceph-mon[134520]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:18.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.435+0000 7f189cff9700 1 -- 172.21.15.45:0/3812630753 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f18a0061190 con 0x7f18880a0b20 2023-10-17T01:10:18.437 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2023-10-17T01:10:18.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.437+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f18880a0b20 msgr2=0x7f18880a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:18.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.437+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f18880a0b20 0x7f18880a2fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18a0101e40 tx=0x7f1890009260 comp rx=0 tx=0).stop 2023-10-17T01:10:18.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.437+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 msgr2=0x7f18a0197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:18.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.437+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0197090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18940095c0 tx=0x7f189400f690 comp rx=0 tx=0).stop 2023-10-17T01:10:18.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 shutdown_connections 2023-10-17T01:10:18.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f18880a0b20 0x7f18880a2fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:18.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18a0102980 0x7f18a019e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:18.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a0100de0 0x7f18a0196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:18.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 --2- 172.21.15.45:0/3812630753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0101fe0 0x7f18a0197090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:18.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 >> 172.21.15.45:0/3812630753 conn(0x7f18a00fc340 msgr2=0x7f18a01055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:10:18.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 shutdown_connections 2023-10-17T01:10:18.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:10:18.438+0000 7f18a5c33700 1 -- 172.21.15.45:0/3812630753 wait complete. 2023-10-17T01:10:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:18.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-17T01:10:18.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:18 smithi045 ceph-mon[98474]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-17T01:10:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-17T01:10:18.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:18 smithi181 ceph-mon[134520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-17T01:10:19.031 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.6.service 2023-10-17T01:10:19.038 INFO:tasks.cephadm:Deploying osd.7 on smithi060 with /dev/vg_nvme/lv_2... 2023-10-17T01:10:19.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-17T01:10:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:19.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:19.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:19.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:19.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:19.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:19.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:19.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-17T01:10:19.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:18 smithi060 ceph-mon[134136]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-17T01:10:19.113 INFO:journalctl@ceph.osd.6.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:10:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:10:20.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-mon[98474]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-17T01:10:20.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-mon[98474]: osdmap e44: 7 total, 6 up, 7 in 2023-10-17T01:10:20.438 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:20.438 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-mon[98474]: from='osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:10:20.438 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-mon[98474]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:10:20.439 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:10:20.405+0000 7febf6140700 -1 osd.6 0 waiting for initial osdmap 2023-10-17T01:10:20.439 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:10:20 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:10:20.420+0000 7febeef2f700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:10:20.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:20 smithi181 ceph-mon[134520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-17T01:10:20.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:20 smithi181 ceph-mon[134520]: osdmap e44: 7 total, 6 up, 7 in 2023-10-17T01:10:20.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:20 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:20.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:20 smithi181 ceph-mon[134520]: from='osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:10:20.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:20 smithi181 ceph-mon[134520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:10:20.758 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:20 smithi060 ceph-mon[134136]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-17T01:10:20.759 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:20 smithi060 ceph-mon[134136]: osdmap e44: 7 total, 6 up, 7 in 2023-10-17T01:10:20.759 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:20 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:20.759 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:20 smithi060 ceph-mon[134136]: from='osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:10:20.759 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:20 smithi060 ceph-mon[134136]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:10:21.254 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-17T01:10:21.254 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-17T01:10:21.255 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-10-17T01:10:21.255 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-10-17T01:10:21.255 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133362 s, 786 MB/s 2023-10-17T01:10:21.255 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-10-17T01:10:21.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:21 smithi060 ceph-mon[134136]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:21.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:21 smithi060 ceph-mon[134136]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:10:21.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:21 smithi060 ceph-mon[134136]: osdmap e45: 7 total, 6 up, 7 in 2023-10-17T01:10:21.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:21.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:21.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2023-10-17T01:10:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:21 smithi045 ceph-mon[98474]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:21.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:21 smithi045 ceph-mon[98474]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:10:21.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:21 smithi045 ceph-mon[98474]: osdmap e45: 7 total, 6 up, 7 in 2023-10-17T01:10:21.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:21.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:21 smithi181 ceph-mon[134520]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-17T01:10:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:21 smithi181 ceph-mon[134520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:10:21.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:21 smithi181 ceph-mon[134520]: osdmap e45: 7 total, 6 up, 7 in 2023-10-17T01:10:21.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:21.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:10:22.683 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:22 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:10:22.684 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:22 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:10:22.684 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:22 smithi060 ceph-mon[134136]: osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] boot 2023-10-17T01:10:22.684 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:22 smithi060 ceph-mon[134136]: osdmap e46: 7 total, 7 up, 7 in 2023-10-17T01:10:22.684 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:22 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:22.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:22 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:10:22.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:22 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:10:22.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:22 smithi045 ceph-mon[98474]: osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] boot 2023-10-17T01:10:22.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:22 smithi045 ceph-mon[98474]: osdmap e46: 7 total, 7 up, 7 in 2023-10-17T01:10:22.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:22 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:22.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:22 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:10:22.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:22 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:10:22.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:22 smithi181 ceph-mon[134520]: osd.6 [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] boot 2023-10-17T01:10:22.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:22 smithi181 ceph-mon[134520]: osdmap e46: 7 total, 7 up, 7 in 2023-10-17T01:10:22.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:22 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-17T01:10:23.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2731472029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 msgr2=0x7f5f24101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:23.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2731472029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24101340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5f100099c0 tx=0x7f5f10009cd0 comp rx=0 tx=0).stop 2023-10-17T01:10:23.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2731472029 shutdown_connections 2023-10-17T01:10:23.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2731472029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f24101880 0x7f5f24108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:23.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2731472029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f240ffce0 0x7f5f24100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:23.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2731472029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24101340 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:23.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2731472029 >> 172.21.15.60:0/2731472029 conn(0x7f5f240fb260 msgr2=0x7f5f240fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:10:23.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2731472029 shutdown_connections 2023-10-17T01:10:23.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.503+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2731472029 wait complete. 2023-10-17T01:10:23.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 Processor -- start 2023-10-17T01:10:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 -- start start 2023-10-17T01:10:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f240ffce0 0x7f5f24195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f24101880 0x7f5f2419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2419d400 con 0x7f5f24101880 2023-10-17T01:10:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2419d540 con 0x7f5f24100ee0 2023-10-17T01:10:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f2adb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2419d6d0 con 0x7f5f240ffce0 2023-10-17T01:10:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f28b51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f240ffce0 0x7f5f24195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f23fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f28b51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f240ffce0 0x7f5f24195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.60:37722/0 (socket says 172.21.15.60:37722) 2023-10-17T01:10:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f28b51700 1 -- 172.21.15.60:0/2063284333 learned_addr learned my addr 172.21.15.60:0/2063284333 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-10-17T01:10:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.504+0000 7f5f29352700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f24101880 0x7f5f2419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f23fff700 1 -- 172.21.15.60:0/2063284333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f240ffce0 msgr2=0x7f5f24195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f23fff700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f240ffce0 0x7f5f24195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f23fff700 1 -- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f24101880 msgr2=0x7f5f2419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f23fff700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f24101880 0x7f5f2419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f23fff700 1 -- 172.21.15.60:0/2063284333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100095f0 con 0x7f5f24100ee0 2023-10-17T01:10:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f29352700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f24101880 0x7f5f2419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:10:23.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f23fff700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24195db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009990 tx=0x7f5f10003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:10:23.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1001d070 con 0x7f5f24100ee0 2023-10-17T01:10:23.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f10004640 con 0x7f5f24100ee0 2023-10-17T01:10:23.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f24068810 con 0x7f5f24100ee0 2023-10-17T01:10:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10022e90 con 0x7f5f24100ee0 2023-10-17T01:10:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.505+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f24068d00 con 0x7f5f24100ee0 2023-10-17T01:10:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.506+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f240696f0 con 0x7f5f24100ee0 2023-10-17T01:10:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.507+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f5f10003fa0 con 0x7f5f24100ee0 2023-10-17T01:10:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.508+0000 7f5f21ffb700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5f140a0ba0 0x7f5f140a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.508+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f5f100ccf50 con 0x7f5f24100ee0 2023-10-17T01:10:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.508+0000 7f5f28b51700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5f140a0ba0 0x7f5f140a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.509+0000 7f5f28b51700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5f140a0ba0 0x7f5f140a3060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800cef0 tx=0x7f5f1800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:10:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.511+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f5f10097680 con 0x7f5f24100ee0 2023-10-17T01:10:23.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:23.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: osdmap e47: 7 total, 7 up, 7 in 2023-10-17T01:10:23.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.581 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:23.581 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:23.581 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:23.723+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f24068f20 con 0x7f5f140a0ba0 2023-10-17T01:10:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: osdmap e47: 7 total, 7 up, 7 in 2023-10-17T01:10:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:23.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:23.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: osdmap e47: 7 total, 7 up, 7 in 2023-10-17T01:10:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:23.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:23.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:23.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:23.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:24.725 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:24 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi045 2023-10-17T01:10:24.725 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:24 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi045 to 5758M 2023-10-17T01:10:24.726 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:10:24.726 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:10:24.726 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:24 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi045 2023-10-17T01:10:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:24 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi045 to 5758M 2023-10-17T01:10:24.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:10:24.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:10:24.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:24 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi045 2023-10-17T01:10:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:24 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi045 to 5758M 2023-10-17T01:10:24.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:10:24.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:10:24.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:25.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:25 smithi060 ceph-mon[134136]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:10:25.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:25 smithi060 ceph-mon[134136]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:25 smithi045 ceph-mon[98474]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:10:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:25 smithi045 ceph-mon[98474]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:25 smithi181 ceph-mon[134520]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:10:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:25 smithi181 ceph-mon[134520]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:26 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/1795851791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]: dispatch 2023-10-17T01:10:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:26 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]: dispatch 2023-10-17T01:10:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:26 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]': finished 2023-10-17T01:10:26.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:26 smithi045 ceph-mon[98474]: osdmap e48: 8 total, 7 up, 8 in 2023-10-17T01:10:26.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:26.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:26 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/1795851791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]: dispatch 2023-10-17T01:10:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:26 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]: dispatch 2023-10-17T01:10:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:26 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]': finished 2023-10-17T01:10:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:26 smithi060 ceph-mon[134136]: osdmap e48: 8 total, 7 up, 8 in 2023-10-17T01:10:26.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:26 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:26.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:26 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/1795851791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]: dispatch 2023-10-17T01:10:26.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:26 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]: dispatch 2023-10-17T01:10:26.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:26 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d58f8b41-8d53-4e6c-b2d2-7997cb77e063"}]': finished 2023-10-17T01:10:26.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:26 smithi181 ceph-mon[134520]: osdmap e48: 8 total, 7 up, 8 in 2023-10-17T01:10:26.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:27 smithi045 ceph-mon[98474]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:27 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/2906632221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:27 smithi060 ceph-mon[134136]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:27 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/2906632221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:27 smithi181 ceph-mon[134520]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:27 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/2906632221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:29 smithi045 ceph-mon[98474]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:29 smithi060 ceph-mon[134136]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:29 smithi181 ceph-mon[134520]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:31.715 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:31 smithi060 ceph-mon[134136]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:31 smithi045 ceph-mon[98474]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:31 smithi181 ceph-mon[134520]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:33.724 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:33 smithi060 ceph-mon[134136]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:33.724 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-17T01:10:33.724 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:33 smithi045 ceph-mon[98474]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-17T01:10:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:33 smithi181 ceph-mon[134520]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-17T01:10:33.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:34.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:34 smithi045 ceph-mon[98474]: Deploying daemon osd.7 on smithi060 2023-10-17T01:10:34.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:34 smithi060 ceph-mon[134136]: Deploying daemon osd.7 on smithi060 2023-10-17T01:10:34.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:34 smithi181 ceph-mon[134520]: Deploying daemon osd.7 on smithi060 2023-10-17T01:10:35.574 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:35 smithi060 ceph-mon[134136]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:35 smithi045 ceph-mon[98474]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:35.849 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:35 smithi181 ceph-mon[134520]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:37.782 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:37 smithi060 ceph-mon[134136]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:37.783 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:37.783 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:37.783 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:37 smithi045 ceph-mon[98474]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:37.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:37 smithi181 ceph-mon[134520]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:37.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:37.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:37.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:39 smithi045 ceph-mon[98474]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:39 smithi060 ceph-mon[134136]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:39 smithi181 ceph-mon[134520]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:39.997 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 7 on host 'smithi060' 2023-10-17T01:10:39.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.996+0000 7f5f21ffb700 1 -- 172.21.15.60:0/2063284333 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5f24068f20 con 0x7f5f140a0ba0 2023-10-17T01:10:39.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5f140a0ba0 msgr2=0x7f5f140a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:39.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5f140a0ba0 0x7f5f140a3060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800cef0 tx=0x7f5f1800e040 comp rx=0 tx=0).stop 2023-10-17T01:10:39.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 msgr2=0x7f5f24195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:39.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24195db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009990 tx=0x7f5f10003af0 comp rx=0 tx=0).stop 2023-10-17T01:10:40.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 shutdown_connections 2023-10-17T01:10:40.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5f140a0ba0 0x7f5f140a3060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:40.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f24101880 0x7f5f2419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:40.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f240ffce0 0x7f5f24195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:40.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 --2- 172.21.15.60:0/2063284333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f24100ee0 0x7f5f24195db0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:40.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 >> 172.21.15.60:0/2063284333 conn(0x7f5f240fb260 msgr2=0x7f5f241042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:10:40.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 shutdown_connections 2023-10-17T01:10:40.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:10:39.998+0000 7f5f2adb5700 1 -- 172.21.15.60:0/2063284333 wait complete. 2023-10-17T01:10:40.365 DEBUG:teuthology.orchestra.run.smithi060:osd.7> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.7.service 2023-10-17T01:10:40.367 INFO:tasks.cephadm:Deploying osd.8 on smithi181 with /dev/vg_nvme/lv_2... 2023-10-17T01:10:40.368 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-17T01:10:40.398 INFO:journalctl@ceph.osd.7.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:10:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:10:41.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:41.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:40 smithi045 ceph-mon[98474]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:41.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:41.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:40 smithi060 ceph-mon[134136]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:41.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.152 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.153 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:41.153 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:41.153 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.153 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.153 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:41.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:41.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:40 smithi181 ceph-mon[134520]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:41.458 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:10:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:10:41.319+0000 7f22ee7ab640 -1 osd.7 0 log_to_monitors true 2023-10-17T01:10:42.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:41 smithi181 ceph-mon[134520]: from='osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-17T01:10:42.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:41 smithi181 ceph-mon[134520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-17T01:10:42.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:41 smithi045 ceph-mon[98474]: from='osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-17T01:10:42.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:41 smithi045 ceph-mon[98474]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-17T01:10:42.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:41 smithi060 ceph-mon[134136]: from='osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-17T01:10:42.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:41 smithi060 ceph-mon[134136]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-17T01:10:42.397 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-17T01:10:42.398 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-17T01:10:42.398 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-10-17T01:10:42.398 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-10-17T01:10:42.398 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110731 s, 947 MB/s 2023-10-17T01:10:42.398 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-10-17T01:10:42.773 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2023-10-17T01:10:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:42 smithi181 ceph-mon[134520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-17T01:10:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:42 smithi181 ceph-mon[134520]: osdmap e49: 8 total, 7 up, 8 in 2023-10-17T01:10:43.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:42 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:43.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:42 smithi181 ceph-mon[134520]: from='osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:10:43.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:42 smithi181 ceph-mon[134520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:10:43.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:42 smithi181 ceph-mon[134520]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:43.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:42 smithi045 ceph-mon[98474]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-17T01:10:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:42 smithi045 ceph-mon[98474]: osdmap e49: 8 total, 7 up, 8 in 2023-10-17T01:10:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:42 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:42 smithi045 ceph-mon[98474]: from='osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:10:43.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:42 smithi045 ceph-mon[98474]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:10:43.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:42 smithi045 ceph-mon[98474]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:43.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:10:42.987+0000 7f22e3e22700 -1 osd.7 0 waiting for initial osdmap 2023-10-17T01:10:43.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:10:42.998+0000 7f22df416700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:10:43.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-mon[134136]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-17T01:10:43.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-mon[134136]: osdmap e49: 8 total, 7 up, 8 in 2023-10-17T01:10:43.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:43.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-mon[134136]: from='osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:10:43.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-mon[134136]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:10:43.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:42 smithi060 ceph-mon[134136]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-10-17T01:10:43.331 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:10:44.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:10:44.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: osdmap e50: 8 total, 7 up, 8 in 2023-10-17T01:10:44.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:44.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:44.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi060 2023-10-17T01:10:44.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi060 to 5724M 2023-10-17T01:10:44.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:44.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:44.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:10:44.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: osdmap e50: 8 total, 7 up, 8 in 2023-10-17T01:10:44.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:44.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:44.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi060 2023-10-17T01:10:44.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi060 to 5724M 2023-10-17T01:10:44.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:44.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:44.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:10:44.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: osdmap e50: 8 total, 7 up, 8 in 2023-10-17T01:10:44.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:44.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:44.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi060 2023-10-17T01:10:44.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:10:44.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi060 to 5724M 2023-10-17T01:10:44.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:44.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:10:44.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:44.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 -- 172.21.15.181:0/4191430260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c78100ee0 msgr2=0x7f2c78101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/4191430260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c78100ee0 0x7f2c78101340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009990 tx=0x7f2c74009ca0 comp rx=0 tx=0).stop 2023-10-17T01:10:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 -- 172.21.15.181:0/4191430260 shutdown_connections 2023-10-17T01:10:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/4191430260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78101880 0x7f2c78108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/4191430260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c780ffce0 0x7f2c78100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/4191430260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c78100ee0 0x7f2c78101340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 -- 172.21.15.181:0/4191430260 >> 172.21.15.181:0/4191430260 conn(0x7f2c780fb260 msgr2=0x7f2c780fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:10:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 -- 172.21.15.181:0/4191430260 shutdown_connections 2023-10-17T01:10:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.486+0000 7f2c7fecc700 1 -- 172.21.15.181:0/4191430260 wait complete. 2023-10-17T01:10:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 Processor -- start 2023-10-17T01:10:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 -- start start 2023-10-17T01:10:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 0x7f2c78195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78100ee0 0x7f2c78196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101880 0x7f2c7819d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7819d690 con 0x7f2c78100ee0 2023-10-17T01:10:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7819d7d0 con 0x7f2c78101880 2023-10-17T01:10:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7fecc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7819d960 con 0x7f2c780ffce0 2023-10-17T01:10:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7dc68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 0x7f2c78195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7dc68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 0x7f2c78195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45300/0 (socket says 172.21.15.181:45300) 2023-10-17T01:10:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.487+0000 7f2c7dc68700 1 -- 172.21.15.181:0/1666607597 learned_addr learned my addr 172.21.15.181:0/1666607597 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:10:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7e469700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101880 0x7f2c7819d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7dc68700 1 -- 172.21.15.181:0/1666607597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101880 msgr2=0x7f2c7819d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7dc68700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101880 0x7f2c7819d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7d467700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78100ee0 0x7f2c78196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7dc68700 1 -- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78100ee0 msgr2=0x7f2c78196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:10:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7dc68700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78100ee0 0x7f2c78196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:10:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7dc68700 1 -- 172.21.15.181:0/1666607597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740095f0 con 0x7f2c780ffce0 2023-10-17T01:10:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7d467700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78100ee0 0x7f2c78196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:10:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7dc68700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 0x7f2c78195b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b530 tx=0x7f2c6800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:10:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011830 con 0x7f2c780ffce0 2023-10-17T01:10:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7819dc40 con 0x7f2c780ffce0 2023-10-17T01:10:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c68011e50 con 0x7f2c780ffce0 2023-10-17T01:10:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800f2e0 con 0x7f2c780ffce0 2023-10-17T01:10:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.488+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7819e140 con 0x7f2c780ffce0 2023-10-17T01:10:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.489+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78107380 con 0x7f2c780ffce0 2023-10-17T01:10:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.490+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2c68010580 con 0x7f2c780ffce0 2023-10-17T01:10:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.491+0000 7f2c6effd700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c640a0b20 0x7f2c640a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:10:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.491+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f2c680cb720 con 0x7f2c780ffce0 2023-10-17T01:10:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.491+0000 7f2c7d467700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c640a0b20 0x7f2c640a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:10:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.491+0000 7f2c7d467700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c640a0b20 0x7f2c640a2fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c740095c0 tx=0x7f2c74009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:10:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.494+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2c68095bf0 con 0x7f2c780ffce0 2023-10-17T01:10:44.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:10:44.693+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c78061190 con 0x7f2c640a0b20 2023-10-17T01:10:45.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:10:45.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:10:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] boot 2023-10-17T01:10:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: osdmap e51: 8 total, 8 up, 8 in 2023-10-17T01:10:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: osdmap e52: 8 total, 8 up, 8 in 2023-10-17T01:10:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:45.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:10:45.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:10:45.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:45.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:10:45.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:10:45.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] boot 2023-10-17T01:10:45.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: osdmap e51: 8 total, 8 up, 8 in 2023-10-17T01:10:45.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:45.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: osdmap e52: 8 total, 8 up, 8 in 2023-10-17T01:10:45.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:45.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:10:45.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:10:45.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:45.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:10:45.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:10:45.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: osd.7 [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] boot 2023-10-17T01:10:45.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: osdmap e51: 8 total, 8 up, 8 in 2023-10-17T01:10:45.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-17T01:10:45.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: osdmap e52: 8 total, 8 up, 8 in 2023-10-17T01:10:45.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:45.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:10:45.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:10:45.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:46.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:46 smithi181 ceph-mon[134520]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:10:46.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:46 smithi181 ceph-mon[134520]: osdmap e53: 8 total, 8 up, 8 in 2023-10-17T01:10:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:46 smithi045 ceph-mon[98474]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:10:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:46 smithi045 ceph-mon[98474]: osdmap e53: 8 total, 8 up, 8 in 2023-10-17T01:10:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:46 smithi060 ceph-mon[134136]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:10:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:46 smithi060 ceph-mon[134136]: osdmap e53: 8 total, 8 up, 8 in 2023-10-17T01:10:47.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: osdmap e54: 8 total, 8 up, 8 in 2023-10-17T01:10:47.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: pgmap v171: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:10:47.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/1903813647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]: dispatch 2023-10-17T01:10:47.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]: dispatch 2023-10-17T01:10:47.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]': finished 2023-10-17T01:10:47.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: osdmap e55: 9 total, 8 up, 9 in 2023-10-17T01:10:47.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:10:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: osdmap e54: 8 total, 8 up, 8 in 2023-10-17T01:10:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: pgmap v171: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:10:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/1903813647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]: dispatch 2023-10-17T01:10:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]: dispatch 2023-10-17T01:10:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]': finished 2023-10-17T01:10:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: osdmap e55: 9 total, 8 up, 9 in 2023-10-17T01:10:47.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:10:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: osdmap e54: 8 total, 8 up, 8 in 2023-10-17T01:10:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: pgmap v171: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:10:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/1903813647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]: dispatch 2023-10-17T01:10:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]: dispatch 2023-10-17T01:10:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58034e5c-8cae-4694-8a22-bfec322b43b1"}]': finished 2023-10-17T01:10:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: osdmap e55: 9 total, 8 up, 9 in 2023-10-17T01:10:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:10:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:48 smithi181 ceph-mon[134520]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-10-17T01:10:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:48 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/2365532081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:48 smithi045 ceph-mon[98474]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-10-17T01:10:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:48 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/2365532081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:48 smithi060 ceph-mon[134136]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-10-17T01:10:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:48 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/2365532081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:10:49.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:49 smithi181 ceph-mon[134520]: pgmap v173: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:10:49.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:49 smithi045 ceph-mon[98474]: pgmap v173: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:10:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:49 smithi060 ceph-mon[134136]: pgmap v173: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:10:50.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:50 smithi045 ceph-mon[98474]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:50.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:50 smithi045 ceph-mon[98474]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-10-17T01:10:50.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:50 smithi045 ceph-mon[98474]: Cluster is now healthy 2023-10-17T01:10:50.819 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:50 smithi181 ceph-mon[134520]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:50.820 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:50 smithi181 ceph-mon[134520]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-10-17T01:10:50.820 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:50 smithi181 ceph-mon[134520]: Cluster is now healthy 2023-10-17T01:10:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:50 smithi060 ceph-mon[134136]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:50 smithi060 ceph-mon[134136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-10-17T01:10:50.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:50 smithi060 ceph-mon[134136]: Cluster is now healthy 2023-10-17T01:10:53.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:53 smithi045 ceph-mon[98474]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:53 smithi060 ceph-mon[134136]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:53 smithi181 ceph-mon[134520]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:54.718 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-17T01:10:54.719 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:54.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-17T01:10:54.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-17T01:10:54.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:10:55.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:55 smithi181 ceph-mon[134520]: Deploying daemon osd.8 on smithi181 2023-10-17T01:10:55.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:55 smithi181 ceph-mon[134520]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:55 smithi045 ceph-mon[98474]: Deploying daemon osd.8 on smithi181 2023-10-17T01:10:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:55 smithi045 ceph-mon[98474]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:55 smithi060 ceph-mon[134136]: Deploying daemon osd.8 on smithi181 2023-10-17T01:10:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:55 smithi060 ceph-mon[134136]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:57.771 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:57 smithi181 ceph-mon[134520]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:57 smithi045 ceph-mon[98474]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:57 smithi060 ceph-mon[134136]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:58.926 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:58.926 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:58.927 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:58.927 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:10:58 smithi181 ceph-mon[134520]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:10:58 smithi045 ceph-mon[98474]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:10:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:10:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:10:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:10:58 smithi060 ceph-mon[134136]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:00.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.405+0000 7f2c6effd700 1 -- 172.21.15.181:0/1666607597 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c78061190 con 0x7f2c640a0b20 2023-10-17T01:11:00.406 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 8 on host 'smithi181' 2023-10-17T01:11:00.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c640a0b20 msgr2=0x7f2c640a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:00.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c640a0b20 0x7f2c640a2fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c740095c0 tx=0x7f2c74009510 comp rx=0 tx=0).stop 2023-10-17T01:11:00.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 msgr2=0x7f2c78195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:00.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 0x7f2c78195b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b530 tx=0x7f2c6800b8f0 comp rx=0 tx=0).stop 2023-10-17T01:11:00.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 shutdown_connections 2023-10-17T01:11:00.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c640a0b20 0x7f2c640a2fe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:00.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101880 0x7f2c7819d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:00.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c780ffce0 0x7f2c78195b00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:00.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 --2- 172.21.15.181:0/1666607597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c78100ee0 0x7f2c78196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:00.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 >> 172.21.15.181:0/1666607597 conn(0x7f2c780fb260 msgr2=0x7f2c781044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:11:00.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 shutdown_connections 2023-10-17T01:11:00.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:00.407+0000 7f2c7fecc700 1 -- 172.21.15.181:0/1666607597 wait complete. 2023-10-17T01:11:00.803 DEBUG:teuthology.orchestra.run.smithi181:osd.8> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.8.service 2023-10-17T01:11:00.806 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2023-10-17T01:11:00.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-17T01:11:00.838 INFO:journalctl@ceph.osd.8.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:11:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:11:01.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:01 smithi181 ceph-mon[134520]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:01.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:01 smithi045 ceph-mon[98474]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:01 smithi060 ceph-mon[134136]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:01.907 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:11:01 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:11:01.823+0000 7fea74a72640 -1 osd.8 0 log_to_monitors true 2023-10-17T01:11:02.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:02 smithi045 ceph-mon[98474]: from='osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-17T01:11:02.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:02 smithi045 ceph-mon[98474]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-17T01:11:02.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:02 smithi181 ceph-mon[134520]: from='osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-17T01:11:02.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:02 smithi181 ceph-mon[134520]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-17T01:11:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:02 smithi060 ceph-mon[134136]: from='osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-17T01:11:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:02 smithi060 ceph-mon[134136]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-17T01:11:03.771 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-17T01:11:03.772 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-17T01:11:03.772 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2023-10-17T01:11:03.772 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2023-10-17T01:11:03.772 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0309723 s, 339 MB/s 2023-10-17T01:11:03.772 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2023-10-17T01:11:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:03 smithi045 ceph-mon[98474]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-17T01:11:03.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:03 smithi045 ceph-mon[98474]: osdmap e56: 9 total, 8 up, 9 in 2023-10-17T01:11:03.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:03.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:03 smithi045 ceph-mon[98474]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:03.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:03 smithi045 ceph-mon[98474]: from='osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:11:03.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:03 smithi045 ceph-mon[98474]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:11:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:03 smithi060 ceph-mon[134136]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-17T01:11:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:03 smithi060 ceph-mon[134136]: osdmap e56: 9 total, 8 up, 9 in 2023-10-17T01:11:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:03 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:03 smithi060 ceph-mon[134136]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:03 smithi060 ceph-mon[134136]: from='osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:11:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:03 smithi060 ceph-mon[134136]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:11:03.907 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:11:03.413+0000 7fea6a0e9700 -1 osd.8 0 waiting for initial osdmap 2023-10-17T01:11:03.907 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:11:03.426+0000 7fea626d7700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:11:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-mon[134520]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-17T01:11:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-mon[134520]: osdmap e56: 9 total, 8 up, 9 in 2023-10-17T01:11:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:03.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-mon[134520]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:03.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-mon[134520]: from='osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:11:03.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:03 smithi181 ceph-mon[134520]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:11:04.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2023-10-17T01:11:04.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:11:04.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: osdmap e57: 9 total, 8 up, 9 in 2023-10-17T01:11:04.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:04.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:04.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:04.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:04.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:04.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] boot 2023-10-17T01:11:04.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: osdmap e58: 9 total, 9 up, 9 in 2023-10-17T01:11:04.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:11:04.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: osdmap e57: 9 total, 8 up, 9 in 2023-10-17T01:11:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:04.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:04.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:04.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:04.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:04.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] boot 2023-10-17T01:11:04.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: osdmap e58: 9 total, 9 up, 9 in 2023-10-17T01:11:04.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:11:04.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:11:04.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: osdmap e57: 9 total, 8 up, 9 in 2023-10-17T01:11:04.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:04.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:04.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:04.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: osd.8 [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] boot 2023-10-17T01:11:04.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: osdmap e58: 9 total, 9 up, 9 in 2023-10-17T01:11:04.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-17T01:11:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:05 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:11:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:05 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:11:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:05 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi181 2023-10-17T01:11:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:05 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi181 to 5722M 2023-10-17T01:11:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:05 smithi045 ceph-mon[98474]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:05 smithi045 ceph-mon[98474]: osdmap e59: 9 total, 9 up, 9 in 2023-10-17T01:11:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:05 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:11:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:05 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:11:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:05 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi181 2023-10-17T01:11:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:05 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi181 to 5722M 2023-10-17T01:11:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:05 smithi060 ceph-mon[134136]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:05 smithi060 ceph-mon[134136]: osdmap e59: 9 total, 9 up, 9 in 2023-10-17T01:11:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:05 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:11:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:05 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:11:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:05 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi181 2023-10-17T01:11:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:05 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi181 to 5722M 2023-10-17T01:11:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:05 smithi181 ceph-mon[134520]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-17T01:11:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:05 smithi181 ceph-mon[134520]: osdmap e59: 9 total, 9 up, 9 in 2023-10-17T01:11:06.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.398+0000 7f04ab4cc700 1 -- 172.21.15.45:0/585373897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a40737b0 msgr2=0x7f04a4073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:06.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.398+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/585373897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a40737b0 0x7f04a4073c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0494009990 tx=0x7f0494009ca0 comp rx=0 tx=0).stop 2023-10-17T01:11:06.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 -- 172.21.15.45:0/585373897 shutdown_connections 2023-10-17T01:11:06.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/585373897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04a4108cb0 0x7f04a410b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:06.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/585373897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a4074d80 0x7f04a40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:06.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/585373897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a40737b0 0x7f04a4073c30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:06.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 -- 172.21.15.45:0/585373897 >> 172.21.15.45:0/585373897 conn(0x7f04a40fc340 msgr2=0x7f04a40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:11:06.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 -- 172.21.15.45:0/585373897 shutdown_connections 2023-10-17T01:11:06.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.399+0000 7f04ab4cc700 1 -- 172.21.15.45:0/585373897 wait complete. 2023-10-17T01:11:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 Processor -- start 2023-10-17T01:11:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 -- start start 2023-10-17T01:11:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a40737b0 0x7f04a4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a4074d80 0x7f04a4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04a4108cb0 0x7f04a41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a41a0830 con 0x7f04a40737b0 2023-10-17T01:11:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a41a0970 con 0x7f04a4108cb0 2023-10-17T01:11:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.400+0000 7f04ab4cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a41a0b30 con 0x7f04a4074d80 2023-10-17T01:11:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a9268700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a40737b0 0x7f04a4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a9268700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a40737b0 0x7f04a4198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49180/0 (socket says 172.21.15.45:49180) 2023-10-17T01:11:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a9268700 1 -- 172.21.15.45:0/2553365719 learned_addr learned my addr 172.21.15.45:0/2553365719 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:11:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a4074d80 0x7f04a4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a9a69700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04a4108cb0 0x7f04a41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 -- 172.21.15.45:0/2553365719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04a4108cb0 msgr2=0x7f04a41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04a4108cb0 0x7f04a41a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 -- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a40737b0 msgr2=0x7f04a4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:06.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a40737b0 0x7f04a4198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:06.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 -- 172.21.15.45:0/2553365719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940095f0 con 0x7f04a4074d80 2023-10-17T01:11:06.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.401+0000 7f04a8a67700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a4074d80 0x7f04a4199220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04940098e0 tx=0x7f0494009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:11:06.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.402+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049401d070 con 0x7f04a4074d80 2023-10-17T01:11:06.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.402+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a41a0db0 con 0x7f04a4074d80 2023-10-17T01:11:06.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.402+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0494021410 con 0x7f04a4074d80 2023-10-17T01:11:06.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.402+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494017650 con 0x7f04a4074d80 2023-10-17T01:11:06.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.402+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a41a1170 con 0x7f04a4074d80 2023-10-17T01:11:06.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.403+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a4107360 con 0x7f04a4074d80 2023-10-17T01:11:06.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.405+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0494021810 con 0x7f04a4074d80 2023-10-17T01:11:06.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.407+0000 7f049a7fc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04900a0b20 0x7f04900a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:06.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.407+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f04940ccc40 con 0x7f04a4074d80 2023-10-17T01:11:06.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.407+0000 7f04a9268700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04900a0b20 0x7f04900a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:06.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.407+0000 7f04a9268700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04900a0b20 0x7f04900a2fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f04a419a1c0 tx=0x7f04a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:11:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.411+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0494096f50 con 0x7f04a4074d80 2023-10-17T01:11:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:06.631+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f04a4061190 con 0x7f04900a0b20 2023-10-17T01:11:07.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:07 smithi181 ceph-mon[134520]: osdmap e60: 9 total, 9 up, 9 in 2023-10-17T01:11:07.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:07 smithi181 ceph-mon[134520]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:07.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:07.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:07.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:07.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:07 smithi045 ceph-mon[98474]: osdmap e60: 9 total, 9 up, 9 in 2023-10-17T01:11:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:07 smithi045 ceph-mon[98474]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:07 smithi060 ceph-mon[134136]: osdmap e60: 9 total, 9 up, 9 in 2023-10-17T01:11:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:07 smithi060 ceph-mon[134136]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:08.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:08 smithi045 ceph-mon[98474]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:08.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:08 smithi045 ceph-mon[98474]: osdmap e61: 9 total, 9 up, 9 in 2023-10-17T01:11:08.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:08 smithi181 ceph-mon[134520]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:08.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:08 smithi181 ceph-mon[134520]: osdmap e61: 9 total, 9 up, 9 in 2023-10-17T01:11:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:08 smithi060 ceph-mon[134136]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:08 smithi060 ceph-mon[134136]: osdmap e61: 9 total, 9 up, 9 in 2023-10-17T01:11:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:09 smithi045 ceph-mon[98474]: osdmap e62: 9 total, 9 up, 9 in 2023-10-17T01:11:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:09 smithi045 ceph-mon[98474]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:09 smithi060 ceph-mon[134136]: osdmap e62: 9 total, 9 up, 9 in 2023-10-17T01:11:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:09 smithi060 ceph-mon[134136]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:09 smithi181 ceph-mon[134520]: osdmap e62: 9 total, 9 up, 9 in 2023-10-17T01:11:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:09 smithi181 ceph-mon[134520]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:10 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2586935822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6c8b883-1f5b-4e01-8453-7ecae38051e6"}]: dispatch 2023-10-17T01:11:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:10 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2586935822' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6c8b883-1f5b-4e01-8453-7ecae38051e6"}]': finished 2023-10-17T01:11:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:10 smithi045 ceph-mon[98474]: osdmap e63: 10 total, 9 up, 10 in 2023-10-17T01:11:10.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:10 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:10 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2586935822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6c8b883-1f5b-4e01-8453-7ecae38051e6"}]: dispatch 2023-10-17T01:11:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:10 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2586935822' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6c8b883-1f5b-4e01-8453-7ecae38051e6"}]': finished 2023-10-17T01:11:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:10 smithi060 ceph-mon[134136]: osdmap e63: 10 total, 9 up, 10 in 2023-10-17T01:11:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:10 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:10 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2586935822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6c8b883-1f5b-4e01-8453-7ecae38051e6"}]: dispatch 2023-10-17T01:11:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:10 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2586935822' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6c8b883-1f5b-4e01-8453-7ecae38051e6"}]': finished 2023-10-17T01:11:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:10 smithi181 ceph-mon[134520]: osdmap e63: 10 total, 9 up, 10 in 2023-10-17T01:11:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:10 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:11 smithi045 ceph-mon[98474]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-17T01:11:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:11 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4150868719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:11 smithi060 ceph-mon[134136]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-17T01:11:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:11 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/4150868719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:11 smithi181 ceph-mon[134520]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-17T01:11:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:11 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/4150868719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:13.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:13 smithi045 ceph-mon[98474]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:11:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:13 smithi060 ceph-mon[134136]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:11:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:13 smithi181 ceph-mon[134520]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-17T01:11:15.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:15 smithi045 ceph-mon[98474]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-10-17T01:11:15.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:15 smithi181 ceph-mon[134520]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-10-17T01:11:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:15 smithi060 ceph-mon[134136]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-10-17T01:11:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:17 smithi045 ceph-mon[98474]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:11:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-17T01:11:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:17 smithi060 ceph-mon[134136]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:11:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-17T01:11:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:17 smithi181 ceph-mon[134520]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:11:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-17T01:11:17.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:18.712 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:18 smithi045 ceph-mon[98474]: Deploying daemon osd.9 on smithi045 2023-10-17T01:11:18.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:18 smithi060 ceph-mon[134136]: Deploying daemon osd.9 on smithi045 2023-10-17T01:11:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:18 smithi181 ceph-mon[134520]: Deploying daemon osd.9 on smithi045 2023-10-17T01:11:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:19 smithi045 ceph-mon[98474]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-17T01:11:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:19 smithi060 ceph-mon[134136]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-17T01:11:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:19 smithi181 ceph-mon[134520]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-17T01:11:21.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:21 smithi045 ceph-mon[98474]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:21.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:21 smithi060 ceph-mon[134136]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:21 smithi181 ceph-mon[134520]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:23.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:23 smithi045 ceph-mon[98474]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:23 smithi060 ceph-mon[134136]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:23 smithi181 ceph-mon[134520]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:24.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:24 smithi181 ceph-mon[134520]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:24.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:24.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:24.973 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:24.973 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:24 smithi045 ceph-mon[98474]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:25.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:25.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:25.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:25.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:24 smithi060 ceph-mon[134136]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:27.200 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2023-10-17T01:11:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.199+0000 7f049a7fc700 1 -- 172.21.15.45:0/2553365719 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f04a4061190 con 0x7f04900a0b20 2023-10-17T01:11:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04900a0b20 msgr2=0x7f04900a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04900a0b20 0x7f04900a2fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f04a419a1c0 tx=0x7f04a0009260 comp rx=0 tx=0).stop 2023-10-17T01:11:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a4074d80 msgr2=0x7f04a4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:27.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a4074d80 0x7f04a4199220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04940098e0 tx=0x7f0494009580 comp rx=0 tx=0).stop 2023-10-17T01:11:27.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 shutdown_connections 2023-10-17T01:11:27.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04900a0b20 0x7f04900a2fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:27.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04a4108cb0 0x7f04a41a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04a40737b0 0x7f04a4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 --2- 172.21.15.45:0/2553365719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a4074d80 0x7f04a4199220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 >> 172.21.15.45:0/2553365719 conn(0x7f04a40fc340 msgr2=0x7f04a40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:11:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 shutdown_connections 2023-10-17T01:11:27.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:11:27.201+0000 7f04ab4cc700 1 -- 172.21.15.45:0/2553365719 wait complete. 2023-10-17T01:11:27.639 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.9.service 2023-10-17T01:11:27.644 INFO:tasks.cephadm:Deploying osd.10 on smithi060 with /dev/vg_nvme/lv_1... 2023-10-17T01:11:27.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-17T01:11:27.719 INFO:journalctl@ceph.osd.9.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:11:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:28.218 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:11:28.718 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:28 smithi045 ceph-mon[98474]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-17T01:11:28.808 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:28 smithi060 ceph-mon[134136]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-17T01:11:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:28 smithi181 ceph-mon[134520]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-17T01:11:29.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:29 smithi060 ceph-mon[134136]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:29.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:29 smithi060 ceph-mon[134136]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-17T01:11:29.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:29 smithi060 ceph-mon[134136]: osdmap e64: 10 total, 9 up, 10 in 2023-10-17T01:11:29.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:29.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:29 smithi060 ceph-mon[134136]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:11:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-mon[98474]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-mon[98474]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-17T01:11:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-mon[98474]: osdmap e64: 10 total, 9 up, 10 in 2023-10-17T01:11:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-mon[98474]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:11:29.802 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:11:29.479+0000 7f5b28492700 -1 osd.9 0 waiting for initial osdmap 2023-10-17T01:11:29.803 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:11:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:11:29.502+0000 7f5b21281700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:11:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-17T01:11:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-17T01:11:29.823 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-10-17T01:11:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-10-17T01:11:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00855837 s, 1.2 GB/s 2023-10-17T01:11:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-10-17T01:11:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:29 smithi181 ceph-mon[134520]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:29 smithi181 ceph-mon[134520]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-17T01:11:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:29 smithi181 ceph-mon[134520]: osdmap e64: 10 total, 9 up, 10 in 2023-10-17T01:11:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:29 smithi181 ceph-mon[134520]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-10-17T01:11:30.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2023-10-17T01:11:30.739 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:11:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:30 smithi045 ceph-mon[98474]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:11:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:30 smithi045 ceph-mon[98474]: osdmap e65: 10 total, 9 up, 10 in 2023-10-17T01:11:30.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:30.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:30 smithi060 ceph-mon[134136]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:11:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:30 smithi060 ceph-mon[134136]: osdmap e65: 10 total, 9 up, 10 in 2023-10-17T01:11:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:30 smithi181 ceph-mon[134520]: from='osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-10-17T01:11:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:30 smithi181 ceph-mon[134520]: osdmap e65: 10 total, 9 up, 10 in 2023-10-17T01:11:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:31.740 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:11:31.740 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:11:31.741 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:31.741 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] boot 2023-10-17T01:11:31.741 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: osdmap e66: 10 total, 10 up, 10 in 2023-10-17T01:11:31.741 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:31.741 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.742 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.742 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.742 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.742 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.742 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.743 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.743 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:31.743 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:11:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:11:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] boot 2023-10-17T01:11:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: osdmap e66: 10 total, 10 up, 10 in 2023-10-17T01:11:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:31.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:11:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:11:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-17T01:11:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: osd.9 [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] boot 2023-10-17T01:11:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: osdmap e66: 10 total, 10 up, 10 in 2023-10-17T01:11:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-17T01:11:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:31.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:31.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:31.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.944+0000 7f053359e700 1 -- 172.21.15.60:0/3932808078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0534100e90 msgr2=0x7f0534107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.944+0000 7f053359e700 1 --2- 172.21.15.60:0/3932808078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0534100e90 0x7f0534107740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c240 tx=0x7f052c00c550 comp rx=0 tx=0).stop 2023-10-17T01:11:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 -- 172.21.15.60:0/3932808078 shutdown_connections 2023-10-17T01:11:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 --2- 172.21.15.60:0/3932808078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0534100e90 0x7f0534107740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 --2- 172.21.15.60:0/3932808078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05340ff2f0 0x7f05340ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 --2- 172.21.15.60:0/3932808078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05341004f0 0x7f0534100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 -- 172.21.15.60:0/3932808078 >> 172.21.15.60:0/3932808078 conn(0x7f05340fa870 msgr2=0x7f05340fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:11:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 -- 172.21.15.60:0/3932808078 shutdown_connections 2023-10-17T01:11:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 -- 172.21.15.60:0/3932808078 wait complete. 2023-10-17T01:11:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 Processor -- start 2023-10-17T01:11:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.945+0000 7f053359e700 1 -- start start 2023-10-17T01:11:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05340ff2f0 0x7f0534196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 0x7f05341972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0534100e90 0x7f053419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053419e930 con 0x7f0534100e90 2023-10-17T01:11:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053419ea70 con 0x7f05341004f0 2023-10-17T01:11:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053419ec00 con 0x7f05340ff2f0 2023-10-17T01:11:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 0x7f05341972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05340ff2f0 0x7f0534196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 0x7f05341972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37688/0 (socket says 172.21.15.60:37688) 2023-10-17T01:11:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 -- 172.21.15.60:0/1233384769 learned_addr learned my addr 172.21.15.60:0/1233384769 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-10-17T01:11:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0532d9d700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0534100e90 0x7f053419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 -- 172.21.15.60:0/1233384769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05340ff2f0 msgr2=0x7f0534196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05340ff2f0 0x7f0534196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 -- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0534100e90 msgr2=0x7f053419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0534100e90 0x7f053419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 -- 172.21.15.60:0/1233384769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c00ae10 con 0x7f05341004f0 2023-10-17T01:11:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0532d9d700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0534100e90 0x7f053419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:11:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f0531d9b700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 0x7f05341972e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f052800b530 tx=0x7f052800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:11:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528011830 con 0x7f05341004f0 2023-10-17T01:11:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053419eee0 con 0x7f05341004f0 2023-10-17T01:11:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0528011e50 con 0x7f05341004f0 2023-10-17T01:11:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.946+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800f2e0 con 0x7f05341004f0 2023-10-17T01:11:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.947+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053419f350 con 0x7f05341004f0 2023-10-17T01:11:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.947+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053406a450 con 0x7f05341004f0 2023-10-17T01:11:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.949+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0528010380 con 0x7f05341004f0 2023-10-17T01:11:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.949+0000 7f05237fe700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f051c0a0b20 0x7f051c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.949+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f05280cba00 con 0x7f05341004f0 2023-10-17T01:11:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.950+0000 7f053259c700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f051c0a0b20 0x7f051c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.950+0000 7f053259c700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f051c0a0b20 0x7f051c0a2fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0534100350 tx=0x7f052400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:11:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:31.953+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0528095b90 con 0x7f05341004f0 2023-10-17T01:11:32.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:32.163+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f053419f5a0 con 0x7f051c0a0b20 2023-10-17T01:11:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi045 2023-10-17T01:11:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi045 to 4319M 2023-10-17T01:11:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: osdmap e67: 10 total, 10 up, 10 in 2023-10-17T01:11:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi045 2023-10-17T01:11:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi045 to 4319M 2023-10-17T01:11:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: osdmap e67: 10 total, 10 up, 10 in 2023-10-17T01:11:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi045 2023-10-17T01:11:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi045 to 4319M 2023-10-17T01:11:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: osdmap e67: 10 total, 10 up, 10 in 2023-10-17T01:11:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:33 smithi045 ceph-mon[98474]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:33 smithi045 ceph-mon[98474]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:33 smithi060 ceph-mon[134136]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:33 smithi060 ceph-mon[134136]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:33 smithi181 ceph-mon[134520]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:33 smithi181 ceph-mon[134520]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:35.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:35.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/2242648630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]: dispatch 2023-10-17T01:11:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]: dispatch 2023-10-17T01:11:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]': finished 2023-10-17T01:11:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: osdmap e68: 11 total, 10 up, 11 in 2023-10-17T01:11:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:35.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:35 smithi181 ceph-mon[134520]: from='client.? 172.21.15.60:0/1232810878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/2242648630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]: dispatch 2023-10-17T01:11:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]: dispatch 2023-10-17T01:11:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]': finished 2023-10-17T01:11:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: osdmap e68: 11 total, 10 up, 11 in 2023-10-17T01:11:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:35.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:35 smithi045 ceph-mon[98474]: from='client.? 172.21.15.60:0/1232810878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/2242648630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]: dispatch 2023-10-17T01:11:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]: dispatch 2023-10-17T01:11:35.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50b41c1b-61b5-4b1b-9c52-93d9adb7e080"}]': finished 2023-10-17T01:11:35.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: osdmap e68: 11 total, 10 up, 11 in 2023-10-17T01:11:35.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:35.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:35 smithi060 ceph-mon[134136]: from='client.? 172.21.15.60:0/1232810878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:37 smithi045 ceph-mon[98474]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:37 smithi060 ceph-mon[134136]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:37.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:37 smithi181 ceph-mon[134520]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:39.640 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:39 smithi060 ceph-mon[134136]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:39 smithi045 ceph-mon[98474]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:39 smithi181 ceph-mon[134520]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:41 smithi045 ceph-mon[98474]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-17T01:11:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:41.823 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:41 smithi060 ceph-mon[134136]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:41.824 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-17T01:11:41.824 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:41 smithi181 ceph-mon[134520]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-17T01:11:41.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:42.707 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:42 smithi060 ceph-mon[134136]: Deploying daemon osd.10 on smithi060 2023-10-17T01:11:42.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:42 smithi045 ceph-mon[98474]: Deploying daemon osd.10 on smithi060 2023-10-17T01:11:42.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:42 smithi181 ceph-mon[134520]: Deploying daemon osd.10 on smithi060 2023-10-17T01:11:43.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:43 smithi060 ceph-mon[134136]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:43 smithi045 ceph-mon[98474]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:43 smithi181 ceph-mon[134520]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:45.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:45 smithi181 ceph-mon[134520]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:45.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:45.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:45.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:45 smithi045 ceph-mon[98474]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:45.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:45 smithi060 ceph-mon[134136]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:47 smithi045 ceph-mon[98474]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:47 smithi060 ceph-mon[134136]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:47 smithi181 ceph-mon[134520]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.923+0000 7f05237fe700 1 -- 172.21.15.60:0/1233384769 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f053419f5a0 con 0x7f051c0a0b20 2023-10-17T01:11:47.925 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 10 on host 'smithi060' 2023-10-17T01:11:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.925+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f051c0a0b20 msgr2=0x7f051c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.925+0000 7f053359e700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f051c0a0b20 0x7f051c0a2fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0534100350 tx=0x7f052400a3b0 comp rx=0 tx=0).stop 2023-10-17T01:11:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.925+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 msgr2=0x7f05341972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.925+0000 7f053359e700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 0x7f05341972e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f052800b530 tx=0x7f052800b840 comp rx=0 tx=0).stop 2023-10-17T01:11:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 shutdown_connections 2023-10-17T01:11:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f051c0a0b20 0x7f051c0a2fe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0534100e90 0x7f053419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05340ff2f0 0x7f0534196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 --2- 172.21.15.60:0/1233384769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05341004f0 0x7f05341972e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 >> 172.21.15.60:0/1233384769 conn(0x7f05340fa870 msgr2=0x7f0534105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:11:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 shutdown_connections 2023-10-17T01:11:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-10-17T01:11:47.926+0000 7f053359e700 1 -- 172.21.15.60:0/1233384769 wait complete. 2023-10-17T01:11:48.335 DEBUG:teuthology.orchestra.run.smithi060:osd.10> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.10.service 2023-10-17T01:11:48.338 INFO:tasks.cephadm:Deploying osd.11 on smithi181 with /dev/vg_nvme/lv_1... 2023-10-17T01:11:48.338 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-17T01:11:48.371 INFO:journalctl@ceph.osd.10.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:11:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:48.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:48.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:48.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:48 smithi181 ceph-mon[134520]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:48.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:48.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:48.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:48.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:48 smithi060 ceph-mon[134136]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:48.901 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:11:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:11:49.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:48 smithi045 ceph-mon[98474]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:49.578 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:11:49 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:11:49.106+0000 7f186c7be640 -1 osd.10 0 log_to_monitors true 2023-10-17T01:11:50.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:49 smithi181 ceph-mon[134520]: from='osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-17T01:11:50.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:49 smithi181 ceph-mon[134520]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-17T01:11:50.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:49 smithi045 ceph-mon[98474]: from='osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-17T01:11:50.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:49 smithi045 ceph-mon[98474]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-17T01:11:50.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:49 smithi060 ceph-mon[134136]: from='osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-17T01:11:50.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:49 smithi060 ceph-mon[134136]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-17T01:11:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-17T01:11:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-17T01:11:50.353 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-10-17T01:11:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-10-17T01:11:50.354 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145748 s, 719 MB/s 2023-10-17T01:11:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-10-17T01:11:50.740 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2023-10-17T01:11:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:50 smithi181 ceph-mon[134520]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-17T01:11:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:50 smithi181 ceph-mon[134520]: osdmap e69: 11 total, 10 up, 11 in 2023-10-17T01:11:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:50 smithi181 ceph-mon[134520]: from='osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:11:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:50 smithi181 ceph-mon[134520]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:11:51.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:50 smithi181 ceph-mon[134520]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:51.292 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:11:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:50 smithi045 ceph-mon[98474]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-17T01:11:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:50 smithi045 ceph-mon[98474]: osdmap e69: 11 total, 10 up, 11 in 2023-10-17T01:11:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:51.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:50 smithi045 ceph-mon[98474]: from='osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:11:51.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:50 smithi045 ceph-mon[98474]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:11:51.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:50 smithi045 ceph-mon[98474]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:51.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:11:50.926+0000 7f1861e35700 -1 osd.10 0 waiting for initial osdmap 2023-10-17T01:11:51.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:11:50.946+0000 7f185a423700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:11:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-mon[134136]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-17T01:11:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-mon[134136]: osdmap e69: 11 total, 10 up, 11 in 2023-10-17T01:11:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-mon[134136]: from='osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:11:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-mon[134136]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-10-17T01:11:51.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:50 smithi060 ceph-mon[134136]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-17T01:11:52.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:11:52.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: osdmap e70: 11 total, 10 up, 11 in 2023-10-17T01:11:52.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:52.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:52.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi060 2023-10-17T01:11:52.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi060 to 4293M 2023-10-17T01:11:52.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:52.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:52.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:51 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:11:52.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: osdmap e70: 11 total, 10 up, 11 in 2023-10-17T01:11:52.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:52.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:52.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi060 2023-10-17T01:11:52.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi060 to 4293M 2023-10-17T01:11:52.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:52.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:52.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:51 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-10-17T01:11:52.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: osdmap e70: 11 total, 10 up, 11 in 2023-10-17T01:11:52.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:52.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:52.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi060 2023-10-17T01:11:52.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:11:52.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi060 to 4293M 2023-10-17T01:11:52.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:52.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:11:52.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:51 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:11:52.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 -- 172.21.15.181:0/1184058605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2901012c0 msgr2=0x7fa2901016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 --2- 172.21.15.181:0/1184058605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2901012c0 0x7fa2901016e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa284009990 tx=0x7fa284009ca0 comp rx=0 tx=0).stop 2023-10-17T01:11:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 -- 172.21.15.181:0/1184058605 shutdown_connections 2023-10-17T01:11:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 --2- 172.21.15.181:0/1184058605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa290100910 0x7fa2901081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 --2- 172.21.15.181:0/1184058605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2901012c0 0x7fa2901016e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:52.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 --2- 172.21.15.181:0/1184058605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2900fff70 0x7fa2901003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 -- 172.21.15.181:0/1184058605 >> 172.21.15.181:0/1184058605 conn(0x7fa2900faa00 msgr2=0x7fa2900fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:11:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 -- 172.21.15.181:0/1184058605 shutdown_connections 2023-10-17T01:11:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 -- 172.21.15.181:0/1184058605 wait complete. 2023-10-17T01:11:52.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.407+0000 7fa2960c2700 1 Processor -- start 2023-10-17T01:11:52.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 -- start start 2023-10-17T01:11:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 0x7fa290103d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa290100910 0x7fa290102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901012c0 0x7fa2901028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa290103020 con 0x7fa290100910 2023-10-17T01:11:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa290103190 con 0x7fa2901012c0 2023-10-17T01:11:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa2960c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2901a0ad0 con 0x7fa2900fff70 2023-10-17T01:11:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 0x7fa290103d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 0x7fa290103d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45442/0 (socket says 172.21.15.181:45442) 2023-10-17T01:11:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 -- 172.21.15.181:0/2566660963 learned_addr learned my addr 172.21.15.181:0/2566660963 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:11:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28ffff700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901012c0 0x7fa2901028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 -- 172.21.15.181:0/2566660963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901012c0 msgr2=0x7fa2901028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901012c0 0x7fa2901028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 -- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa290100910 msgr2=0x7fa290102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:11:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa290100910 0x7fa290102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:11:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 -- 172.21.15.181:0/2566660963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840095f0 con 0x7fa2900fff70 2023-10-17T01:11:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.408+0000 7fa28f7fe700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 0x7fa290103d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2840098e0 tx=0x7fa28400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:11:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.409+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28401d070 con 0x7fa2900fff70 2023-10-17T01:11:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.409+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa284022470 con 0x7fa2900fff70 2023-10-17T01:11:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.409+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2901a0d50 con 0x7fa2900fff70 2023-10-17T01:11:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.409+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284017650 con 0x7fa2900fff70 2023-10-17T01:11:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.409+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2901a11c0 con 0x7fa2900fff70 2023-10-17T01:11:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.410+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29010b4d0 con 0x7fa2900fff70 2023-10-17T01:11:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.411+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa2840225e0 con 0x7fa2900fff70 2023-10-17T01:11:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.411+0000 7fa28cff9700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa2780a0ba0 0x7fa2780a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:11:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.412+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fa2840ccd80 con 0x7fa2900fff70 2023-10-17T01:11:52.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.412+0000 7fa28effd700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa2780a0ba0 0x7fa2780a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:11:52.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.412+0000 7fa28effd700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa2780a0ba0 0x7fa2780a3060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa280005d90 tx=0x7fa280005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:11:52.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.416+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fa284096df0 con 0x7fa2900fff70 2023-10-17T01:11:52.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:11:52.613+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa29010b780 con 0x7fa2780a0ba0 2023-10-17T01:11:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:11:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:11:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] boot 2023-10-17T01:11:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: osdmap e71: 11 total, 11 up, 11 in 2023-10-17T01:11:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-10-17T01:11:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:52 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:11:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:11:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] boot 2023-10-17T01:11:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: osdmap e71: 11 total, 11 up, 11 in 2023-10-17T01:11:53.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:53.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-10-17T01:11:53.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:53.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:53.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:52 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:53.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:11:53.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:11:53.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: osd.10 [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] boot 2023-10-17T01:11:53.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: osdmap e71: 11 total, 11 up, 11 in 2023-10-17T01:11:53.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-17T01:11:53.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-10-17T01:11:53.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-17T01:11:53.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-17T01:11:53.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:52 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:11:54.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:53 smithi181 ceph-mon[134520]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:54.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:53 smithi181 ceph-mon[134520]: osdmap e72: 11 total, 11 up, 11 in 2023-10-17T01:11:54.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:53 smithi045 ceph-mon[98474]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:54.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:53 smithi045 ceph-mon[98474]: osdmap e72: 11 total, 11 up, 11 in 2023-10-17T01:11:54.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:53 smithi060 ceph-mon[134136]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:11:54.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:53 smithi060 ceph-mon[134136]: osdmap e72: 11 total, 11 up, 11 in 2023-10-17T01:11:55.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:54 smithi045 ceph-mon[98474]: osdmap e73: 11 total, 11 up, 11 in 2023-10-17T01:11:55.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:54 smithi045 ceph-mon[98474]: osdmap e74: 11 total, 11 up, 11 in 2023-10-17T01:11:55.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:54 smithi045 ceph-mon[98474]: pgmap v225: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:55.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:54 smithi060 ceph-mon[134136]: osdmap e73: 11 total, 11 up, 11 in 2023-10-17T01:11:55.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:54 smithi060 ceph-mon[134136]: osdmap e74: 11 total, 11 up, 11 in 2023-10-17T01:11:55.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:54 smithi060 ceph-mon[134136]: pgmap v225: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:55.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:54 smithi181 ceph-mon[134520]: osdmap e73: 11 total, 11 up, 11 in 2023-10-17T01:11:55.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:54 smithi181 ceph-mon[134520]: osdmap e74: 11 total, 11 up, 11 in 2023-10-17T01:11:55.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:54 smithi181 ceph-mon[134520]: pgmap v225: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:56.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/217911150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]: dispatch 2023-10-17T01:11:56.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]: dispatch 2023-10-17T01:11:56.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]': finished 2023-10-17T01:11:56.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: osdmap e75: 12 total, 11 up, 12 in 2023-10-17T01:11:56.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:11:56.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-10-17T01:11:56.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:55 smithi045 ceph-mon[98474]: from='client.? 172.21.15.181:0/3322379631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:56.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/217911150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]: dispatch 2023-10-17T01:11:56.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]: dispatch 2023-10-17T01:11:56.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]': finished 2023-10-17T01:11:56.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: osdmap e75: 12 total, 11 up, 12 in 2023-10-17T01:11:56.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:11:56.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-10-17T01:11:56.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:55 smithi060 ceph-mon[134136]: from='client.? 172.21.15.181:0/3322379631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:56.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/217911150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]: dispatch 2023-10-17T01:11:56.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]: dispatch 2023-10-17T01:11:56.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a70d229-f101-42fe-8231-5e3cc447d528"}]': finished 2023-10-17T01:11:56.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: osdmap e75: 12 total, 11 up, 12 in 2023-10-17T01:11:56.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:11:56.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-10-17T01:11:56.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:55 smithi181 ceph-mon[134520]: from='client.? 172.21.15.181:0/3322379631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-17T01:11:57.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:56 smithi045 ceph-mon[98474]: pgmap v227: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:57.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:56 smithi060 ceph-mon[134136]: pgmap v227: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:57.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:56 smithi181 ceph-mon[134520]: pgmap v227: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:11:59 smithi045 ceph-mon[98474]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:11:59 smithi060 ceph-mon[134136]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:11:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:11:59 smithi181 ceph-mon[134520]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-10-17T01:12:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:00 smithi045 ceph-mon[98474]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-10-17T01:12:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:00 smithi045 ceph-mon[98474]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-10-17T01:12:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:00 smithi045 ceph-mon[98474]: Cluster is now healthy 2023-10-17T01:12:00.804 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:00 smithi181 ceph-mon[134520]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-10-17T01:12:00.804 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:00 smithi181 ceph-mon[134520]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-10-17T01:12:00.804 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:00 smithi181 ceph-mon[134520]: Cluster is now healthy 2023-10-17T01:12:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:00 smithi060 ceph-mon[134136]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-10-17T01:12:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:00 smithi060 ceph-mon[134136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-10-17T01:12:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:00 smithi060 ceph-mon[134136]: Cluster is now healthy 2023-10-17T01:12:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-17T01:12:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:01 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:01 smithi045 ceph-mon[98474]: Deploying daemon osd.11 on smithi181 2023-10-17T01:12:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-17T01:12:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:01 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:01 smithi060 ceph-mon[134136]: Deploying daemon osd.11 on smithi181 2023-10-17T01:12:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-17T01:12:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:01 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:01 smithi181 ceph-mon[134520]: Deploying daemon osd.11 on smithi181 2023-10-17T01:12:02.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:02 smithi045 ceph-mon[98474]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:12:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:02 smithi060 ceph-mon[134136]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:12:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:02 smithi181 ceph-mon[134520]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-17T01:12:05.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:05 smithi181 ceph-mon[134520]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-17T01:12:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:05 smithi045 ceph-mon[98474]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-17T01:12:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:05 smithi060 ceph-mon[134136]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-17T01:12:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:07.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:07.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:06 smithi181 ceph-mon[134520]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-17T01:12:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:07.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:06 smithi045 ceph-mon[98474]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-17T01:12:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:06 smithi060 ceph-mon[134136]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-17T01:12:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.678+0000 7fa28cff9700 1 -- 172.21.15.181:0/2566660963 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fa29010b780 con 0x7fa2780a0ba0 2023-10-17T01:12:08.679 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 11 on host 'smithi181' 2023-10-17T01:12:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa2780a0ba0 msgr2=0x7fa2780a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa2780a0ba0 0x7fa2780a3060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa280005d90 tx=0x7fa280005d00 comp rx=0 tx=0).stop 2023-10-17T01:12:08.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 msgr2=0x7fa290103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:08.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 0x7fa290103d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2840098e0 tx=0x7fa28400f690 comp rx=0 tx=0).stop 2023-10-17T01:12:08.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 shutdown_connections 2023-10-17T01:12:08.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa2780a0ba0 0x7fa2780a3060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:08.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901012c0 0x7fa2901028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2900fff70 0x7fa290103d10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 --2- 172.21.15.181:0/2566660963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa290100910 0x7fa290102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 >> 172.21.15.181:0/2566660963 conn(0x7fa2900faa00 msgr2=0x7fa2900fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 shutdown_connections 2023-10-17T01:12:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:08.681+0000 7fa2960c2700 1 -- 172.21.15.181:0/2566660963 wait complete. 2023-10-17T01:12:09.102 DEBUG:teuthology.orchestra.run.smithi181:osd.11> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.11.service 2023-10-17T01:12:09.105 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-17T01:12:09.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-17T01:12:09.136 INFO:journalctl@ceph.osd.11.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:12:09.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:12:09.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:09.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:09.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:09.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:09 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.672 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.673 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:12:09.673 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.673 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:09.673 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:09.673 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.674 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.674 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:09.674 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:12:09.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:09.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:09.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:09.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:09 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:12:10.123 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:12:09 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:12:09.799+0000 7f8a2d442640 -1 osd.11 0 log_to_monitors true 2023-10-17T01:12:10.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:10 smithi045 ceph-mon[98474]: from='osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-17T01:12:10.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:10 smithi045 ceph-mon[98474]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-17T01:12:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:10 smithi060 ceph-mon[134136]: from='osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-17T01:12:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:10 smithi060 ceph-mon[134136]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-17T01:12:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:10 smithi181 ceph-mon[134520]: from='osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-17T01:12:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:10 smithi181 ceph-mon[134520]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-17T01:12:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 -- 172.21.15.45:0/2772520794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20100de0 msgr2=0x7f7d20101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 --2- 172.21.15.45:0/2772520794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20100de0 0x7f7d20101200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0099c0 tx=0x7f7d1c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 -- 172.21.15.45:0/2772520794 shutdown_connections 2023-10-17T01:12:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 --2- 172.21.15.45:0/2772520794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d20102980 0x7f7d20109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 --2- 172.21.15.45:0/2772520794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20100de0 0x7f7d20101200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 --2- 172.21.15.45:0/2772520794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20101fe0 0x7f7d20102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 -- 172.21.15.45:0/2772520794 >> 172.21.15.45:0/2772520794 conn(0x7f7d200fc340 msgr2=0x7f7d200fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 -- 172.21.15.45:0/2772520794 shutdown_connections 2023-10-17T01:12:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.513+0000 7f7d2749d700 1 -- 172.21.15.45:0/2772520794 wait complete. 2023-10-17T01:12:11.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.514+0000 7f7d2749d700 1 Processor -- start 2023-10-17T01:12:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.514+0000 7f7d2749d700 1 -- start start 2023-10-17T01:12:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d2749d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20100de0 0x7f7d20196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d2749d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d20101fe0 0x7f7d20197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d2749d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20102980 0x7f7d2019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d2749d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019e680 con 0x7f7d20102980 2023-10-17T01:12:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d2749d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019e7c0 con 0x7f7d20100de0 2023-10-17T01:12:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d2749d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019e950 con 0x7f7d20101fe0 2023-10-17T01:12:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25239700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20100de0 0x7f7d20196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25239700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20100de0 0x7f7d20196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33610/0 (socket says 172.21.15.45:33610) 2023-10-17T01:12:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25239700 1 -- 172.21.15.45:0/3336249187 learned_addr learned my addr 172.21.15.45:0/3336249187 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:11.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d24a38700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d20101fe0 0x7f7d20197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25a3a700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20102980 0x7f7d2019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25a3a700 1 -- 172.21.15.45:0/3336249187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d20101fe0 msgr2=0x7f7d20197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25a3a700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d20101fe0 0x7f7d20197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25a3a700 1 -- 172.21.15.45:0/3336249187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20100de0 msgr2=0x7f7d20196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25a3a700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20100de0 0x7f7d20196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.515+0000 7f7d25a3a700 1 -- 172.21.15.45:0/3336249187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0095f0 con 0x7f7d20102980 2023-10-17T01:12:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.516+0000 7f7d25a3a700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20102980 0x7f7d2019e0b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009f60 tx=0x7f7d18009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.516+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18018070 con 0x7f7d20102980 2023-10-17T01:12:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.516+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2019ebd0 con 0x7f7d20102980 2023-10-17T01:12:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.516+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1800f040 con 0x7f7d20102980 2023-10-17T01:12:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.516+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800ccc0 con 0x7f7d20102980 2023-10-17T01:12:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.516+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2019f0d0 con 0x7f7d20102980 2023-10-17T01:12:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.518+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d201083d0 con 0x7f7d20102980 2023-10-17T01:12:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.520+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f7d18003a00 con 0x7f7d20102980 2023-10-17T01:12:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.521+0000 7f7d167fc700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d0c0a0ba0 0x7f7d0c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.521+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f7d180cb630 con 0x7f7d20102980 2023-10-17T01:12:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.521+0000 7f7d25239700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d0c0a0ba0 0x7f7d0c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:11.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.522+0000 7f7d25239700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d0c0a0ba0 0x7f7d0c0a3060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000c00 tx=0x7f7d1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:11.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.526+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7d18095570 con 0x7f7d20102980 2023-10-17T01:12:11.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:11 smithi045 ceph-mon[98474]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:12:11.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:11 smithi045 ceph-mon[98474]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-17T01:12:11.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:11 smithi045 ceph-mon[98474]: osdmap e76: 12 total, 11 up, 12 in 2023-10-17T01:12:11.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:11 smithi045 ceph-mon[98474]: from='osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:12:11.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:11 smithi045 ceph-mon[98474]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:12:11.577 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:12:11.430+0000 7f8a22ab9700 -1 osd.11 0 waiting for initial osdmap 2023-10-17T01:12:11.577 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:12:11.444+0000 7f8a1a8a6700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-17T01:12:11.577 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-mon[134520]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:12:11.578 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-mon[134520]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-17T01:12:11.578 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-mon[134520]: osdmap e76: 12 total, 11 up, 12 in 2023-10-17T01:12:11.578 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:11.578 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-mon[134520]: from='osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:12:11.579 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:11 smithi181 ceph-mon[134520]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:12:11.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.750+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7d2002cc30 con 0x7f7d20102980 2023-10-17T01:12:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.750+0000 7f7d167fc700 1 -- 172.21.15.45:0/3336249187 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7d18098b90 con 0x7f7d20102980 2023-10-17T01:12:11.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d0c0a0ba0 msgr2=0x7f7d0c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d0c0a0ba0 0x7f7d0c0a3060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000c00 tx=0x7f7d1c005fb0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20102980 msgr2=0x7f7d2019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20102980 0x7f7d2019e0b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009f60 tx=0x7f7d18009f90 comp rx=0 tx=0).stop 2023-10-17T01:12:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 shutdown_connections 2023-10-17T01:12:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d0c0a0ba0 0x7f7d0c0a3060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d20102980 0x7f7d2019e0b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d20100de0 0x7f7d20196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 --2- 172.21.15.45:0/3336249187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d20101fe0 0x7f7d20197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 >> 172.21.15.45:0/3336249187 conn(0x7f7d200fc340 msgr2=0x7f7d20105510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 shutdown_connections 2023-10-17T01:12:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:11.753+0000 7f7d2749d700 1 -- 172.21.15.45:0/3336249187 wait complete. 2023-10-17T01:12:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:11 smithi060 ceph-mon[134136]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-17T01:12:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:11 smithi060 ceph-mon[134136]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-17T01:12:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:11 smithi060 ceph-mon[134136]: osdmap e76: 12 total, 11 up, 12 in 2023-10-17T01:12:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:11 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:11.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:11 smithi060 ceph-mon[134136]: from='osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:12:11.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:11 smithi060 ceph-mon[134136]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-10-17T01:12:12.237 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1697505111,"num_in_osds":12,"osd_in_since":1697505114,"num_remapped_pgs":0} 2023-10-17T01:12:12.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:12:12.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: osdmap e77: 12 total, 11 up, 12 in 2023-10-17T01:12:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:12.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:12.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:12 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3336249187' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-17T01:12:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:12:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: osdmap e77: 12 total, 11 up, 12 in 2023-10-17T01:12:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:12.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:12.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:12.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:12.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:12 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3336249187' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-17T01:12:12.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-10-17T01:12:12.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: osdmap e77: 12 total, 11 up, 12 in 2023-10-17T01:12:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:12:12.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:12.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:12.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:12.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:12 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3336249187' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-17T01:12:13.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-17T01:12:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: purged_snaps scrub starts 2023-10-17T01:12:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: purged_snaps scrub ok 2023-10-17T01:12:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: Detected new or changed devices on smithi181 2023-10-17T01:12:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi181 to 4292M 2023-10-17T01:12:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-17T01:12:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:13.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] boot 2023-10-17T01:12:13.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: osdmap e78: 12 total, 12 up, 12 in 2023-10-17T01:12:13.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:13 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: purged_snaps scrub starts 2023-10-17T01:12:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: purged_snaps scrub ok 2023-10-17T01:12:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: Detected new or changed devices on smithi181 2023-10-17T01:12:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi181 to 4292M 2023-10-17T01:12:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-17T01:12:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] boot 2023-10-17T01:12:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: osdmap e78: 12 total, 12 up, 12 in 2023-10-17T01:12:13.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:13 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: purged_snaps scrub starts 2023-10-17T01:12:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: purged_snaps scrub ok 2023-10-17T01:12:13.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: Detected new or changed devices on smithi181 2023-10-17T01:12:13.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi181 to 4292M 2023-10-17T01:12:13.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-17T01:12:13.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:13.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: osd.11 [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] boot 2023-10-17T01:12:13.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: osdmap e78: 12 total, 12 up, 12 in 2023-10-17T01:12:13.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:13 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-17T01:12:13.990 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:12:14.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:14 smithi045 ceph-mon[98474]: osdmap e79: 12 total, 12 up, 12 in 2023-10-17T01:12:14.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:14 smithi045 ceph-mon[98474]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:15.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:14 smithi060 ceph-mon[134136]: osdmap e79: 12 total, 12 up, 12 in 2023-10-17T01:12:15.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:14 smithi060 ceph-mon[134136]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:14 smithi181 ceph-mon[134520]: osdmap e79: 12 total, 12 up, 12 in 2023-10-17T01:12:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:14 smithi181 ceph-mon[134520]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:15.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 -- 172.21.15.45:0/3330079331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 msgr2=0x7f9304109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 --2- 172.21.15.45:0/3330079331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f9304109230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b0d0 tx=0x7f92fc00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 -- 172.21.15.45:0/3330079331 shutdown_connections 2023-10-17T01:12:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 --2- 172.21.15.45:0/3330079331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f9304109230 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 --2- 172.21.15.45:0/3330079331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9304100de0 0x7f9304101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 --2- 172.21.15.45:0/3330079331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 0x7f9304102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.583+0000 7f930b73e700 1 -- 172.21.15.45:0/3330079331 >> 172.21.15.45:0/3330079331 conn(0x7f93040fc340 msgr2=0x7f93040fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.584+0000 7f930b73e700 1 -- 172.21.15.45:0/3330079331 shutdown_connections 2023-10-17T01:12:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.584+0000 7f930b73e700 1 -- 172.21.15.45:0/3330079331 wait complete. 2023-10-17T01:12:15.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 Processor -- start 2023-10-17T01:12:15.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 -- start start 2023-10-17T01:12:15.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9304100de0 0x7f9304198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 0x7f9304199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f93041a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a0960 con 0x7f9304102980 2023-10-17T01:12:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a0aa0 con 0x7f9304101fe0 2023-10-17T01:12:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f930b73e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a0c30 con 0x7f9304100de0 2023-10-17T01:12:15.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f9309cdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f93041a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:15.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f9308cd9700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 0x7f9304199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:15.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f9309cdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f93041a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40110/0 (socket says 172.21.15.45:40110) 2023-10-17T01:12:15.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.585+0000 7f9309cdb700 1 -- 172.21.15.45:0/3748627155 learned_addr learned my addr 172.21.15.45:0/3748627155 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:15.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f93094da700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9304100de0 0x7f9304198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:15.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9308cd9700 1 -- 172.21.15.45:0/3748627155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9304100de0 msgr2=0x7f9304198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:15.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9308cd9700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9304100de0 0x7f9304198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9308cd9700 1 -- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 msgr2=0x7f93041a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:15.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9308cd9700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f93041a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9308cd9700 1 -- 172.21.15.45:0/3748627155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc009d00 con 0x7f9304101fe0 2023-10-17T01:12:15.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9309cdb700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f93041a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:15.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f9308cd9700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 0x7f9304199310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e9f0 tx=0x7f92f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:15.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400c9c0 con 0x7f9304101fe0 2023-10-17T01:12:15.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f4010430 con 0x7f9304101fe0 2023-10-17T01:12:15.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93041a0f10 con 0x7f9304101fe0 2023-10-17T01:12:15.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.586+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4018680 con 0x7f9304101fe0 2023-10-17T01:12:15.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.587+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93041a1380 con 0x7f9304101fe0 2023-10-17T01:12:15.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.587+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930410c640 con 0x7f9304101fe0 2023-10-17T01:12:15.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.590+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f92f4010860 con 0x7f9304101fe0 2023-10-17T01:12:15.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.591+0000 7f92fa7fc700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f92f00a0b20 0x7f92f00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.591+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f92f4014070 con 0x7f9304101fe0 2023-10-17T01:12:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.591+0000 7f93094da700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f92f00a0b20 0x7f92f00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.592+0000 7f93094da700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f92f00a0b20 0x7f92f00a2fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9304101e40 tx=0x7f9300009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:15.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.593+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f92f4095ed0 con 0x7f9304101fe0 2023-10-17T01:12:15.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.805+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f93041a1990 con 0x7f9304101fe0 2023-10-17T01:12:15.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.806+0000 7f92fa7fc700 1 -- 172.21.15.45:0/3748627155 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f92f40994f0 con 0x7f9304101fe0 2023-10-17T01:12:15.807 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f92f00a0b20 msgr2=0x7f92f00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f92f00a0b20 0x7f92f00a2fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9304101e40 tx=0x7f9300009260 comp rx=0 tx=0).stop 2023-10-17T01:12:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 msgr2=0x7f9304199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 0x7f9304199310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e9f0 tx=0x7f92f400ed00 comp rx=0 tx=0).stop 2023-10-17T01:12:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 shutdown_connections 2023-10-17T01:12:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f92f00a0b20 0x7f92f00a2fe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9304102980 0x7f93041a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9304100de0 0x7f9304198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 --2- 172.21.15.45:0/3748627155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9304101fe0 0x7f9304199310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 >> 172.21.15.45:0/3748627155 conn(0x7f93040fc340 msgr2=0x7f9304107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 shutdown_connections 2023-10-17T01:12:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:15.808+0000 7f930b73e700 1 -- 172.21.15.45:0/3748627155 wait complete. 2023-10-17T01:12:16.274 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1697505132,"num_in_osds":12,"osd_in_since":1697505114,"num_remapped_pgs":0} 2023-10-17T01:12:16.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-17T01:12:16.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:16 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3748627155' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-17T01:12:16.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:16 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3748627155' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-17T01:12:16.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:16 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3748627155' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-17T01:12:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:12:17.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:17 smithi045 ceph-mon[98474]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:17.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:17 smithi181 ceph-mon[134520]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:17 smithi060 ceph-mon[134136]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 -- 172.21.15.45:0/2797984392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e41060b0 msgr2=0x7ff3e410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2797984392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e41060b0 0x7ff3e410dc80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00b0a0 tx=0x7ff3dc00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 -- 172.21.15.45:0/2797984392 shutdown_connections 2023-10-17T01:12:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2797984392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e41060b0 0x7ff3e410dc80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2797984392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e4100fd0 0x7ff3e41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2797984392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e4101980 0x7ff3e41059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 -- 172.21.15.45:0/2797984392 >> 172.21.15.45:0/2797984392 conn(0x7ff3e40fc840 msgr2=0x7ff3e40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 -- 172.21.15.45:0/2797984392 shutdown_connections 2023-10-17T01:12:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.609+0000 7ff3eb732700 1 -- 172.21.15.45:0/2797984392 wait complete. 2023-10-17T01:12:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.610+0000 7ff3eb732700 1 Processor -- start 2023-10-17T01:12:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 -- start start 2023-10-17T01:12:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e4100fd0 0x7ff3e4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e4101980 0x7ff3e41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 0x7ff3e419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e419e150 con 0x7ff3e4100fd0 2023-10-17T01:12:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e419e2c0 con 0x7ff3e41060b0 2023-10-17T01:12:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3eb732700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e419e480 con 0x7ff3e4101980 2023-10-17T01:12:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3e94ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e4100fd0 0x7ff3e4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3e9ccf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 0x7ff3e419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3e94ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e4100fd0 0x7ff3e4199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55122/0 (socket says 172.21.15.45:55122) 2023-10-17T01:12:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3e9ccf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 0x7ff3e419db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33228/0 (socket says 172.21.15.45:33228) 2023-10-17T01:12:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3e94ce700 1 -- 172.21.15.45:0/2129042912 learned_addr learned my addr 172.21.15.45:0/2129042912 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.611+0000 7ff3e8ccd700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e4101980 0x7ff3e41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3e9ccf700 1 -- 172.21.15.45:0/2129042912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e4101980 msgr2=0x7ff3e41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3e9ccf700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e4101980 0x7ff3e41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3e9ccf700 1 -- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e4100fd0 msgr2=0x7ff3e4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3e9ccf700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e4100fd0 0x7ff3e4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3e9ccf700 1 -- 172.21.15.45:0/2129042912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc009d00 con 0x7ff3e41060b0 2023-10-17T01:12:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3e9ccf700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 0x7ff3e419db40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009fd0 tx=0x7ff3dc007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00e040 con 0x7ff3e41060b0 2023-10-17T01:12:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e419e700 con 0x7ff3e41060b0 2023-10-17T01:12:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3dc01d470 con 0x7ff3e41060b0 2023-10-17T01:12:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0052a0 con 0x7ff3e41060b0 2023-10-17T01:12:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.612+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e41a5fc0 con 0x7ff3e41060b0 2023-10-17T01:12:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.613+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e404ea50 con 0x7ff3e41060b0 2023-10-17T01:12:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.615+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff3dc01d5e0 con 0x7ff3e41060b0 2023-10-17T01:12:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.617+0000 7ff3da7fc700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff3d00a0b20 0x7ff3d00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.617+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff3dc0cd780 con 0x7ff3e41060b0 2023-10-17T01:12:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.617+0000 7ff3e94ce700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff3d00a0b20 0x7ff3d00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.617+0000 7ff3e94ce700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff3d00a0b20 0x7ff3d00a2fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009d20 tx=0x7ff3e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.619+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7ff3dc0975a0 con 0x7ff3e41060b0 2023-10-17T01:12:18.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.831+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3e41a6150 con 0x7ff3e41060b0 2023-10-17T01:12:18.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.832+0000 7ff3da7fc700 1 -- 172.21.15.45:0/2129042912 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18362 (secure 0 0 0) 0x7ff3dc09abc0 con 0x7ff3e41060b0 2023-10-17T01:12:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:18.835 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:12:13.667312+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:12:18.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff3d00a0b20 msgr2=0x7ff3d00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:18.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff3d00a0b20 0x7ff3d00a2fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009d20 tx=0x7ff3e0009260 comp rx=0 tx=0).stop 2023-10-17T01:12:18.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 msgr2=0x7ff3e419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:18.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 0x7ff3e419db40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009fd0 tx=0x7ff3dc007930 comp rx=0 tx=0).stop 2023-10-17T01:12:18.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 shutdown_connections 2023-10-17T01:12:18.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff3d00a0b20 0x7ff3d00a2fe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e41060b0 0x7ff3e419db40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e4100fd0 0x7ff3e4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 --2- 172.21.15.45:0/2129042912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e4101980 0x7ff3e41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:18.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 >> 172.21.15.45:0/2129042912 conn(0x7ff3e40fc840 msgr2=0x7ff3e40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:18.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 shutdown_connections 2023-10-17T01:12:18.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:18.835+0000 7ff3eb732700 1 -- 172.21.15.45:0/2129042912 wait complete. 2023-10-17T01:12:19.303 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:12:19.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-17T01:12:19.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:19 smithi045 ceph-mon[98474]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:19.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:19 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2129042912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:19 smithi060 ceph-mon[134136]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:19 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2129042912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:19 smithi181 ceph-mon[134520]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:19 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2129042912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:20.044 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:12:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3312101570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a241060b0 msgr2=0x7f0a2410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3312101570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a241060b0 0x7f0a2410dc80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b0a0 tx=0x7f0a1c00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3312101570 shutdown_connections 2023-10-17T01:12:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3312101570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a241060b0 0x7f0a2410dc80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3312101570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a24100fd0 0x7f0a241013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3312101570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24101980 0x7f0a241059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3312101570 >> 172.21.15.45:0/3312101570 conn(0x7f0a240fc840 msgr2=0x7f0a240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3312101570 shutdown_connections 2023-10-17T01:12:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.628+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3312101570 wait complete. 2023-10-17T01:12:21.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.629+0000 7f0a2a6a1700 1 Processor -- start 2023-10-17T01:12:21.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.629+0000 7f0a2a6a1700 1 -- start start 2023-10-17T01:12:21.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a2a6a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 0x7f0a24196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:21.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a2a6a1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a24101980 0x7f0a241974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a2a6a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a241060b0 0x7f0a2419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a2a6a1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419bf40 con 0x7f0a241060b0 2023-10-17T01:12:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a2a6a1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419c0b0 con 0x7f0a24101980 2023-10-17T01:12:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a2a6a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419c270 con 0x7f0a24100fd0 2023-10-17T01:12:21.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 0x7f0a24196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:21.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 0x7f0a24196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50040/0 (socket says 172.21.15.45:50040) 2023-10-17T01:12:21.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 -- 172.21.15.45:0/3598558617 learned_addr learned my addr 172.21.15.45:0/3598558617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:21.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a28c3e700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a241060b0 0x7f0a2419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:21.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a237fe700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a24101980 0x7f0a241974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 -- 172.21.15.45:0/3598558617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a24101980 msgr2=0x7f0a241974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a24101980 0x7f0a241974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 -- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a241060b0 msgr2=0x7f0a2419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a241060b0 0x7f0a2419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.630+0000 7f0a23fff700 1 -- 172.21.15.45:0/3598558617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c009d00 con 0x7f0a24100fd0 2023-10-17T01:12:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a28c3e700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a241060b0 0x7f0a2419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a23fff700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 0x7f0a24196f80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400e9a0 tx=0x7f0a1400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14009790 con 0x7f0a24100fd0 2023-10-17T01:12:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2419c550 con 0x7f0a24100fd0 2023-10-17T01:12:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a14010430 con 0x7f0a24100fd0 2023-10-17T01:12:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14018680 con 0x7f0a24100fd0 2023-10-17T01:12:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.631+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a241a3db0 con 0x7f0a24100fd0 2023-10-17T01:12:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.632+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2404ea50 con 0x7f0a24100fd0 2023-10-17T01:12:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.634+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0a140105a0 con 0x7f0a24100fd0 2023-10-17T01:12:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.636+0000 7f0a217fa700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0a0c0a0ba0 0x7f0a0c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.636+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a14014070 con 0x7f0a24100fd0 2023-10-17T01:12:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.636+0000 7f0a237fe700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0a0c0a0ba0 0x7f0a0c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.636+0000 7f0a237fe700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0a0c0a0ba0 0x7f0a0c0a3060 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a241985a0 tx=0x7f0a18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.640+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0a140520b0 con 0x7f0a24100fd0 2023-10-17T01:12:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:21 smithi045 ceph-mon[98474]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:21.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:21 smithi060 ceph-mon[134136]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.849+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a241a40a0 con 0x7f0a24100fd0 2023-10-17T01:12:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.849+0000 7f0a217fa700 1 -- 172.21.15.45:0/3598558617 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a14095c90 con 0x7f0a24100fd0 2023-10-17T01:12:21.852 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:12:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0a0c0a0ba0 msgr2=0x7f0a0c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0a0c0a0ba0 0x7f0a0c0a3060 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a241985a0 tx=0x7f0a18009260 comp rx=0 tx=0).stop 2023-10-17T01:12:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 msgr2=0x7f0a24196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 0x7f0a24196f80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400e9a0 tx=0x7f0a1400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 shutdown_connections 2023-10-17T01:12:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0a0c0a0ba0 0x7f0a0c0a3060 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a241060b0 0x7f0a2419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a24100fd0 0x7f0a24196f80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 --2- 172.21.15.45:0/3598558617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a24101980 0x7f0a241974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 >> 172.21.15.45:0/3598558617 conn(0x7f0a240fc840 msgr2=0x7f0a240fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 shutdown_connections 2023-10-17T01:12:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:21.852+0000 7f0a2a6a1700 1 -- 172.21.15.45:0/3598558617 wait complete. 2023-10-17T01:12:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:21 smithi181 ceph-mon[134520]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:22.425 INFO:tasks.cephadm:Adding mds.a on smithi045 2023-10-17T01:12:22.426 INFO:tasks.cephadm:Adding mds.d on smithi045 2023-10-17T01:12:22.426 INFO:tasks.cephadm:Adding mds.g on smithi045 2023-10-17T01:12:22.426 INFO:tasks.cephadm:Adding mds.j on smithi045 2023-10-17T01:12:22.427 INFO:tasks.cephadm:Adding mds.b on smithi060 2023-10-17T01:12:22.427 INFO:tasks.cephadm:Adding mds.e on smithi060 2023-10-17T01:12:22.427 INFO:tasks.cephadm:Adding mds.h on smithi060 2023-10-17T01:12:22.427 INFO:tasks.cephadm:Adding mds.k on smithi060 2023-10-17T01:12:22.428 INFO:tasks.cephadm:Adding mds.c on smithi181 2023-10-17T01:12:22.428 INFO:tasks.cephadm:Adding mds.f on smithi181 2023-10-17T01:12:22.428 INFO:tasks.cephadm:Adding mds.i on smithi181 2023-10-17T01:12:22.428 INFO:tasks.cephadm:Adding mds.l on smithi181 2023-10-17T01:12:22.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l' 2023-10-17T01:12:22.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:22 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3598558617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:22.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:22 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3598558617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:22.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:22 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3598558617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:12:23.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:23 smithi181 ceph-mon[134520]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:23 smithi045 ceph-mon[98474]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:23 smithi060 ceph-mon[134136]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 -- 172.21.15.181:0/328742876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308102930 msgr2=0x7ff308109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 --2- 172.21.15.181:0/328742876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308102930 0x7ff308109250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff30000b0a0 tx=0x7ff30000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 -- 172.21.15.181:0/328742876 shutdown_connections 2023-10-17T01:12:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 --2- 172.21.15.181:0/328742876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308102930 0x7ff308109250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 --2- 172.21.15.181:0/328742876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff308100e10 0x7ff308101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 --2- 172.21.15.181:0/328742876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff308101f70 0x7ff3081023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.211+0000 7ff30ee07700 1 -- 172.21.15.181:0/328742876 >> 172.21.15.181:0/328742876 conn(0x7ff3080fc3d0 msgr2=0x7ff3080fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 -- 172.21.15.181:0/328742876 shutdown_connections 2023-10-17T01:12:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 -- 172.21.15.181:0/328742876 wait complete. 2023-10-17T01:12:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 Processor -- start 2023-10-17T01:12:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 -- start start 2023-10-17T01:12:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff308100e10 0x7ff308074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 0x7ff308073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff308102930 0x7ff308073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308073e00 con 0x7ff308100e10 2023-10-17T01:12:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308073f70 con 0x7ff308102930 2023-10-17T01:12:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff30ee07700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081a0b40 con 0x7ff308101f70 2023-10-17T01:12:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff307fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 0x7ff308073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff307fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 0x7ff308073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45572/0 (socket says 172.21.15.181:45572) 2023-10-17T01:12:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.212+0000 7ff307fff700 1 -- 172.21.15.181:0/2070647212 learned_addr learned my addr 172.21.15.181:0/2070647212 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-17T01:12:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff30d3a4700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff308102930 0x7ff308073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff30cba3700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff308100e10 0x7ff308074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff307fff700 1 -- 172.21.15.181:0/2070647212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff308102930 msgr2=0x7ff308073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff307fff700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff308102930 0x7ff308073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff307fff700 1 -- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff308100e10 msgr2=0x7ff308074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff307fff700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff308100e10 0x7ff308074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff307fff700 1 -- 172.21.15.181:0/2070647212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300009d00 con 0x7ff308101f70 2023-10-17T01:12:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff307fff700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 0x7ff308073140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400b530 tx=0x7ff2f400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011830 con 0x7ff308101f70 2023-10-17T01:12:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3081a0e20 con 0x7ff308101f70 2023-10-17T01:12:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2f4011e50 con 0x7ff308101f70 2023-10-17T01:12:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f2e0 con 0x7ff308101f70 2023-10-17T01:12:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.213+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081a1310 con 0x7ff308101f70 2023-10-17T01:12:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.214+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30810c660 con 0x7ff308101f70 2023-10-17T01:12:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.215+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff2f4010380 con 0x7ff308101f70 2023-10-17T01:12:24.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.216+0000 7ff305ffb700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff2f80a0b20 0x7ff2f80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.216+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2f40cb860 con 0x7ff308101f70 2023-10-17T01:12:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.216+0000 7ff30cba3700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff2f80a0b20 0x7ff2f80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.217+0000 7ff30cba3700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff2f80a0b20 0x7ff2f80a2fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006fd0 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:24.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.220+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7ff2f4095690 con 0x7ff308101f70 2023-10-17T01:12:24.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.418+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}) v1 -- 0x7ff308061290 con 0x7ff2f80a0b20 2023-10-17T01:12:24.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.427+0000 7ff305ffb700 1 -- 172.21.15.181:0/2070647212 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff308061290 con 0x7ff2f80a0b20 2023-10-17T01:12:24.428 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mds.all update... 2023-10-17T01:12:24.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff2f80a0b20 msgr2=0x7ff2f80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:24.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff2f80a0b20 0x7ff2f80a2fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006fd0 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2023-10-17T01:12:24.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 msgr2=0x7ff308073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:24.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 0x7ff308073140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400b530 tx=0x7ff2f400b840 comp rx=0 tx=0).stop 2023-10-17T01:12:24.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 shutdown_connections 2023-10-17T01:12:24.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff2f80a0b20 0x7ff2f80a2fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff308102930 0x7ff308073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff308100e10 0x7ff308074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 --2- 172.21.15.181:0/2070647212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff308101f70 0x7ff308073140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:24.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 >> 172.21.15.181:0/2070647212 conn(0x7ff3080fc3d0 msgr2=0x7ff3080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:24.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 shutdown_connections 2023-10-17T01:12:24.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-17T01:12:24.429+0000 7ff30ee07700 1 -- 172.21.15.181:0/2070647212 wait complete. 2023-10-17T01:12:24.854 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.a.service 2023-10-17T01:12:24.860 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.d.service 2023-10-17T01:12:24.906 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.g.service 2023-10-17T01:12:24.944 INFO:journalctl@ceph.mds.a.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:12:24.948 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.j.service 2023-10-17T01:12:24.984 INFO:journalctl@ceph.mds.d.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:12:24.988 DEBUG:teuthology.orchestra.run.smithi060:mds.b> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.b.service 2023-10-17T01:12:24.991 DEBUG:teuthology.orchestra.run.smithi060:mds.e> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.e.service 2023-10-17T01:12:25.021 INFO:journalctl@ceph.mds.b.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:12:25.022 INFO:journalctl@ceph.mds.g.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:12:25.023 DEBUG:teuthology.orchestra.run.smithi060:mds.h> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.h.service 2023-10-17T01:12:25.049 INFO:journalctl@ceph.mds.e.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:12:25.050 DEBUG:teuthology.orchestra.run.smithi060:mds.k> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.k.service 2023-10-17T01:12:25.059 INFO:journalctl@ceph.mds.j.smithi045.stdout:-- Logs begin at Tue 2023-10-17 00:47:58 UTC. -- 2023-10-17T01:12:25.076 INFO:journalctl@ceph.mds.h.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:12:25.078 DEBUG:teuthology.orchestra.run.smithi181:mds.c> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.c.service 2023-10-17T01:12:25.080 DEBUG:teuthology.orchestra.run.smithi181:mds.f> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.f.service 2023-10-17T01:12:25.104 INFO:journalctl@ceph.mds.k.smithi060.stdout:-- Logs begin at Tue 2023-10-17 00:43:53 UTC. -- 2023-10-17T01:12:25.109 INFO:journalctl@ceph.mds.c.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:12:25.110 DEBUG:teuthology.orchestra.run.smithi181:mds.i> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.i.service 2023-10-17T01:12:25.136 INFO:journalctl@ceph.mds.f.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:12:25.137 DEBUG:teuthology.orchestra.run.smithi181:mds.l> sudo journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.l.service 2023-10-17T01:12:25.162 INFO:journalctl@ceph.mds.i.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:12:25.163 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-17T01:12:25.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:12:25.191 INFO:journalctl@ceph.mds.l.smithi181.stdout:-- Logs begin at Tue 2023-10-17 00:44:09 UTC. -- 2023-10-17T01:12:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 -- 172.21.15.45:0/3491734182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d0108c10 msgr2=0x7f48d010b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 --2- 172.21.15.45:0/3491734182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d0108c10 0x7f48d010b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48c800b0a0 tx=0x7f48c800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 -- 172.21.15.45:0/3491734182 shutdown_connections 2023-10-17T01:12:25.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 --2- 172.21.15.45:0/3491734182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d0108c10 0x7f48d010b0a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 --2- 172.21.15.45:0/3491734182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0074d90 0x7f48d00731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 --2- 172.21.15.45:0/3491734182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d00737b0 0x7f48d0073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 -- 172.21.15.45:0/3491734182 >> 172.21.15.45:0/3491734182 conn(0x7f48d00fc280 msgr2=0x7f48d00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 -- 172.21.15.45:0/3491734182 shutdown_connections 2023-10-17T01:12:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.382+0000 7f48d51ff700 1 -- 172.21.15.45:0/3491734182 wait complete. 2023-10-17T01:12:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 Processor -- start 2023-10-17T01:12:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 -- start start 2023-10-17T01:12:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d00737b0 0x7f48d0198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d0074d90 0x7f48d0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 0x7f48d01a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01a0870 con 0x7f48d0074d90 2023-10-17T01:12:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01a09e0 con 0x7f48d0108c10 2023-10-17T01:12:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.383+0000 7f48d51ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01a0ba0 con 0x7f48d00737b0 2023-10-17T01:12:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d00737b0 0x7f48d0198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 0x7f48d01a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48ce59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d0074d90 0x7f48d0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 0x7f48d01a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33258/0 (socket says 172.21.15.45:33258) 2023-10-17T01:12:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 -- 172.21.15.45:0/2835287307 learned_addr learned my addr 172.21.15.45:0/2835287307 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d00737b0 0x7f48d0198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50054/0 (socket says 172.21.15.45:50054) 2023-10-17T01:12:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 -- 172.21.15.45:0/2835287307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d00737b0 msgr2=0x7f48d0198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d00737b0 0x7f48d0198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 -- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d0074d90 msgr2=0x7f48d0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d0074d90 0x7f48d0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 -- 172.21.15.45:0/2835287307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8009d00 con 0x7f48d0108c10 2023-10-17T01:12:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.384+0000 7f48cf59e700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 0x7f48d01a02a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009fd0 tx=0x7f48c800bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.385+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800e040 con 0x7f48d0108c10 2023-10-17T01:12:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.385+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48c80094a0 con 0x7f48d0108c10 2023-10-17T01:12:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.385+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c801b6c0 con 0x7f48d0108c10 2023-10-17T01:12:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.385+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d01a0e20 con 0x7f48d0108c10 2023-10-17T01:12:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.385+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d01a1190 con 0x7f48d0108c10 2023-10-17T01:12:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.386+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d01072c0 con 0x7f48d0108c10 2023-10-17T01:12:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.388+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f48c801b820 con 0x7f48d0108c10 2023-10-17T01:12:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.389+0000 7f48b7fff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48b80a0b70 0x7f48b80a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.389+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48c80ccf40 con 0x7f48d0108c10 2023-10-17T01:12:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.389+0000 7f48ced9d700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48b80a0b70 0x7f48b80a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.390+0000 7f48ced9d700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48b80a0b70 0x7f48b80a3030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48d019a200 tx=0x7f48c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.391+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f48c8096d60 con 0x7f48d0108c10 2023-10-17T01:12:25.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:12:25.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-10-17T01:12:25.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:25.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:25.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:25.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:25.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:25.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:25.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:25.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:25.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.613+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48d002ce00 con 0x7f48d0108c10 2023-10-17T01:12:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.614+0000 7f48b7fff700 1 -- 172.21.15.45:0/2835287307 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f48c809a380 con 0x7f48d0108c10 2023-10-17T01:12:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:12:13.667312+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:12:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48b80a0b70 msgr2=0x7f48b80a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48b80a0b70 0x7f48b80a3030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48d019a200 tx=0x7f48c0009260 comp rx=0 tx=0).stop 2023-10-17T01:12:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 msgr2=0x7f48d01a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 0x7f48d01a02a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009fd0 tx=0x7f48c800bce0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 shutdown_connections 2023-10-17T01:12:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48b80a0b70 0x7f48b80a3030 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d0108c10 0x7f48d01a02a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d00737b0 0x7f48d0198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 --2- 172.21.15.45:0/2835287307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d0074d90 0x7f48d0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 >> 172.21.15.45:0/2835287307 conn(0x7f48d00fc280 msgr2=0x7f48d00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 shutdown_connections 2023-10-17T01:12:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.617+0000 7f48d51ff700 1 -- 172.21.15.45:0/2835287307 wait complete. 2023-10-17T01:12:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:12:25.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:12:25.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:12:25.753 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-10-17T01:12:25.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:25.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:25.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:25.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:25.754 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:25.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:25.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:25.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:25.755 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:12:25.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-10-17T01:12:25.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:25.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:25.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:25.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:25.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:25.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:25.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:25.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:25.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:25.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 -- 172.21.15.45:0/2544725880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c106a80 msgr2=0x7f1b7c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2544725880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c106a80 0x7f1b7c108e90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400b0a0 tx=0x7f1b7400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 -- 172.21.15.45:0/2544725880 shutdown_connections 2023-10-17T01:12:25.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2544725880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c106a80 0x7f1b7c108e90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2544725880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b7c101820 0x7f1b7c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2544725880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b7c104150 0x7f1b7c106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 -- 172.21.15.45:0/2544725880 >> 172.21.15.45:0/2544725880 conn(0x7f1b7c0fb1b0 msgr2=0x7f1b7c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:25.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 -- 172.21.15.45:0/2544725880 shutdown_connections 2023-10-17T01:12:25.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.848+0000 7f1b81fff700 1 -- 172.21.15.45:0/2544725880 wait complete. 2023-10-17T01:12:25.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.849+0000 7f1b81fff700 1 Processor -- start 2023-10-17T01:12:25.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.849+0000 7f1b81fff700 1 -- start start 2023-10-17T01:12:25.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b81fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 0x7f1b7c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b81fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b7c104150 0x7f1b7c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b81fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b7c106a80 0x7f1b7c1a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b81fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c1a0900 con 0x7f1b7c104150 2023-10-17T01:12:25.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b81fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c1a0a70 con 0x7f1b7c106a80 2023-10-17T01:12:25.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b81fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c1a0c30 con 0x7f1b7c101820 2023-10-17T01:12:25.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 0x7f1b7c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 0x7f1b7c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50082/0 (socket says 172.21.15.45:50082) 2023-10-17T01:12:25.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b7c104150 0x7f1b7c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 -- 172.21.15.45:0/2552549531 learned_addr learned my addr 172.21.15.45:0/2552549531 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:25.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7bfff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b7c106a80 0x7f1b7c1a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 -- 172.21.15.45:0/2552549531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b7c106a80 msgr2=0x7f1b7c1a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b7c106a80 0x7f1b7c1a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 -- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b7c104150 msgr2=0x7f1b7c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:25.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.850+0000 7f1b7b7fe700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b7c104150 0x7f1b7c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b7b7fe700 1 -- 172.21.15.45:0/2552549531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b74009d00 con 0x7f1b7c101820 2023-10-17T01:12:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b7b7fe700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 0x7f1b7c198dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e9a0 tx=0x7f1b6c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c009790 con 0x7f1b7c101820 2023-10-17T01:12:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b6c010430 con 0x7f1b7c101820 2023-10-17T01:12:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c1a0f10 con 0x7f1b7c101820 2023-10-17T01:12:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c018680 con 0x7f1b7c101820 2023-10-17T01:12:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.851+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c1a1330 con 0x7f1b7c101820 2023-10-17T01:12:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.852+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c108600 con 0x7f1b7c101820 2023-10-17T01:12:25.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.854+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f1b6c0105a0 con 0x7f1b7c101820 2023-10-17T01:12:25.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.856+0000 7f1b797fa700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b680a0b20 0x7f1b680a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:25.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.856+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1b6c014070 con 0x7f1b7c101820 2023-10-17T01:12:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.856+0000 7f1b73fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b680a0b20 0x7f1b680a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.856+0000 7f1b73fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b680a0b20 0x7f1b680a2fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009c60 tx=0x7f1b64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:25.858+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1b6c096000 con 0x7f1b7c101820 2023-10-17T01:12:26.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.069+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b7c02cea0 con 0x7f1b7c101820 2023-10-17T01:12:26.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.069+0000 7f1b797fa700 1 -- 172.21.15.45:0/2552549531 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b6c099620 con 0x7f1b7c101820 2023-10-17T01:12:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:12:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b680a0b20 msgr2=0x7f1b680a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b680a0b20 0x7f1b680a2fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009c60 tx=0x7f1b64009260 comp rx=0 tx=0).stop 2023-10-17T01:12:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 msgr2=0x7f1b7c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 0x7f1b7c198dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e9a0 tx=0x7f1b6c00ecb0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 shutdown_connections 2023-10-17T01:12:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b680a0b20 0x7f1b680a2fe0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b7c106a80 0x7f1b7c1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b7c101820 0x7f1b7c198dc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 --2- 172.21.15.45:0/2552549531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b7c104150 0x7f1b7c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 >> 172.21.15.45:0/2552549531 conn(0x7f1b7c0fb1b0 msgr2=0x7f1b7c102530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:26.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 shutdown_connections 2023-10-17T01:12:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.072+0000 7f1b81fff700 1 -- 172.21.15.45:0/2552549531 wait complete. 2023-10-17T01:12:26.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:26.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.295+0000 7fa7159ff700 1 -- 172.21.15.45:0/1062648975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa710100d60 msgr2=0x7fa710101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.295+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1062648975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa710100d60 0x7fa710101180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa7000099c0 tx=0x7fa700009cd0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.295+0000 7fa7159ff700 1 -- 172.21.15.45:0/1062648975 shutdown_connections 2023-10-17T01:12:26.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.295+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1062648975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7101028f0 0x7fa7101091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.295+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1062648975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa710100d60 0x7fa710101180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.295+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1062648975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 0x7fa7101023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.296+0000 7fa7159ff700 1 -- 172.21.15.45:0/1062648975 >> 172.21.15.45:0/1062648975 conn(0x7fa7100fc290 msgr2=0x7fa7100fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:26.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.296+0000 7fa7159ff700 1 -- 172.21.15.45:0/1062648975 shutdown_connections 2023-10-17T01:12:26.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.296+0000 7fa7159ff700 1 -- 172.21.15.45:0/1062648975 wait complete. 2023-10-17T01:12:26.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 Processor -- start 2023-10-17T01:12:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 -- start start 2023-10-17T01:12:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 0x7fa710074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 0x7fa710073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7101028f0 0x7fa710073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710073e20 con 0x7fa710100d60 2023-10-17T01:12:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710073f90 con 0x7fa7101028f0 2023-10-17T01:12:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa7159ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7101a0a80 con 0x7fa710101f50 2023-10-17T01:12:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa70effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 0x7fa710074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa70e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 0x7fa710073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa70effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 0x7fa710074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55170/0 (socket says 172.21.15.45:55170) 2023-10-17T01:12:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa70effd700 1 -- 172.21.15.45:0/1720170867 learned_addr learned my addr 172.21.15.45:0/1720170867 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.297+0000 7fa70f7fe700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7101028f0 0x7fa710073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70e7fc700 1 -- 172.21.15.45:0/1720170867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7101028f0 msgr2=0x7fa710073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70e7fc700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7101028f0 0x7fa710073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70e7fc700 1 -- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 msgr2=0x7fa710074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70e7fc700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 0x7fa710074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70e7fc700 1 -- 172.21.15.45:0/1720170867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000095f0 con 0x7fa710101f50 2023-10-17T01:12:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70effd700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 0x7fa710074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa70e7fc700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 0x7fa710073140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa70400e9f0 tx=0x7fa70400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400c9c0 con 0x7fa710101f50 2023-10-17T01:12:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa704010430 con 0x7fa710101f50 2023-10-17T01:12:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7101a0d60 con 0x7fa710101f50 2023-10-17T01:12:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704018680 con 0x7fa710101f50 2023-10-17T01:12:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.298+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101a1210 con 0x7fa710101f50 2023-10-17T01:12:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.299+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71010c5c0 con 0x7fa710101f50 2023-10-17T01:12:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.301+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa704010860 con 0x7fa710101f50 2023-10-17T01:12:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.303+0000 7fa7149fd700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f80a0b70 0x7fa6f80a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.303+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa704014070 con 0x7fa710101f50 2023-10-17T01:12:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.303+0000 7fa70effd700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f80a0b70 0x7fa6f80a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.304+0000 7fa70effd700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f80a0b70 0x7fa6f80a3030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa700005f50 tx=0x7fa700005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.305+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fa704095fb0 con 0x7fa710101f50 2023-10-17T01:12:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.564+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa710107c10 con 0x7fa710101f50 2023-10-17T01:12:26.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.565+0000 7fa7149fd700 1 -- 172.21.15.45:0/1720170867 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fa7040995d0 con 0x7fa710101f50 2023-10-17T01:12:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:26.567 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-17T01:12:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f80a0b70 msgr2=0x7fa6f80a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f80a0b70 0x7fa6f80a3030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa700005f50 tx=0x7fa700005e80 comp rx=0 tx=0).stop 2023-10-17T01:12:26.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 msgr2=0x7fa710073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:26.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 0x7fa710073140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa70400e9f0 tx=0x7fa70400ed00 comp rx=0 tx=0).stop 2023-10-17T01:12:26.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 shutdown_connections 2023-10-17T01:12:26.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f80a0b70 0x7fa6f80a3030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7101028f0 0x7fa710073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa710100d60 0x7fa710074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 --2- 172.21.15.45:0/1720170867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa710101f50 0x7fa710073140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 >> 172.21.15.45:0/1720170867 conn(0x7fa7100fc290 msgr2=0x7fa7100fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.567+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 shutdown_connections 2023-10-17T01:12:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:26.568+0000 7fa7159ff700 1 -- 172.21.15.45:0/1720170867 wait complete. 2023-10-17T01:12:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2023-10-17T01:12:26.773 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:26 smithi181 ceph-mon[134520]: Deploying daemon mds.l on smithi181 2023-10-17T01:12:26.773 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:26 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2835287307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:26.773 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:26 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2552549531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:26.774 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:12:26 smithi181 systemd[1]: Starting Ceph mds.l for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:26 smithi045 ceph-mon[98474]: Deploying daemon mds.l on smithi181 2023-10-17T01:12:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:26 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2835287307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:26 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2552549531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:26.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:26 smithi060 ceph-mon[134136]: Deploying daemon mds.l on smithi181 2023-10-17T01:12:26.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:26 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2835287307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:26.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:26 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2552549531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:27.052 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:12:26 smithi181 podman[171028]: 2023-10-17T01:12:27.407 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:12:27 smithi181 bash[171028]: b027706a655cd90c85c368508c6e6426be1f503960f69fc313dcdccce8ad330f 2023-10-17T01:12:27.407 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:12:27 smithi181 systemd[1]: Started Ceph mds.l for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:27.408 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mds[171068]: -- 172.21.15.181:0/4057563898 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640e37551e0 con 0x5640e372ac00 2023-10-17T01:12:27.408 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l[171044]: starting mds.l at 2023-10-17T01:12:27.766 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:27.766 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1720170867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:27.766 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6832/2648148860,v1:172.21.15.181:6833/2648148860] up:boot 2023-10-17T01:12:27.767 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: fsmap 1 up:standby 2023-10-17T01:12:27.767 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-17T01:12:27.767 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.767 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.767 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.768 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:27.768 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:27.768 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1720170867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6832/2648148860,v1:172.21.15.181:6833/2648148860] up:boot 2023-10-17T01:12:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: fsmap 1 up:standby 2023-10-17T01:12:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-17T01:12:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1720170867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6832/2648148860,v1:172.21.15.181:6833/2648148860] up:boot 2023-10-17T01:12:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: fsmap 1 up:standby 2023-10-17T01:12:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-17T01:12:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:28.588 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:28.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:28 smithi045 ceph-mon[98474]: Deploying daemon mds.k on smithi060 2023-10-17T01:12:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 -- 172.21.15.45:0/1348765399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 msgr2=0x7fb98c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 --2- 172.21.15.45:0/1348765399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c109190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb98400b0d0 tx=0x7fb98400b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 -- 172.21.15.45:0/1348765399 shutdown_connections 2023-10-17T01:12:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 --2- 172.21.15.45:0/1348765399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c109190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 --2- 172.21.15.45:0/1348765399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c100d50 0x7fb98c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 --2- 172.21.15.45:0/1348765399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c101f40 0x7fb98c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:28.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 -- 172.21.15.45:0/1348765399 >> 172.21.15.45:0/1348765399 conn(0x7fb98c0fc280 msgr2=0x7fb98c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:28.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 -- 172.21.15.45:0/1348765399 shutdown_connections 2023-10-17T01:12:28.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.803+0000 7fb9921ff700 1 -- 172.21.15.45:0/1348765399 wait complete. 2023-10-17T01:12:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 Processor -- start 2023-10-17T01:12:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 -- start start 2023-10-17T01:12:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c100d50 0x7fb98c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 0x7fb98c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c1a0880 con 0x7fb98c1028e0 2023-10-17T01:12:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c1a09f0 con 0x7fb98c101f40 2023-10-17T01:12:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.804+0000 7fb9921ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c1a0bb0 con 0x7fb98c100d50 2023-10-17T01:12:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 0x7fb98c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 0x7fb98c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40866/0 (socket says 172.21.15.45:40866) 2023-10-17T01:12:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98affd700 1 -- 172.21.15.45:0/3409495995 learned_addr learned my addr 172.21.15.45:0/3409495995 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98b7fe700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c100d50 0x7fb98c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 -- 172.21.15.45:0/3409495995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c100d50 msgr2=0x7fb98c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c100d50 0x7fb98c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 -- 172.21.15.45:0/3409495995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 msgr2=0x7fb98c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 0x7fb98c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 -- 172.21.15.45:0/3409495995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009520 con 0x7fb98c1028e0 2023-10-17T01:12:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98affd700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 0x7fb98c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.805+0000 7fb98bfff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c1a02b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb984007a30 tx=0x7fb984008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.806+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400e040 con 0x7fb98c1028e0 2023-10-17T01:12:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.806+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb98400bd30 con 0x7fb98c1028e0 2023-10-17T01:12:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.806+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98401d470 con 0x7fb98c1028e0 2023-10-17T01:12:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.806+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb984009d00 con 0x7fb98c1028e0 2023-10-17T01:12:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.806+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c1a10d0 con 0x7fb98c1028e0 2023-10-17T01:12:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.807+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c10c5c0 con 0x7fb98c1028e0 2023-10-17T01:12:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.809+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb984005000 con 0x7fb98c1028e0 2023-10-17T01:12:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.810+0000 7fb988ff9700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb9740a0b70 0x7fb9740a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:28.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.811+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9840cd890 con 0x7fb98c1028e0 2023-10-17T01:12:28.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.811+0000 7fb98b7fe700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb9740a0b70 0x7fb9740a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:28.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.811+0000 7fb98b7fe700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb9740a0b70 0x7fb9740a3030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb98c19a210 tx=0x7fb97c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:28.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:28.813+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb9840977d0 con 0x7fb98c1028e0 2023-10-17T01:12:28.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:28 smithi060 ceph-mon[134136]: Deploying daemon mds.k on smithi060 2023-10-17T01:12:28.829 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:12:28 smithi060 systemd[1]: Starting Ceph mds.k for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:28 smithi181 ceph-mon[134520]: Deploying daemon mds.k on smithi060 2023-10-17T01:12:29.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.068+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb98c10c7e0 con 0x7fb98c1028e0 2023-10-17T01:12:29.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.068+0000 7fb988ff9700 1 -- 172.21.15.45:0/3409495995 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fb98409adf0 con 0x7fb98c1028e0 2023-10-17T01:12:29.070 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:29.070 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-17T01:12:29.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb9740a0b70 msgr2=0x7fb9740a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:29.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb9740a0b70 0x7fb9740a3030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb98c19a210 tx=0x7fb97c009260 comp rx=0 tx=0).stop 2023-10-17T01:12:29.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 msgr2=0x7fb98c1a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:29.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c1a02b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb984007a30 tx=0x7fb984008eb0 comp rx=0 tx=0).stop 2023-10-17T01:12:29.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 shutdown_connections 2023-10-17T01:12:29.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb9740a0b70 0x7fb9740a3030 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:29.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98c1028e0 0x7fb98c1a02b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:29.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c100d50 0x7fb98c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:29.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 --2- 172.21.15.45:0/3409495995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb98c101f40 0x7fb98c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.070+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 >> 172.21.15.45:0/3409495995 conn(0x7fb98c0fc280 msgr2=0x7fb98c1077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.071+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 shutdown_connections 2023-10-17T01:12:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:29.071+0000 7fb9921ff700 1 -- 172.21.15.45:0/3409495995 wait complete. 2023-10-17T01:12:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2023-10-17T01:12:29.113 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:12:28 smithi060 podman[169976]: 2023-10-17T01:12:29.114 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:12:29 smithi060 bash[169976]: 603319b9b7922a7e8716a7d4d4b00ee7bce861636f97b39163225d07bf747fac 2023-10-17T01:12:29.114 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mds[170016]: -- 172.21.15.60:0/4013156527 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56158e1d51e0 con 0x56158e1aac00 2023-10-17T01:12:29.114 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k[169992]: starting mds.k at 2023-10-17T01:12:29.445 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:12:29 smithi060 systemd[1]: Started Ceph mds.k for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3409495995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:29.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:29.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:29 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3409495995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:29.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:29 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3409495995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:29 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:30 smithi045 ceph-mon[98474]: Deploying daemon mds.j on smithi045 2023-10-17T01:12:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:30 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6832/3345979209,v1:172.21.15.60:6833/3345979209] up:boot 2023-10-17T01:12:30.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:30 smithi045 ceph-mon[98474]: fsmap 2 up:standby 2023-10-17T01:12:30.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-17T01:12:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:30 smithi060 ceph-mon[134136]: Deploying daemon mds.j on smithi045 2023-10-17T01:12:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:30 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6832/3345979209,v1:172.21.15.60:6833/3345979209] up:boot 2023-10-17T01:12:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:30 smithi060 ceph-mon[134136]: fsmap 2 up:standby 2023-10-17T01:12:30.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-17T01:12:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:30 smithi181 ceph-mon[134520]: Deploying daemon mds.j on smithi045 2023-10-17T01:12:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:30 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6832/3345979209,v1:172.21.15.60:6833/3345979209] up:boot 2023-10-17T01:12:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:30 smithi181 ceph-mon[134520]: fsmap 2 up:standby 2023-10-17T01:12:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-17T01:12:31.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/1731145060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 msgr2=0x7f2c98109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/1731145060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c98109190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000b0d0 tx=0x7f2c9000b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/1731145060 shutdown_connections 2023-10-17T01:12:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/1731145060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c98109190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/1731145060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 0x7f2c98101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/1731145060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c98101f40 0x7f2c981023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/1731145060 >> 172.21.15.45:0/1731145060 conn(0x7f2c980fc280 msgr2=0x7f2c980fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/1731145060 shutdown_connections 2023-10-17T01:12:31.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.349+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/1731145060 wait complete. 2023-10-17T01:12:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 Processor -- start 2023-10-17T01:12:31.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 -- start start 2023-10-17T01:12:31.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 0x7f2c98198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c98101f40 0x7f2c981991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c981a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981a0820 con 0x7f2c981028e0 2023-10-17T01:12:31.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981a0990 con 0x7f2c98100d50 2023-10-17T01:12:31.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.350+0000 7f2c9e9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981a0b50 con 0x7f2c98101f40 2023-10-17T01:12:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c97fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 0x7f2c98198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c981a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c981a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55970/0 (socket says 172.21.15.45:55970) 2023-10-17T01:12:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 -- 172.21.15.45:0/2827010794 learned_addr learned my addr 172.21.15.45:0/2827010794 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c97fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 0x7f2c98198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40890/0 (socket says 172.21.15.45:40890) 2023-10-17T01:12:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c977fe700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c98101f40 0x7f2c981991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 -- 172.21.15.45:0/2827010794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c98101f40 msgr2=0x7f2c981991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c98101f40 0x7f2c981991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 -- 172.21.15.45:0/2827010794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 msgr2=0x7f2c98198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 0x7f2c98198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 -- 172.21.15.45:0/2827010794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c90009d00 con 0x7f2c981028e0 2023-10-17T01:12:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.351+0000 7f2c9cf9c700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c981a0250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2c90008f10 tx=0x7f2c90008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.352+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000e040 con 0x7f2c981028e0 2023-10-17T01:12:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.352+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c90005070 con 0x7f2c981028e0 2023-10-17T01:12:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.352+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c981a0dd0 con 0x7f2c981028e0 2023-10-17T01:12:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.352+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9001b6a0 con 0x7f2c981028e0 2023-10-17T01:12:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.352+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c981a11a0 con 0x7f2c981028e0 2023-10-17T01:12:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.353+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9810c570 con 0x7f2c981028e0 2023-10-17T01:12:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.355+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2c900051e0 con 0x7f2c981028e0 2023-10-17T01:12:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.356+0000 7f2c957fa700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c800a0b70 0x7f2c800a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.356+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2c900ccf80 con 0x7f2c981028e0 2023-10-17T01:12:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.356+0000 7f2c97fff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c800a0b70 0x7f2c800a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.357+0000 7f2c97fff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c800a0b70 0x7f2c800a3030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c98199b80 tx=0x7f2c88007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.360+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2c90096da0 con 0x7f2c981028e0 2023-10-17T01:12:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.625+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c981a1750 con 0x7f2c981028e0 2023-10-17T01:12:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.626+0000 7f2c957fa700 1 -- 172.21.15.45:0/2827010794 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f2c9009a3c0 con 0x7f2c981028e0 2023-10-17T01:12:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:31.628 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-17T01:12:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.628+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c800a0b70 msgr2=0x7f2c800a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.628+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c800a0b70 0x7f2c800a3030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c98199b80 tx=0x7f2c88007400 comp rx=0 tx=0).stop 2023-10-17T01:12:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.628+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 msgr2=0x7f2c981a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.628+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c981a0250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2c90008f10 tx=0x7f2c90008f40 comp rx=0 tx=0).stop 2023-10-17T01:12:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 shutdown_connections 2023-10-17T01:12:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c800a0b70 0x7f2c800a3030 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c981028e0 0x7f2c981a0250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c98100d50 0x7f2c98198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 --2- 172.21.15.45:0/2827010794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c98101f40 0x7f2c981991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 >> 172.21.15.45:0/2827010794 conn(0x7f2c980fc280 msgr2=0x7f2c98107720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 shutdown_connections 2023-10-17T01:12:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:31.629+0000 7f2c9e9ff700 1 -- 172.21.15.45:0/2827010794 wait complete. 2023-10-17T01:12:31.633 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2023-10-17T01:12:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:31 smithi045 ceph-mon[98474]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:31.801 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:12:31 smithi045 systemd[1]: Starting Ceph mds.j for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:31 smithi060 ceph-mon[134136]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:31 smithi181 ceph-mon[134520]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:32.301 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:12:32 smithi045 podman[147714]: 2023-10-17T01:12:32.302 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:12:32 smithi045 bash[147714]: b9d0043a23700ab1f56efa9ccaf9945094e7b370d4aa767861a7423afd486d91 2023-10-17T01:12:32.302 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mds[147754]: -- 172.21.15.45:0/3835031264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e02ce0b1e0 con 0x55e02cde2c00 2023-10-17T01:12:32.302 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-j[147730]: starting mds.j at 2023-10-17T01:12:32.801 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:12:32 smithi045 systemd[1]: Started Ceph mds.j for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2827010794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2827010794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2827010794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:33.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:33.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:33 smithi045 ceph-mon[98474]: Deploying daemon mds.d on smithi045 2023-10-17T01:12:33.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:33 smithi045 ceph-mon[98474]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:33.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:33 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.45:6834/1708283883,v1:172.21.15.45:6835/1708283883] up:boot 2023-10-17T01:12:33.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:33 smithi045 ceph-mon[98474]: fsmap 3 up:standby 2023-10-17T01:12:33.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-17T01:12:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:33 smithi060 ceph-mon[134136]: Deploying daemon mds.d on smithi045 2023-10-17T01:12:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:33 smithi060 ceph-mon[134136]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:33 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.45:6834/1708283883,v1:172.21.15.45:6835/1708283883] up:boot 2023-10-17T01:12:33.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:33 smithi060 ceph-mon[134136]: fsmap 3 up:standby 2023-10-17T01:12:33.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-17T01:12:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.872+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/2853270290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a641040e0 msgr2=0x7f7a641064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.872+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/2853270290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a641040e0 0x7f7a641064d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a540098f0 tx=0x7f7a54009c00 comp rx=0 tx=0).stop 2023-10-17T01:12:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/2853270290 shutdown_connections 2023-10-17T01:12:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/2853270290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a64106a10 0x7f7a64108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/2853270290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/2853270290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a641040e0 0x7f7a641064d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/2853270290 >> 172.21.15.45:0/2853270290 conn(0x7f7a640fb180 msgr2=0x7f7a640fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/2853270290 shutdown_connections 2023-10-17T01:12:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.873+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/2853270290 wait complete. 2023-10-17T01:12:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 Processor -- start 2023-10-17T01:12:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 -- start start 2023-10-17T01:12:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a641040e0 0x7f7a640ff760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a64106a10 0x7f7a640ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64100440 con 0x7f7a641017b0 2023-10-17T01:12:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641005b0 con 0x7f7a64106a10 2023-10-17T01:12:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6b5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641a0ae0 con 0x7f7a641040e0 2023-10-17T01:12:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56000/0 (socket says 172.21.15.45:56000) 2023-10-17T01:12:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.874+0000 7f7a6939b700 1 -- 172.21.15.45:0/539735570 learned_addr learned my addr 172.21.15.45:0/539735570 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a69b9c700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a64106a10 0x7f7a640ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a68b9a700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a641040e0 0x7f7a640ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6939b700 1 -- 172.21.15.45:0/539735570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a641040e0 msgr2=0x7f7a640ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6939b700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a641040e0 0x7f7a640ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6939b700 1 -- 172.21.15.45:0/539735570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a64106a10 msgr2=0x7f7a640ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6939b700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a64106a10 0x7f7a640ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6939b700 1 -- 172.21.15.45:0/539735570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a60009520 con 0x7f7a641017b0 2023-10-17T01:12:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6939b700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64101120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000eb30 tx=0x7f7a6000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60009880 con 0x7f7a641017b0 2023-10-17T01:12:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a54009520 con 0x7f7a641017b0 2023-10-17T01:12:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a600044e0 con 0x7f7a641017b0 2023-10-17T01:12:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60003d80 con 0x7f7a641017b0 2023-10-17T01:12:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.875+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a641a1140 con 0x7f7a641017b0 2023-10-17T01:12:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.876+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6410c570 con 0x7f7a641017b0 2023-10-17T01:12:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.878+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f7a60003f20 con 0x7f7a641017b0 2023-10-17T01:12:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.880+0000 7f7a5a7fc700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a500a0b70 0x7f7a500a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.880+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a60014070 con 0x7f7a641017b0 2023-10-17T01:12:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.880+0000 7f7a68b9a700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a500a0b70 0x7f7a500a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.881+0000 7f7a68b9a700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a500a0b70 0x7f7a500a3030 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005e90 tx=0x7f7a54019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:33.884+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7a60096020 con 0x7f7a641017b0 2023-10-17T01:12:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:33 smithi181 ceph-mon[134520]: Deploying daemon mds.d on smithi045 2023-10-17T01:12:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:33 smithi181 ceph-mon[134520]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:33 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.45:6834/1708283883,v1:172.21.15.45:6835/1708283883] up:boot 2023-10-17T01:12:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:33 smithi181 ceph-mon[134520]: fsmap 3 up:standby 2023-10-17T01:12:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-17T01:12:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.145+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a641a18f0 con 0x7f7a641017b0 2023-10-17T01:12:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.146+0000 7f7a5a7fc700 1 -- 172.21.15.45:0/539735570 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f7a60099640 con 0x7f7a641017b0 2023-10-17T01:12:34.147 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:34.147 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-17T01:12:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a500a0b70 msgr2=0x7f7a500a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a500a0b70 0x7f7a500a3030 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005e90 tx=0x7f7a54019040 comp rx=0 tx=0).stop 2023-10-17T01:12:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 msgr2=0x7f7a64101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64101120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000eb30 tx=0x7f7a6000ee40 comp rx=0 tx=0).stop 2023-10-17T01:12:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 shutdown_connections 2023-10-17T01:12:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a500a0b70 0x7f7a500a3030 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a64106a10 0x7f7a640ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a641017b0 0x7f7a64101120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 --2- 172.21.15.45:0/539735570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a641040e0 0x7f7a640ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 >> 172.21.15.45:0/539735570 conn(0x7f7a640fb180 msgr2=0x7f7a64102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.148+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 shutdown_connections 2023-10-17T01:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:34.149+0000 7f7a6b5ff700 1 -- 172.21.15.45:0/539735570 wait complete. 2023-10-17T01:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2023-10-17T01:12:34.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:34 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/539735570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:34.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:34 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/539735570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:34.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:34 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/539735570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:35.301 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:12:35 smithi045 systemd[1]: Starting Ceph mds.d for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:35 smithi045 ceph-mon[98474]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:35.802 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:12:35 smithi045 podman[148103]: 2023-10-17T01:12:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:35 smithi060 ceph-mon[134136]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:35 smithi181 ceph-mon[134520]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:36.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:36.238 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:12:35 smithi045 bash[148103]: a9a81720182051ec2ceeb546c8c3b481b6ec7abd75dc823e60bcf21cf2861c69 2023-10-17T01:12:36.238 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:12:35 smithi045 ceph-mds[148143]: -- 172.21.15.45:0/702443602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55890966d1e0 con 0x558909645000 2023-10-17T01:12:36.238 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:12:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-d[148119]: starting mds.d at 2023-10-17T01:12:36.238 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:12:35 smithi045 systemd[1]: Started Ceph mds.d for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/627986200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e81028e0 msgr2=0x7fe8e8109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/627986200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e81028e0 0x7fe8e8109190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009fc0 tx=0x7fe8e000b330 comp rx=0 tx=0).stop 2023-10-17T01:12:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/627986200 shutdown_connections 2023-10-17T01:12:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/627986200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e81028e0 0x7fe8e8109190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/627986200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e8100d50 0x7fe8e8101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/627986200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e8101f40 0x7fe8e81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/627986200 >> 172.21.15.45:0/627986200 conn(0x7fe8e80fc280 msgr2=0x7fe8e80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/627986200 shutdown_connections 2023-10-17T01:12:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.383+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/627986200 wait complete. 2023-10-17T01:12:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 Processor -- start 2023-10-17T01:12:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 -- start start 2023-10-17T01:12:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e8100d50 0x7fe8e8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e8101f40 0x7fe8e8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 0x7fe8e81a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a0860 con 0x7fe8e8100d50 2023-10-17T01:12:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a09d0 con 0x7fe8e81028e0 2023-10-17T01:12:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.384+0000 7fe8ef3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a0b90 con 0x7fe8e8101f40 2023-10-17T01:12:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 0x7fe8e81a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e8100d50 0x7fe8e8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 0x7fe8e81a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40952/0 (socket says 172.21.15.45:40952) 2023-10-17T01:12:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 -- 172.21.15.45:0/2592029867 learned_addr learned my addr 172.21.15.45:0/2592029867 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ec99a700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e8101f40 0x7fe8e8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 -- 172.21.15.45:0/2592029867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e8101f40 msgr2=0x7fe8e8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e8101f40 0x7fe8e8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 -- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e8100d50 msgr2=0x7fe8e8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e8100d50 0x7fe8e8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 -- 172.21.15.45:0/2592029867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009c40 con 0x7fe8e81028e0 2023-10-17T01:12:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.385+0000 7fe8ed99c700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 0x7fe8e81a0290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004c40 tx=0x7fe8e0004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.386+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000e040 con 0x7fe8e81028e0 2023-10-17T01:12:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.386+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8e000bbf0 con 0x7fe8e81028e0 2023-10-17T01:12:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.386+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e81a0e10 con 0x7fe8e81028e0 2023-10-17T01:12:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.386+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00052a0 con 0x7fe8e81028e0 2023-10-17T01:12:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.386+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81a11d0 con 0x7fe8e81028e0 2023-10-17T01:12:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.387+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e810c5c0 con 0x7fe8e81028e0 2023-10-17T01:12:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.389+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fe8e001d470 con 0x7fe8e81028e0 2023-10-17T01:12:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.390+0000 7fe8de7fc700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe8d40a0bf0 0x7fe8d40a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.390+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe8e00cd830 con 0x7fe8e81028e0 2023-10-17T01:12:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.390+0000 7fe8ed19b700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe8d40a0bf0 0x7fe8d40a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.391+0000 7fe8ed19b700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe8d40a0bf0 0x7fe8d40a30b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8e819a1f0 tx=0x7fe8e4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.396+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe8e0097650 con 0x7fe8e81028e0 2023-10-17T01:12:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.658+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8e81a1780 con 0x7fe8e81028e0 2023-10-17T01:12:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.659+0000 7fe8de7fc700 1 -- 172.21.15.45:0/2592029867 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3531 (secure 0 0 0) 0x7fe8e0021de0 con 0x7fe8e81028e0 2023-10-17T01:12:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:36.660 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-17T01:12:36.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe8d40a0bf0 msgr2=0x7fe8d40a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:36.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe8d40a0bf0 0x7fe8d40a30b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8e819a1f0 tx=0x7fe8e4009310 comp rx=0 tx=0).stop 2023-10-17T01:12:36.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 msgr2=0x7fe8e81a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:36.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 0x7fe8e81a0290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004c40 tx=0x7fe8e0004c70 comp rx=0 tx=0).stop 2023-10-17T01:12:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 shutdown_connections 2023-10-17T01:12:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe8d40a0bf0 0x7fe8d40a30b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e81028e0 0x7fe8e81a0290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e8100d50 0x7fe8e8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 --2- 172.21.15.45:0/2592029867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e8101f40 0x7fe8e8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:36.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 >> 172.21.15.45:0/2592029867 conn(0x7fe8e80fc280 msgr2=0x7fe8e8107780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:36.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 shutdown_connections 2023-10-17T01:12:36.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:36.661+0000 7fe8ef3ff700 1 -- 172.21.15.45:0/2592029867 wait complete. 2023-10-17T01:12:36.665 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2023-10-17T01:12:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:37.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:37.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:37.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: Deploying daemon mds.e on smithi060 2023-10-17T01:12:37.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:37.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.45:6836/1507187876,v1:172.21.15.45:6837/1507187876] up:boot 2023-10-17T01:12:37.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: fsmap 4 up:standby 2023-10-17T01:12:37.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-17T01:12:37.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:36 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2592029867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:37.109 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.110 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.110 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.110 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:37.110 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:37.110 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:37.111 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: Deploying daemon mds.e on smithi060 2023-10-17T01:12:37.111 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:37.111 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.45:6836/1507187876,v1:172.21.15.45:6837/1507187876] up:boot 2023-10-17T01:12:37.111 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: fsmap 4 up:standby 2023-10-17T01:12:37.111 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-17T01:12:37.112 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:36 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2592029867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:37.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:37.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:37.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:37.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:37.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: Deploying daemon mds.e on smithi060 2023-10-17T01:12:37.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:37.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.45:6836/1507187876,v1:172.21.15.45:6837/1507187876] up:boot 2023-10-17T01:12:37.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: fsmap 4 up:standby 2023-10-17T01:12:37.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-17T01:12:37.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:36 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2592029867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:37.655 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:12:37 smithi060 systemd[1]: Starting Ceph mds.e for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:37.962 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:12:37 smithi060 podman[170316]: 2023-10-17T01:12:37.962 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:12:37 smithi060 bash[170316]: b3fe1e674b6199bb9f2750e797f7f1f7da12da6154b1536a680a7f9c4338966e 2023-10-17T01:12:37.963 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:12:37 smithi060 ceph-mds[170356]: -- 172.21.15.60:0/578138378 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e73b5d71e0 con 0x55e73b5acc00 2023-10-17T01:12:37.963 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:12:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e[170332]: starting mds.e at 2023-10-17T01:12:38.328 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:12:37 smithi060 systemd[1]: Started Ceph mds.e for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:38.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:38.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 -- 172.21.15.45:0/2416100532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c100d50 msgr2=0x7fa91c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 --2- 172.21.15.45:0/2416100532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c100d50 0x7fa91c101170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0099c0 tx=0x7fa90c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:12:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 -- 172.21.15.45:0/2416100532 shutdown_connections 2023-10-17T01:12:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 --2- 172.21.15.45:0/2416100532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c1028e0 0x7fa91c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 --2- 172.21.15.45:0/2416100532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c100d50 0x7fa91c101170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 --2- 172.21.15.45:0/2416100532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c101f40 0x7fa91c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 -- 172.21.15.45:0/2416100532 >> 172.21.15.45:0/2416100532 conn(0x7fa91c0fc280 msgr2=0x7fa91c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 -- 172.21.15.45:0/2416100532 shutdown_connections 2023-10-17T01:12:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.897+0000 7fa9217ff700 1 -- 172.21.15.45:0/2416100532 wait complete. 2023-10-17T01:12:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.898+0000 7fa9217ff700 1 Processor -- start 2023-10-17T01:12:38.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.898+0000 7fa9217ff700 1 -- start start 2023-10-17T01:12:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa9217ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c100d50 0x7fa91c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa9217ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c101f40 0x7fa91c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa9217ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 0x7fa91c073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa9217ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c073e20 con 0x7fa91c101f40 2023-10-17T01:12:38.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa9217ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c073f90 con 0x7fa91c100d50 2023-10-17T01:12:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa9217ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a0aa0 con 0x7fa91c1028e0 2023-10-17T01:12:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c100d50 0x7fa91c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 0x7fa91c073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c101f40 0x7fa91c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c100d50 0x7fa91c074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46252/0 (socket says 172.21.15.45:46252) 2023-10-17T01:12:38.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 0x7fa91c073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34628/0 (socket says 172.21.15.45:34628) 2023-10-17T01:12:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91affd700 1 -- 172.21.15.45:0/3491363132 learned_addr learned my addr 172.21.15.45:0/3491363132 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 -- 172.21.15.45:0/3491363132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c100d50 msgr2=0x7fa91c074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c100d50 0x7fa91c074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 -- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c101f40 msgr2=0x7fa91c073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c101f40 0x7fa91c073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91b7fe700 1 -- 172.21.15.45:0/3491363132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0095f0 con 0x7fa91c1028e0 2023-10-17T01:12:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.899+0000 7fa91a7fc700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c101f40 0x7fa91c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.900+0000 7fa91b7fe700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 0x7fa91c073680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa91400efc0 tx=0x7fa91400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.900+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914018070 con 0x7fa91c1028e0 2023-10-17T01:12:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.900+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91400f040 con 0x7fa91c1028e0 2023-10-17T01:12:38.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.900+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400cd40 con 0x7fa91c1028e0 2023-10-17T01:12:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.900+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c1a0d20 con 0x7fa91c1028e0 2023-10-17T01:12:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.900+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1a1180 con 0x7fa91c1028e0 2023-10-17T01:12:38.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.901+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c10c5b0 con 0x7fa91c1028e0 2023-10-17T01:12:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.903+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa914003a00 con 0x7fa91c1028e0 2023-10-17T01:12:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.904+0000 7fa903fff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9040a0bf0 0x7fa9040a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.904+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9140cb9c0 con 0x7fa91c1028e0 2023-10-17T01:12:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.905+0000 7fa91affd700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9040a0bf0 0x7fa9040a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.905+0000 7fa91affd700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9040a0bf0 0x7fa9040a30b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000c00 tx=0x7fa90c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:38.907+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fa9140957e0 con 0x7fa91c1028e0 2023-10-17T01:12:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6834/1725566254,v1:172.21.15.60:6835/1725566254] up:boot 2023-10-17T01:12:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: fsmap 5 up:standby 2023-10-17T01:12:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-17T01:12:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:39.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:39.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: Deploying daemon mds.f on smithi181 2023-10-17T01:12:39.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:38 smithi045 ceph-mon[98474]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:39.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.167+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa91c107c70 con 0x7fa91c1028e0 2023-10-17T01:12:39.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.168+0000 7fa903fff700 1 -- 172.21.15.45:0/3491363132 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4288 (secure 0 0 0) 0x7fa914098e00 con 0x7fa91c1028e0 2023-10-17T01:12:39.168 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:39.169 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-10-17T01:12:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9040a0bf0 msgr2=0x7fa9040a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9040a0bf0 0x7fa9040a30b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000c00 tx=0x7fa90c005fb0 comp rx=0 tx=0).stop 2023-10-17T01:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 msgr2=0x7fa91c073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 0x7fa91c073680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa91400efc0 tx=0x7fa91400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 shutdown_connections 2023-10-17T01:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9040a0bf0 0x7fa9040a30b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c1028e0 0x7fa91c073680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa91c100d50 0x7fa91c074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 --2- 172.21.15.45:0/3491363132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91c101f40 0x7fa91c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 >> 172.21.15.45:0/3491363132 conn(0x7fa91c0fc280 msgr2=0x7fa91c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 shutdown_connections 2023-10-17T01:12:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:39.171+0000 7fa9217ff700 1 -- 172.21.15.45:0/3491363132 wait complete. 2023-10-17T01:12:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2023-10-17T01:12:39.309 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6834/1725566254,v1:172.21.15.60:6835/1725566254] up:boot 2023-10-17T01:12:39.310 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: fsmap 5 up:standby 2023-10-17T01:12:39.310 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-17T01:12:39.310 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.310 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.310 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.311 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:39.311 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:39.311 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:39.311 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: Deploying daemon mds.f on smithi181 2023-10-17T01:12:39.311 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:38 smithi181 ceph-mon[134520]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:39.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6834/1725566254,v1:172.21.15.60:6835/1725566254] up:boot 2023-10-17T01:12:39.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: fsmap 5 up:standby 2023-10-17T01:12:39.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-17T01:12:39.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:39.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:39.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:39.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:39.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: Deploying daemon mds.f on smithi181 2023-10-17T01:12:39.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:38 smithi060 ceph-mon[134136]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:39.657 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:12:39 smithi181 systemd[1]: Starting Ceph mds.f for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:40.135 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:40 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3491363132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:40.135 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:12:39 smithi181 podman[171370]: 2023-10-17T01:12:40.135 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:12:40 smithi181 ceph-mds[171410]: -- 172.21.15.181:0/573696038 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f5e02991e0 con 0x55f5e026ec00 2023-10-17T01:12:40.136 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:12:40 smithi181 bash[171370]: 61e06affa1908a99712940e1a80ec2cf5e2945fbfe4a8757b74da48bf00a1dd0 2023-10-17T01:12:40.136 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:12:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f[171386]: starting mds.f at 2023-10-17T01:12:40.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:40 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3491363132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:40.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:40 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3491363132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:40.407 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:12:40 smithi181 systemd[1]: Started Ceph mds.f for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:41.191 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:41.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 -- 172.21.15.45:0/1738717357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4073750 msgr2=0x7fdce4073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 --2- 172.21.15.45:0/1738717357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4073750 0x7fdce4073bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009990 tx=0x7fdcd8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 -- 172.21.15.45:0/1738717357 shutdown_connections 2023-10-17T01:12:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 --2- 172.21.15.45:0/1738717357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4108c30 0x7fdce410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 --2- 172.21.15.45:0/1738717357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4074d30 0x7fdce4073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 --2- 172.21.15.45:0/1738717357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4073750 0x7fdce4073bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 -- 172.21.15.45:0/1738717357 >> 172.21.15.45:0/1738717357 conn(0x7fdce40fc290 msgr2=0x7fdce40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 -- 172.21.15.45:0/1738717357 shutdown_connections 2023-10-17T01:12:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.407+0000 7fdce8dff700 1 -- 172.21.15.45:0/1738717357 wait complete. 2023-10-17T01:12:41.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:41.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:41.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:41.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: Deploying daemon mds.c on smithi181 2023-10-17T01:12:41.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:41.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] up:boot 2023-10-17T01:12:41.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: fsmap 6 up:standby 2023-10-17T01:12:41.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-17T01:12:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 Processor -- start 2023-10-17T01:12:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 -- start start 2023-10-17T01:12:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4073750 0x7fdce4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 0x7fdce4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 0x7fdce41a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41a0760 con 0x7fdce4074d30 2023-10-17T01:12:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41a08d0 con 0x7fdce4073750 2023-10-17T01:12:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.408+0000 7fdce8dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41a0a90 con 0x7fdce4108c30 2023-10-17T01:12:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 0x7fdce4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 0x7fdce41a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 0x7fdce41a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34656/0 (socket says 172.21.15.45:34656) 2023-10-17T01:12:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4073750 0x7fdce4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 0x7fdce4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38210/0 (socket says 172.21.15.45:38210) 2023-10-17T01:12:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 -- 172.21.15.45:0/495791635 learned_addr learned my addr 172.21.15.45:0/495791635 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 -- 172.21.15.45:0/495791635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4073750 msgr2=0x7fdce4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4073750 0x7fdce4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 -- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 msgr2=0x7fdce4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 0x7fdce4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 -- 172.21.15.45:0/495791635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80095f0 con 0x7fdce4108c30 2023-10-17T01:12:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce1d9b700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 0x7fdce4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.409+0000 7fdce2d9d700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 0x7fdce41a0190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00ef30 tx=0x7fdcdc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.410+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc018070 con 0x7fdce4108c30 2023-10-17T01:12:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.410+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc00f040 con 0x7fdce4108c30 2023-10-17T01:12:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.410+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce41a0d10 con 0x7fdce4108c30 2023-10-17T01:12:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.410+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00cd00 con 0x7fdce4108c30 2023-10-17T01:12:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.410+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce41a11c0 con 0x7fdce4108c30 2023-10-17T01:12:41.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.411+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce4107330 con 0x7fdce4108c30 2023-10-17T01:12:41.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.413+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdcdc003a00 con 0x7fdce4108c30 2023-10-17T01:12:41.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.414+0000 7fdcd37fe700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdccc0a0bf0 0x7fdccc0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:41.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.414+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcdc0cb970 con 0x7fdce4108c30 2023-10-17T01:12:41.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.414+0000 7fdce259c700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdccc0a0bf0 0x7fdccc0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:41.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.415+0000 7fdce259c700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdccc0a0bf0 0x7fdccc0a30b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdce419a0f0 tx=0x7fdcd400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:41.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.416+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fdcdc095790 con 0x7fdce4108c30 2023-10-17T01:12:41.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:41.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: Deploying daemon mds.c on smithi181 2023-10-17T01:12:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] up:boot 2023-10-17T01:12:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: fsmap 6 up:standby 2023-10-17T01:12:41.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-17T01:12:41.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:41.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:41.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:41.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:41.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: Deploying daemon mds.c on smithi181 2023-10-17T01:12:41.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:41.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] up:boot 2023-10-17T01:12:41.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: fsmap 6 up:standby 2023-10-17T01:12:41.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-17T01:12:41.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.680+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdce402cc30 con 0x7fdce4108c30 2023-10-17T01:12:41.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.682+0000 7fdcd37fe700 1 -- 172.21.15.45:0/495791635 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5048 (secure 0 0 0) 0x7fdcdc098db0 con 0x7fdce4108c30 2023-10-17T01:12:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:41.682 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-10-17T01:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdccc0a0bf0 msgr2=0x7fdccc0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdccc0a0bf0 0x7fdccc0a30b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdce419a0f0 tx=0x7fdcd400b410 comp rx=0 tx=0).stop 2023-10-17T01:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 msgr2=0x7fdce41a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 0x7fdce41a0190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00ef30 tx=0x7fdcdc00c3a0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 shutdown_connections 2023-10-17T01:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdccc0a0bf0 0x7fdccc0a30b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4108c30 0x7fdce41a0190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4073750 0x7fdce4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 --2- 172.21.15.45:0/495791635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4074d30 0x7fdce4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.684+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 >> 172.21.15.45:0/495791635 conn(0x7fdce40fc290 msgr2=0x7fdce40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.685+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 shutdown_connections 2023-10-17T01:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:41.685+0000 7fdce8dff700 1 -- 172.21.15.45:0/495791635 wait complete. 2023-10-17T01:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2023-10-17T01:12:41.907 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:12:41 smithi181 systemd[1]: Starting Ceph mds.c for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:42.324 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:12:42 smithi181 podman[171710]: 2023-10-17T01:12:42.324 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:42 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/495791635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:42.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:42 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/495791635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:42.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:42 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/495791635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:42.657 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:12:42 smithi181 ceph-mds[171750]: -- 172.21.15.181:0/3957105272 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55900b0ff1e0 con 0x55900b0d4c00 2023-10-17T01:12:42.657 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:12:42 smithi181 bash[171710]: 87760045607caff3440d731ad3b014697e87707b562862c8347b323a386097b2 2023-10-17T01:12:42.658 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:12:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c[171726]: starting mds.c at 2023-10-17T01:12:42.658 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:12:42 smithi181 systemd[1]: Started Ceph mds.c for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:43.692 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.692 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.692 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:43.692 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.693 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:43.693 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:43.693 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:43.693 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: Deploying daemon mds.h on smithi060 2023-10-17T01:12:43.693 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] up:boot 2023-10-17T01:12:43.694 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: fsmap 7 up:standby 2023-10-17T01:12:43.694 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-17T01:12:43.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:43.772 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.772 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:43.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:43.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:43.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:43.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: Deploying daemon mds.h on smithi060 2023-10-17T01:12:43.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] up:boot 2023-10-17T01:12:43.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: fsmap 7 up:standby 2023-10-17T01:12:43.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-17T01:12:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:43.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: Deploying daemon mds.h on smithi060 2023-10-17T01:12:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] up:boot 2023-10-17T01:12:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: fsmap 7 up:standby 2023-10-17T01:12:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-17T01:12:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.919+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/1001674762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a541028e0 msgr2=0x7f6a54109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.919+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/1001674762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a541028e0 0x7f6a54109190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00b0d0 tx=0x7f6a4c00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.919+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/1001674762 shutdown_connections 2023-10-17T01:12:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.920+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/1001674762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a541028e0 0x7f6a54109190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.920+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/1001674762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a54100d50 0x7f6a54101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.920+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/1001674762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a54101f40 0x7f6a541023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.920+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/1001674762 >> 172.21.15.45:0/1001674762 conn(0x7f6a540fc280 msgr2=0x7f6a540fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.920+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/1001674762 shutdown_connections 2023-10-17T01:12:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.920+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/1001674762 wait complete. 2023-10-17T01:12:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 Processor -- start 2023-10-17T01:12:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 -- start start 2023-10-17T01:12:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a54100d50 0x7f6a54198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 0x7f6a54199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a541028e0 0x7f6a541a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541a0860 con 0x7f6a541028e0 2023-10-17T01:12:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541a09d0 con 0x7f6a54101f40 2023-10-17T01:12:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5c5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541a0b90 con 0x7f6a54100d50 2023-10-17T01:12:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a59b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 0x7f6a54199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a59b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 0x7f6a54199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46282/0 (socket says 172.21.15.45:46282) 2023-10-17T01:12:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a59b9a700 1 -- 172.21.15.45:0/3216107192 learned_addr learned my addr 172.21.15.45:0/3216107192 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5a39b700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a54100d50 0x7f6a54198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.921+0000 7f6a5ab9c700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a541028e0 0x7f6a541a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a59b9a700 1 -- 172.21.15.45:0/3216107192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a54100d50 msgr2=0x7f6a54198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a59b9a700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a54100d50 0x7f6a54198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a59b9a700 1 -- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a541028e0 msgr2=0x7f6a541a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a59b9a700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a541028e0 0x7f6a541a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a59b9a700 1 -- 172.21.15.45:0/3216107192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c009d00 con 0x7f6a54101f40 2023-10-17T01:12:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a59b9a700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 0x7f6a54199260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400e9a0 tx=0x7f6a4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44009790 con 0x7f6a54101f40 2023-10-17T01:12:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a44010430 con 0x7f6a54101f40 2023-10-17T01:12:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44018680 con 0x7f6a54101f40 2023-10-17T01:12:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.922+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a541a0e70 con 0x7f6a54101f40 2023-10-17T01:12:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.923+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a541a1240 con 0x7f6a54101f40 2023-10-17T01:12:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.923+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5410c5c0 con 0x7f6a54101f40 2023-10-17T01:12:43.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.926+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f6a440105a0 con 0x7f6a54101f40 2023-10-17T01:12:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.927+0000 7f6a4b7fe700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6a400a0bf0 0x7f6a400a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.927+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6a44014070 con 0x7f6a54101f40 2023-10-17T01:12:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.927+0000 7f6a5a39b700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6a400a0bf0 0x7f6a400a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.928+0000 7f6a5a39b700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6a400a0bf0 0x7f6a400a30b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6a5419a1f0 tx=0x7f6a50007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:43.929+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f6a44095e50 con 0x7f6a54101f40 2023-10-17T01:12:44.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.190+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a5410c800 con 0x7f6a54101f40 2023-10-17T01:12:44.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.191+0000 7f6a4b7fe700 1 -- 172.21.15.45:0/3216107192 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7f6a44026090 con 0x7f6a54101f40 2023-10-17T01:12:44.192 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:44.192 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-10-17T01:12:44.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6a400a0bf0 msgr2=0x7f6a400a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6a400a0bf0 0x7f6a400a30b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6a5419a1f0 tx=0x7f6a50007400 comp rx=0 tx=0).stop 2023-10-17T01:12:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 msgr2=0x7f6a54199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 0x7f6a54199260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400e9a0 tx=0x7f6a4400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:12:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 shutdown_connections 2023-10-17T01:12:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6a400a0bf0 0x7f6a400a30b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:44.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a541028e0 0x7f6a541a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:44.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a54100d50 0x7f6a54198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:44.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 --2- 172.21.15.45:0/3216107192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a54101f40 0x7f6a54199260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:44.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 >> 172.21.15.45:0/3216107192 conn(0x7f6a540fc280 msgr2=0x7f6a54107790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:44.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 shutdown_connections 2023-10-17T01:12:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:44.194+0000 7f6a5c5ff700 1 -- 172.21.15.45:0/3216107192 wait complete. 2023-10-17T01:12:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-17T01:12:44.254 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:12:43 smithi060 systemd[1]: Starting Ceph mds.h for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:44.542 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:44 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3216107192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:44.543 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:12:44 smithi060 podman[170656]: 2023-10-17T01:12:44.543 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:12:44 smithi060 bash[170656]: 88f89fc3afeec205b28350d93cb06430ea43db869f39443671a4911e581189de 2023-10-17T01:12:44.543 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:12:44 smithi060 ceph-mds[170696]: -- 172.21.15.60:0/2998901907 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ea15e11e0 con 0x562ea15b6800 2023-10-17T01:12:44.543 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:12:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h[170672]: starting mds.h at 2023-10-17T01:12:44.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:44 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3216107192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:44.828 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:12:44 smithi060 systemd[1]: Started Ceph mds.h for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:44 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3216107192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:45.526 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:45.527 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.527 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.527 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.527 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:45.528 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:45.528 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:45.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:45.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:45.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:45.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:45.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:46.158 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:12:46 smithi181 systemd[1]: Starting Ceph mds.i for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:46.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 -- 172.21.15.45:0/3623671070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 msgr2=0x7f156c1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 --2- 172.21.15.45:0/3623671070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1023a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f155c0099c0 tx=0x7f155c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 -- 172.21.15.45:0/3623671070 shutdown_connections 2023-10-17T01:12:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 --2- 172.21.15.45:0/3623671070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 0x7f156c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 --2- 172.21.15.45:0/3623671070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 --2- 172.21.15.45:0/3623671070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1023a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.430+0000 7f1573fff700 1 -- 172.21.15.45:0/3623671070 >> 172.21.15.45:0/3623671070 conn(0x7f156c0fc280 msgr2=0x7f156c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.431+0000 7f1573fff700 1 -- 172.21.15.45:0/3623671070 shutdown_connections 2023-10-17T01:12:46.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.431+0000 7f1573fff700 1 -- 172.21.15.45:0/3623671070 wait complete. 2023-10-17T01:12:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 Processor -- start 2023-10-17T01:12:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 -- start start 2023-10-17T01:12:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 0x7f156c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1a07c0 con 0x7f156c1028e0 2023-10-17T01:12:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1a0930 con 0x7f156c101f40 2023-10-17T01:12:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1573fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1a0af0 con 0x7f156c100d50 2023-10-17T01:12:46.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1571d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:46.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1571d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34688/0 (socket says 172.21.15.45:34688) 2023-10-17T01:12:46.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f1571d9b700 1 -- 172.21.15.45:0/138539022 learned_addr learned my addr 172.21.15.45:0/138539022 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f157159a700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.432+0000 7f157259c700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 0x7f156c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1571d9b700 1 -- 172.21.15.45:0/138539022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 msgr2=0x7f156c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1571d9b700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1571d9b700 1 -- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 msgr2=0x7f156c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1571d9b700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 0x7f156c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1571d9b700 1 -- 172.21.15.45:0/138539022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0095f0 con 0x7f156c100d50 2023-10-17T01:12:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f157159a700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f157259c700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 0x7f156c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1571d9b700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c198c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f156800d980 tx=0x7f156800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568004020 con 0x7f156c100d50 2023-10-17T01:12:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c1a0dd0 con 0x7f156c100d50 2023-10-17T01:12:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156800f690 con 0x7f156c100d50 2023-10-17T01:12:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.433+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568003a60 con 0x7f156c100d50 2023-10-17T01:12:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.434+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c1a1250 con 0x7f156c100d50 2023-10-17T01:12:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.435+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c10c5b0 con 0x7f156c100d50 2023-10-17T01:12:46.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.436+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f156800fb00 con 0x7f156c100d50 2023-10-17T01:12:46.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.438+0000 7f1562ffd700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15580a0b70 0x7f15580a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:46.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.438+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f15680cb940 con 0x7f156c100d50 2023-10-17T01:12:46.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.438+0000 7f157159a700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15580a0b70 0x7f15580a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:46.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.438+0000 7f157159a700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15580a0b70 0x7f15580a3030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b580 tx=0x7f155c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:46.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.442+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f15680956d0 con 0x7f156c100d50 2023-10-17T01:12:46.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:46 smithi045 ceph-mon[98474]: Deploying daemon mds.i on smithi181 2023-10-17T01:12:46.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:46 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] up:boot 2023-10-17T01:12:46.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:46 smithi045 ceph-mon[98474]: fsmap 8 up:standby 2023-10-17T01:12:46.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-17T01:12:46.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:46 smithi181 ceph-mon[134520]: Deploying daemon mds.i on smithi181 2023-10-17T01:12:46.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:46 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] up:boot 2023-10-17T01:12:46.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:46 smithi181 ceph-mon[134520]: fsmap 8 up:standby 2023-10-17T01:12:46.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-17T01:12:46.658 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:12:46 smithi181 podman[172053]: 2023-10-17T01:12:46.659 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:12:46 smithi181 ceph-mds[172093]: -- 172.21.15.181:0/3228791540 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56032ab371e0 con 0x56032ab0cc00 2023-10-17T01:12:46.659 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:12:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i[172069]: starting mds.i at 2023-10-17T01:12:46.659 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:12:46 smithi181 bash[172053]: edd3cab65ebb639584ba147cc3c9e05491853f3484e15e2d5032fc52c90211f9 2023-10-17T01:12:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.703+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f156c1a18c0 con 0x7f156c100d50 2023-10-17T01:12:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.704+0000 7f1562ffd700 1 -- 172.21.15.45:0/138539022 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6565 (secure 0 0 0) 0x7f1568098cf0 con 0x7f156c100d50 2023-10-17T01:12:46.705 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:46.705 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-10-17T01:12:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15580a0b70 msgr2=0x7f15580a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15580a0b70 0x7f15580a3030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b580 tx=0x7f155c01a040 comp rx=0 tx=0).stop 2023-10-17T01:12:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 msgr2=0x7f156c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c198c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f156800d980 tx=0x7f156800dc90 comp rx=0 tx=0).stop 2023-10-17T01:12:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 shutdown_connections 2023-10-17T01:12:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15580a0b70 0x7f15580a3030 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f156c1028e0 0x7f156c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f156c100d50 0x7f156c198c80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 --2- 172.21.15.45:0/138539022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c101f40 0x7f156c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 >> 172.21.15.45:0/138539022 conn(0x7f156c0fc280 msgr2=0x7f156c0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 shutdown_connections 2023-10-17T01:12:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:46.706+0000 7f1573fff700 1 -- 172.21.15.45:0/138539022 wait complete. 2023-10-17T01:12:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-10-17T01:12:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:46 smithi060 ceph-mon[134136]: Deploying daemon mds.i on smithi181 2023-10-17T01:12:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:46 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] up:boot 2023-10-17T01:12:46.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:46 smithi060 ceph-mon[134136]: fsmap 8 up:standby 2023-10-17T01:12:46.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-17T01:12:47.157 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:12:46 smithi181 systemd[1]: Started Ceph mds.i for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/138539022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:47.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:47.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/138539022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:47.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:47.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/138539022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:47.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:47.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:47.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:48.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:48.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:48 smithi045 ceph-mon[98474]: Deploying daemon mds.g on smithi045 2023-10-17T01:12:48.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:48 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] up:boot 2023-10-17T01:12:48.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:48 smithi045 ceph-mon[98474]: fsmap 9 up:standby 2023-10-17T01:12:48.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-17T01:12:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:48 smithi060 ceph-mon[134136]: Deploying daemon mds.g on smithi045 2023-10-17T01:12:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:48 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] up:boot 2023-10-17T01:12:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:48 smithi060 ceph-mon[134136]: fsmap 9 up:standby 2023-10-17T01:12:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-17T01:12:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:48 smithi181 ceph-mon[134520]: Deploying daemon mds.g on smithi045 2023-10-17T01:12:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:48 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] up:boot 2023-10-17T01:12:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:48 smithi181 ceph-mon[134520]: fsmap 9 up:standby 2023-10-17T01:12:48.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-17T01:12:48.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 -- 172.21.15.45:0/2096785570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b01028e0 msgr2=0x7ff5b0109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:48.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/2096785570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b01028e0 0x7ff5b0109190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800b0d0 tx=0x7ff5a800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:48.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 -- 172.21.15.45:0/2096785570 shutdown_connections 2023-10-17T01:12:48.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/2096785570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b01028e0 0x7ff5b0109190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:48.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/2096785570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 0x7ff5b0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:48.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/2096785570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b0101f40 0x7ff5b01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:48.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.954+0000 7ff5b49ff700 1 -- 172.21.15.45:0/2096785570 >> 172.21.15.45:0/2096785570 conn(0x7ff5b00fc280 msgr2=0x7ff5b00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:48.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.955+0000 7ff5b49ff700 1 -- 172.21.15.45:0/2096785570 shutdown_connections 2023-10-17T01:12:48.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.955+0000 7ff5b49ff700 1 -- 172.21.15.45:0/2096785570 wait complete. 2023-10-17T01:12:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 Processor -- start 2023-10-17T01:12:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 -- start start 2023-10-17T01:12:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 0x7ff5b0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 0x7ff5b0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b01028e0 0x7ff5b01a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a0890 con 0x7ff5b0101f40 2023-10-17T01:12:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a0a00 con 0x7ff5b01028e0 2023-10-17T01:12:48.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.956+0000 7ff5b49ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b01a0bc0 con 0x7ff5b0100d50 2023-10-17T01:12:48.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 0x7ff5b0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:48.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b01028e0 0x7ff5b01a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:48.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 0x7ff5b0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58792/0 (socket says 172.21.15.45:58792) 2023-10-17T01:12:48.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5add9b700 1 -- 172.21.15.45:0/1369435678 learned_addr learned my addr 172.21.15.45:0/1369435678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:48.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5ae59c700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 0x7ff5b0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:48.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 -- 172.21.15.45:0/1369435678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 msgr2=0x7ff5b0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:48.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 0x7ff5b0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:48.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 -- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 msgr2=0x7ff5b0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:48.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 0x7ff5b0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:48.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 -- 172.21.15.45:0/1369435678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4009520 con 0x7ff5b01028e0 2023-10-17T01:12:48.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5ae59c700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 0x7ff5b0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:12:48.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5add9b700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 0x7ff5b0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:48.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff5aed9d700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b01028e0 0x7ff5b01a02c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8007a30 tx=0x7ff5a8008eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:48.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.957+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a800e040 con 0x7ff5b01028e0 2023-10-17T01:12:48.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.958+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a8009d00 con 0x7ff5b01028e0 2023-10-17T01:12:48.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.958+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5a800bdb0 con 0x7ff5b01028e0 2023-10-17T01:12:48.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.958+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a801d470 con 0x7ff5b01028e0 2023-10-17T01:12:48.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.958+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01a1130 con 0x7ff5b01028e0 2023-10-17T01:12:48.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.959+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b010c5c0 con 0x7ff5b01028e0 2023-10-17T01:12:48.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.962+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff5a8005120 con 0x7ff5b01028e0 2023-10-17T01:12:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.964+0000 7ff59f7fe700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5980a0bf0 0x7ff5980a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.964+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5a80cd650 con 0x7ff5b01028e0 2023-10-17T01:12:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.964+0000 7ff5ae59c700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5980a0bf0 0x7ff5980a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.964+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7ff5a8097590 con 0x7ff5b01028e0 2023-10-17T01:12:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:48.965+0000 7ff5ae59c700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5980a0bf0 0x7ff5980a30b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5b019a220 tx=0x7ff5a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.232+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5b01a16e0 con 0x7ff5b01028e0 2023-10-17T01:12:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.233+0000 7ff59f7fe700 1 -- 172.21.15.45:0/1369435678 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7ff5a8095e60 con 0x7ff5b01028e0 2023-10-17T01:12:49.234 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:49.235 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-10-17T01:12:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5980a0bf0 msgr2=0x7ff5980a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5980a0bf0 0x7ff5980a30b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5b019a220 tx=0x7ff5a0009260 comp rx=0 tx=0).stop 2023-10-17T01:12:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b01028e0 msgr2=0x7ff5b01a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b01028e0 0x7ff5b01a02c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8007a30 tx=0x7ff5a8008eb0 comp rx=0 tx=0).stop 2023-10-17T01:12:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 shutdown_connections 2023-10-17T01:12:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5980a0bf0 0x7ff5980a30b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b01028e0 0x7ff5b01a02c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5b0100d50 0x7ff5b0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 --2- 172.21.15.45:0/1369435678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5b0101f40 0x7ff5b0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.235+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 >> 172.21.15.45:0/1369435678 conn(0x7ff5b00fc280 msgr2=0x7ff5b00fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.236+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 shutdown_connections 2023-10-17T01:12:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:49.236+0000 7ff5b49ff700 1 -- 172.21.15.45:0/1369435678 wait complete. 2023-10-17T01:12:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2023-10-17T01:12:49.440 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:12:49 smithi045 systemd[1]: Starting Ceph mds.g for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:49.701 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:12:49 smithi045 podman[148715]: 2023-10-17T01:12:49.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:49 smithi045 ceph-mon[98474]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:49.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:49 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1369435678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:49 smithi060 ceph-mon[134136]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:49 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1369435678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:49 smithi181 ceph-mon[134520]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:49 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1369435678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:50.051 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:12:49 smithi045 bash[148715]: a0215e2ff39f0dee04c30ad1c4f684e4e007980fb0af0c8a837985f01b8c0a42 2023-10-17T01:12:50.051 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:12:49 smithi045 ceph-mds[148755]: -- 172.21.15.45:0/4264571453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5648602231e0 con 0x5648601fac00 2023-10-17T01:12:50.052 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:12:49 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g[148731]: starting mds.g at 2023-10-17T01:12:50.052 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:12:49 smithi045 systemd[1]: Started Ceph mds.g for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:51.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:51.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:51.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: Deploying daemon mds.a on smithi045 2023-10-17T01:12:51.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:51.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.45:6838/931419854,v1:172.21.15.45:6839/931419854] up:boot 2023-10-17T01:12:51.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: fsmap 10 up:standby 2023-10-17T01:12:51.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:50 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-17T01:12:51.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:51.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:51.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:51.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: Deploying daemon mds.a on smithi045 2023-10-17T01:12:51.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:51.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.45:6838/931419854,v1:172.21.15.45:6839/931419854] up:boot 2023-10-17T01:12:51.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: fsmap 10 up:standby 2023-10-17T01:12:51.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:50 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-17T01:12:51.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:51.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: Deploying daemon mds.a on smithi045 2023-10-17T01:12:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:51.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.45:6838/931419854,v1:172.21.15.45:6839/931419854] up:boot 2023-10-17T01:12:51.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: fsmap 10 up:standby 2023-10-17T01:12:51.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:50 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-17T01:12:51.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 -- 172.21.15.45:0/1667991692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1304101f40 msgr2=0x7f13041023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:51.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 --2- 172.21.15.45:0/1667991692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1304101f40 0x7f13041023a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f12f80099c0 tx=0x7f12f8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 -- 172.21.15.45:0/1667991692 shutdown_connections 2023-10-17T01:12:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 --2- 172.21.15.45:0/1667991692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13041028e0 0x7f1304109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 --2- 172.21.15.45:0/1667991692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1304100d50 0x7f1304101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 --2- 172.21.15.45:0/1667991692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1304101f40 0x7f13041023a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 -- 172.21.15.45:0/1667991692 >> 172.21.15.45:0/1667991692 conn(0x7f13040fc280 msgr2=0x7f13040fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:51.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 -- 172.21.15.45:0/1667991692 shutdown_connections 2023-10-17T01:12:51.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.466+0000 7f13089ff700 1 -- 172.21.15.45:0/1667991692 wait complete. 2023-10-17T01:12:51.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 Processor -- start 2023-10-17T01:12:51.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 -- start start 2023-10-17T01:12:51.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 0x7f1304074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:51.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1304101f40 0x7f1304073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:51.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13041028e0 0x7f1304073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:51.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304073e20 con 0x7f13041028e0 2023-10-17T01:12:51.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304073f90 con 0x7f1304100d50 2023-10-17T01:12:51.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f13089ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041a0af0 con 0x7f1304101f40 2023-10-17T01:12:51.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f130259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 0x7f1304074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:51.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f1302d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13041028e0 0x7f1304073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:51.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f130259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 0x7f1304074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38674/0 (socket says 172.21.15.45:38674) 2023-10-17T01:12:51.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.467+0000 7f130259c700 1 -- 172.21.15.45:0/3404798093 learned_addr learned my addr 172.21.15.45:0/3404798093 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:51.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f1301d9b700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1304101f40 0x7f1304073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:51.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f130259c700 1 -- 172.21.15.45:0/3404798093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1304101f40 msgr2=0x7f1304073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:51.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f130259c700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1304101f40 0x7f1304073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f130259c700 1 -- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13041028e0 msgr2=0x7f1304073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:51.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f130259c700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13041028e0 0x7f1304073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f130259c700 1 -- 172.21.15.45:0/3404798093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80095f0 con 0x7f1304100d50 2023-10-17T01:12:51.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f1302d9d700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13041028e0 0x7f1304073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:51.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f130259c700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 0x7f1304074b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12f400d980 tx=0x7f12f400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:51.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.468+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400f890 con 0x7f1304100d50 2023-10-17T01:12:51.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.469+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13041a0dd0 con 0x7f1304100d50 2023-10-17T01:12:51.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.469+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f4010460 con 0x7f1304100d50 2023-10-17T01:12:51.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.469+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40195c0 con 0x7f1304100d50 2023-10-17T01:12:51.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.469+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13041a1250 con 0x7f1304100d50 2023-10-17T01:12:51.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.470+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13041082f0 con 0x7f1304100d50 2023-10-17T01:12:51.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.472+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f12f40105d0 con 0x7f1304100d50 2023-10-17T01:12:51.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.473+0000 7f12f37fe700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ec0a0bf0 0x7f12ec0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:51.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.473+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12f40cbb00 con 0x7f1304100d50 2023-10-17T01:12:51.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.473+0000 7f1301d9b700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ec0a0bf0 0x7f12ec0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:51.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.474+0000 7f1301d9b700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ec0a0bf0 0x7f12ec0a30b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b580 tx=0x7f12f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:51.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.475+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f12f4095890 con 0x7f1304100d50 2023-10-17T01:12:51.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.786+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f130410c5b0 con 0x7f1304100d50 2023-10-17T01:12:51.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.787+0000 7f12f37fe700 1 -- 172.21.15.45:0/3404798093 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8082 (secure 0 0 0) 0x7f12f4098eb0 con 0x7f1304100d50 2023-10-17T01:12:51.789 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:51.790 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-10-17T01:12:51.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ec0a0bf0 msgr2=0x7f12ec0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:51.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ec0a0bf0 0x7f12ec0a30b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b580 tx=0x7f12f8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 msgr2=0x7f1304074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:51.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 0x7f1304074b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12f400d980 tx=0x7f12f400dc90 comp rx=0 tx=0).stop 2023-10-17T01:12:51.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 shutdown_connections 2023-10-17T01:12:51.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ec0a0bf0 0x7f12ec0a30b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13041028e0 0x7f1304073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1304100d50 0x7f1304074b00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 --2- 172.21.15.45:0/3404798093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1304101f40 0x7f1304073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:51.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 >> 172.21.15.45:0/3404798093 conn(0x7f13040fc280 msgr2=0x7f1304105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:51.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 shutdown_connections 2023-10-17T01:12:51.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:51.790+0000 7f13089ff700 1 -- 172.21.15.45:0/3404798093 wait complete. 2023-10-17T01:12:51.794 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2023-10-17T01:12:52.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:51 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3404798093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:52.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:51 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3404798093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:52.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:51 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3404798093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:52.551 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:12:52 smithi045 systemd[1]: Starting Ceph mds.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:53.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:52 smithi045 ceph-mon[98474]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:53.048 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:12:52 smithi045 podman[149109]: 2023-10-17T01:12:53.301 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:12:53 smithi045 ceph-mds[149149]: -- 172.21.15.45:0/3359327107 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8266231e0 con 0x55a8265fb000 2023-10-17T01:12:53.301 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:12:53 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a[149125]: starting mds.a at 2023-10-17T01:12:53.302 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:12:53 smithi045 bash[149109]: 3a024fcced10c68848f29754013955943d0b595c413ad63d366b3b2e499b5ebb 2023-10-17T01:12:53.302 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:12:53 smithi045 systemd[1]: Started Ceph mds.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:52 smithi060 ceph-mon[134136]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:53.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:52 smithi181 ceph-mon[134520]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:53.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:54.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 -- 172.21.15.45:0/3820194840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 msgr2=0x7f4084109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:54.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 --2- 172.21.15.45:0/3820194840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 0x7f4084109190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b0d0 tx=0x7f407c00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 -- 172.21.15.45:0/3820194840 shutdown_connections 2023-10-17T01:12:54.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 --2- 172.21.15.45:0/3820194840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 0x7f4084109190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 --2- 172.21.15.45:0/3820194840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4084100d50 0x7f4084101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 --2- 172.21.15.45:0/3820194840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084101f40 0x7f40841023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.026+0000 7f4088dff700 1 -- 172.21.15.45:0/3820194840 >> 172.21.15.45:0/3820194840 conn(0x7f40840fc280 msgr2=0x7f40840fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:54.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.027+0000 7f4088dff700 1 -- 172.21.15.45:0/3820194840 shutdown_connections 2023-10-17T01:12:54.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.027+0000 7f4088dff700 1 -- 172.21.15.45:0/3820194840 wait complete. 2023-10-17T01:12:54.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 Processor -- start 2023-10-17T01:12:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 -- start start 2023-10-17T01:12:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 0x7f4084196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4084101f40 0x7f4084197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 0x7f408419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419e690 con 0x7f4084100d50 2023-10-17T01:12:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419e7d0 con 0x7f40841028e0 2023-10-17T01:12:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4088dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419e960 con 0x7f4084101f40 2023-10-17T01:12:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f408259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 0x7f4084196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f408259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 0x7f4084196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58870/0 (socket says 172.21.15.45:58870) 2023-10-17T01:12:54.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f408259c700 1 -- 172.21.15.45:0/626024231 learned_addr learned my addr 172.21.15.45:0/626024231 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.028+0000 7f4081d9b700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4084101f40 0x7f4084197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f4082d9d700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 0x7f408419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f408259c700 1 -- 172.21.15.45:0/626024231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4084101f40 msgr2=0x7f4084197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f408259c700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4084101f40 0x7f4084197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f408259c700 1 -- 172.21.15.45:0/626024231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 msgr2=0x7f408419e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:54.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f408259c700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 0x7f408419e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f408259c700 1 -- 172.21.15.45:0/626024231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c009d00 con 0x7f4084100d50 2023-10-17T01:12:54.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f408259c700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 0x7f4084196b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f407400e970 tx=0x7f407400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:54.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074009790 con 0x7f4084100d50 2023-10-17T01:12:54.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f407400c7e0 con 0x7f4084100d50 2023-10-17T01:12:54.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408419ec40 con 0x7f4084100d50 2023-10-17T01:12:54.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074003c70 con 0x7f4084100d50 2023-10-17T01:12:54.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.029+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408419f150 con 0x7f4084100d50 2023-10-17T01:12:54.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.030+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4084108260 con 0x7f4084100d50 2023-10-17T01:12:54.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.032+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f407400cc50 con 0x7f4084100d50 2023-10-17T01:12:54.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.034+0000 7f40737fe700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f406c0a0b70 0x7f406c0a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:54.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.034+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4074014070 con 0x7f4084100d50 2023-10-17T01:12:54.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.034+0000 7f4081d9b700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f406c0a0b70 0x7f406c0a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:54.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.035+0000 7f4081d9b700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f406c0a0b70 0x7f406c0a3030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4078005950 tx=0x7f407800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:54.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.038+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f4074095ce0 con 0x7f4084100d50 2023-10-17T01:12:54.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.300+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f408402ce90 con 0x7f4084100d50 2023-10-17T01:12:54.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.300+0000 7f40737fe700 1 -- 172.21.15.45:0/626024231 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7f4074003dd0 con 0x7f4084100d50 2023-10-17T01:12:54.302 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:54.302 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-10-17T01:12:54.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f406c0a0b70 msgr2=0x7f406c0a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:54.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f406c0a0b70 0x7f406c0a3030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4078005950 tx=0x7f407800b410 comp rx=0 tx=0).stop 2023-10-17T01:12:54.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 msgr2=0x7f4084196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:54.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 0x7f4084196b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f407400e970 tx=0x7f407400ec80 comp rx=0 tx=0).stop 2023-10-17T01:12:54.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 shutdown_connections 2023-10-17T01:12:54.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f406c0a0b70 0x7f406c0a3030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40841028e0 0x7f408419e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4084100d50 0x7f4084196b10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 --2- 172.21.15.45:0/626024231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4084101f40 0x7f4084197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:54.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 >> 172.21.15.45:0/626024231 conn(0x7f40840fc280 msgr2=0x7f40841053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:54.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 shutdown_connections 2023-10-17T01:12:54.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:54.303+0000 7f4088dff700 1 -- 172.21.15.45:0/626024231 wait complete. 2023-10-17T01:12:54.306 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2023-10-17T01:12:54.492 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.492 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.492 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.492 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:54.493 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:54.493 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:54.493 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: Deploying daemon mds.b on smithi060 2023-10-17T01:12:54.493 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.45:6840/1667366220,v1:172.21.15.45:6841/1667366220] up:boot 2023-10-17T01:12:54.493 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: fsmap 11 up:standby 2023-10-17T01:12:54.494 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:54 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-17T01:12:54.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:54.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:54.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:54.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: Deploying daemon mds.b on smithi060 2023-10-17T01:12:54.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.45:6840/1667366220,v1:172.21.15.45:6841/1667366220] up:boot 2023-10-17T01:12:54.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: fsmap 11 up:standby 2023-10-17T01:12:54.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:54 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-17T01:12:54.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-17T01:12:54.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-17T01:12:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: Deploying daemon mds.b on smithi060 2023-10-17T01:12:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.45:6840/1667366220,v1:172.21.15.45:6841/1667366220] up:boot 2023-10-17T01:12:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: fsmap 11 up:standby 2023-10-17T01:12:54.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:54 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-17T01:12:54.752 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:12:54 smithi060 systemd[1]: Starting Ceph mds.b for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:12:55.321 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:55 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/626024231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:55.321 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:55 smithi060 ceph-mon[134136]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:55.322 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:12:55 smithi060 podman[170996]: 2023-10-17T01:12:55.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:55 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/626024231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:55.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:55 smithi045 ceph-mon[98474]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:55.578 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:12:55 smithi060 bash[170996]: c04ff3c6305f3a9466dd31154cb2b4ebd56ec75bc9198f98fceeafe5689e26bc 2023-10-17T01:12:55.578 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:12:55 smithi060 ceph-mds[171037]: -- 172.21.15.60:0/1263166798 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55adcbda71e0 con 0x55adcbd7c800 2023-10-17T01:12:55.578 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:12:55 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: starting mds.b at 2023-10-17T01:12:55.578 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:12:55 smithi060 systemd[1]: Started Ceph mds.b for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:12:55.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:55 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/626024231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:55.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:55 smithi181 ceph-mon[134520]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:56.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:12:56.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/1661098412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf81028e0 msgr2=0x7f0cf8109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/1661098412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf81028e0 0x7f0cf8109190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b0d0 tx=0x7f0cf000b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/1661098412 shutdown_connections 2023-10-17T01:12:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/1661098412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf81028e0 0x7f0cf8109190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/1661098412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 0x7f0cf8101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/1661098412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf8101f40 0x7f0cf81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/1661098412 >> 172.21.15.45:0/1661098412 conn(0x7f0cf80fc280 msgr2=0x7f0cf80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/1661098412 shutdown_connections 2023-10-17T01:12:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.538+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/1661098412 wait complete. 2023-10-17T01:12:56.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.539+0000 7f0cfe1ff700 1 Processor -- start 2023-10-17T01:12:56.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.539+0000 7f0cfe1ff700 1 -- start start 2023-10-17T01:12:56.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cfe1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 0x7f0cf8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:56.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cfe1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 0x7f0cf8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:56.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cfe1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf81028e0 0x7f0cf81a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:56.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cfe1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a0880 con 0x7f0cf8101f40 2023-10-17T01:12:56.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cfe1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a09f0 con 0x7f0cf8100d50 2023-10-17T01:12:56.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cfe1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a0bb0 con 0x7f0cf81028e0 2023-10-17T01:12:56.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 0x7f0cf8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:56.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 0x7f0cf8199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58900/0 (socket says 172.21.15.45:58900) 2023-10-17T01:12:56.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cf6ffd700 1 -- 172.21.15.45:0/2236202052 learned_addr learned my addr 172.21.15.45:0/2236202052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cf77fe700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 0x7f0cf8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cf7fff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf81028e0 0x7f0cf81a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.540+0000 7f0cf6ffd700 1 -- 172.21.15.45:0/2236202052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf81028e0 msgr2=0x7f0cf81a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf6ffd700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf81028e0 0x7f0cf81a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf6ffd700 1 -- 172.21.15.45:0/2236202052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 msgr2=0x7f0cf8198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:56.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf6ffd700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 0x7f0cf8198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf6ffd700 1 -- 172.21.15.45:0/2236202052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf0009d00 con 0x7f0cf8101f40 2023-10-17T01:12:56.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf77fe700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 0x7f0cf8198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:56.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf6ffd700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 0x7f0cf8199280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e9a0 tx=0x7f0cec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:56.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec009790 con 0x7f0cf8101f40 2023-10-17T01:12:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cec010430 con 0x7f0cf8101f40 2023-10-17T01:12:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec018680 con 0x7f0cf8101f40 2023-10-17T01:12:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf81a0e90 con 0x7f0cf8101f40 2023-10-17T01:12:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.541+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf81a12b0 con 0x7f0cf8101f40 2023-10-17T01:12:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.542+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf810c5c0 con 0x7f0cf8101f40 2023-10-17T01:12:56.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.544+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0cec0105a0 con 0x7f0cf8101f40 2023-10-17T01:12:56.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.546+0000 7f0cf4ff9700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0ce00a0bf0 0x7f0ce00a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:56.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.546+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0cec014070 con 0x7f0cf8101f40 2023-10-17T01:12:56.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.546+0000 7f0cf77fe700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0ce00a0bf0 0x7f0ce00a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:56.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.547+0000 7f0cf77fe700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0ce00a0bf0 0x7f0ce00a30b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0cf819a210 tx=0x7f0ce8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:56.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.550+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0cec0520b0 con 0x7f0cf8101f40 2023-10-17T01:12:56.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:56.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] up:boot 2023-10-17T01:12:56.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: fsmap 12 up:standby 2023-10-17T01:12:56.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:56 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-17T01:12:56.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:56.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] up:boot 2023-10-17T01:12:56.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: fsmap 12 up:standby 2023-10-17T01:12:56.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:56 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-17T01:12:56.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:12:56.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] up:boot 2023-10-17T01:12:56.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: fsmap 12 up:standby 2023-10-17T01:12:56.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:56 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-17T01:12:56.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.831+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cf81a18f0 con 0x7f0cf8101f40 2023-10-17T01:12:56.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.832+0000 7f0cf4ff9700 1 -- 172.21.15.45:0/2236202052 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9595 (secure 0 0 0) 0x7f0cec095c90 con 0x7f0cf8101f40 2023-10-17T01:12:56.833 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:56.834 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14544,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-10-17T01:12:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.834+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0ce00a0bf0 msgr2=0x7f0ce00a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.834+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0ce00a0bf0 0x7f0ce00a30b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0cf819a210 tx=0x7f0ce8009260 comp rx=0 tx=0).stop 2023-10-17T01:12:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.834+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 msgr2=0x7f0cf8199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.834+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 0x7f0cf8199280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e9a0 tx=0x7f0cec00ecb0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.834+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 shutdown_connections 2023-10-17T01:12:56.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0ce00a0bf0 0x7f0ce00a30b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cf81028e0 0x7f0cf81a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf8100d50 0x7f0cf8198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 --2- 172.21.15.45:0/2236202052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf8101f40 0x7f0cf8199280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:56.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 >> 172.21.15.45:0/2236202052 conn(0x7f0cf80fc280 msgr2=0x7f0cf8107790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:56.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 shutdown_connections 2023-10-17T01:12:56.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:56.835+0000 7f0cfe1ff700 1 -- 172.21.15.45:0/2236202052 wait complete. 2023-10-17T01:12:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2023-10-17T01:12:56.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:12:57.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.071+0000 7f41addff700 1 -- 172.21.15.45:0/2151425311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a81040e0 msgr2=0x7f41a81064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.071+0000 7f41addff700 1 --2- 172.21.15.45:0/2151425311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a81040e0 0x7f41a81064d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f419c0098f0 tx=0x7f419c009c00 comp rx=0 tx=0).stop 2023-10-17T01:12:57.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 -- 172.21.15.45:0/2151425311 shutdown_connections 2023-10-17T01:12:57.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 --2- 172.21.15.45:0/2151425311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41a8106a10 0x7f41a8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 --2- 172.21.15.45:0/2151425311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81017b0 0x7f41a8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 --2- 172.21.15.45:0/2151425311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a81040e0 0x7f41a81064d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 -- 172.21.15.45:0/2151425311 >> 172.21.15.45:0/2151425311 conn(0x7f41a80fb180 msgr2=0x7f41a80fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:57.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 -- 172.21.15.45:0/2151425311 shutdown_connections 2023-10-17T01:12:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.072+0000 7f41addff700 1 -- 172.21.15.45:0/2151425311 wait complete. 2023-10-17T01:12:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.073+0000 7f41addff700 1 Processor -- start 2023-10-17T01:12:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.073+0000 7f41addff700 1 -- start start 2023-10-17T01:12:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41addff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41a81017b0 0x7f41a8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41addff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 0x7f41a8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41addff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a8106a10 0x7f41a81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41addff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81a0830 con 0x7f41a8106a10 2023-10-17T01:12:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41addff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81a09a0 con 0x7f41a81017b0 2023-10-17T01:12:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41addff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81a0b60 con 0x7f41a81040e0 2023-10-17T01:12:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 0x7f41a8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 0x7f41a8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47000/0 (socket says 172.21.15.45:47000) 2023-10-17T01:12:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41a6ffd700 1 -- 172.21.15.45:0/1638686009 learned_addr learned my addr 172.21.15.45:0/1638686009 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41a7fff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a8106a10 0x7f41a81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.074+0000 7f41a77fe700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41a81017b0 0x7f41a8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a6ffd700 1 -- 172.21.15.45:0/1638686009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41a81017b0 msgr2=0x7f41a8198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a6ffd700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41a81017b0 0x7f41a8198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a6ffd700 1 -- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a8106a10 msgr2=0x7f41a81a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a6ffd700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a8106a10 0x7f41a81a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a6ffd700 1 -- 172.21.15.45:0/1638686009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980095f0 con 0x7f41a81040e0 2023-10-17T01:12:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a6ffd700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 0x7f41a8199230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f419c009fd0 tx=0x7f419c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c01c070 con 0x7f41a81040e0 2023-10-17T01:12:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419c00f820 con 0x7f41a81040e0 2023-10-17T01:12:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c009520 con 0x7f41a81040e0 2023-10-17T01:12:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c004b00 con 0x7f41a81040e0 2023-10-17T01:12:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.075+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a81a1070 con 0x7f41a81040e0 2023-10-17T01:12:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.077+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a81085b0 con 0x7f41a81040e0 2023-10-17T01:12:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.078+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f419c020070 con 0x7f41a81040e0 2023-10-17T01:12:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.080+0000 7f41a4ff9700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f41900a0b70 0x7f41900a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.080+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f419c0ccb30 con 0x7f41a81040e0 2023-10-17T01:12:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.080+0000 7f41a77fe700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f41900a0b70 0x7f41900a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.081+0000 7f41a77fe700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f41900a0b70 0x7f41900a3030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41a819a1c0 tx=0x7f4198009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:57.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.084+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f419c096950 con 0x7f41a81040e0 2023-10-17T01:12:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.311+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41a802cc30 con 0x7f41a81040e0 2023-10-17T01:12:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.312+0000 7f41a4ff9700 1 -- 172.21.15.45:0/1638686009 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f419c083e60 con 0x7f41a81040e0 2023-10-17T01:12:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:57.315 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:12:13.667312+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:12:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f41900a0b70 msgr2=0x7f41900a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f41900a0b70 0x7f41900a3030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41a819a1c0 tx=0x7f4198009310 comp rx=0 tx=0).stop 2023-10-17T01:12:57.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 msgr2=0x7f41a8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 0x7f41a8199230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f419c009fd0 tx=0x7f419c00b790 comp rx=0 tx=0).stop 2023-10-17T01:12:57.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 shutdown_connections 2023-10-17T01:12:57.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f41900a0b70 0x7f41900a3030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41a8106a10 0x7f41a81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41a81017b0 0x7f41a8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 --2- 172.21.15.45:0/1638686009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a81040e0 0x7f41a8199230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 >> 172.21.15.45:0/1638686009 conn(0x7f41a80fb180 msgr2=0x7f41a80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:57.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 shutdown_connections 2023-10-17T01:12:57.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.315+0000 7f41addff700 1 -- 172.21.15.45:0/1638686009 wait complete. 2023-10-17T01:12:57.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:12:57.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:12:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 -- 172.21.15.45:0/2934788847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240102940 msgr2=0x7f02401091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 --2- 172.21.15.45:0/2934788847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240102940 0x7f02401091f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f023800b0d0 tx=0x7f023800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 -- 172.21.15.45:0/2934788847 shutdown_connections 2023-10-17T01:12:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 --2- 172.21.15.45:0/2934788847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240102940 0x7f02401091f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 --2- 172.21.15.45:0/2934788847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240100db0 0x7f02401011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 --2- 172.21.15.45:0/2934788847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 0x7f0240102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 -- 172.21.15.45:0/2934788847 >> 172.21.15.45:0/2934788847 conn(0x7f02400fc320 msgr2=0x7f02400fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 -- 172.21.15.45:0/2934788847 shutdown_connections 2023-10-17T01:12:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.559+0000 7f02463ff700 1 -- 172.21.15.45:0/2934788847 wait complete. 2023-10-17T01:12:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 Processor -- start 2023-10-17T01:12:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 -- start start 2023-10-17T01:12:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240100db0 0x7f0240074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 0x7f0240073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 0x7f0240073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0240073e20 con 0x7f0240100db0 2023-10-17T01:12:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0240073f90 con 0x7f0240102940 2023-10-17T01:12:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.560+0000 7f02463ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a0bc0 con 0x7f0240101fa0 2023-10-17T01:12:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f023ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240100db0 0x7f0240074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 0x7f0240073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f023f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 0x7f0240073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 0x7f0240073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38764/0 (socket says 172.21.15.45:38764) 2023-10-17T01:12:57.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 -- 172.21.15.45:0/3943929728 learned_addr learned my addr 172.21.15.45:0/3943929728 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:57.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 -- 172.21.15.45:0/3943929728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 msgr2=0x7f0240073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 0x7f0240073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 -- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240100db0 msgr2=0x7f0240074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240100db0 0x7f0240074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 -- 172.21.15.45:0/3943929728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238009d00 con 0x7f0240102940 2023-10-17T01:12:57.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f023f7fe700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 0x7f0240073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:12:57.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f024499c700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 0x7f0240073680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0238008eb0 tx=0x7f0238008ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:57.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800e040 con 0x7f0240102940 2023-10-17T01:12:57.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.561+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02401a0e40 con 0x7f0240102940 2023-10-17T01:12:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.562+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023800be30 con 0x7f0240102940 2023-10-17T01:12:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.562+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023801d470 con 0x7f0240102940 2023-10-17T01:12:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.562+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02401a1210 con 0x7f0240102940 2023-10-17T01:12:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.563+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02401082f0 con 0x7f0240102940 2023-10-17T01:12:57.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.564+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0238005190 con 0x7f0240102940 2023-10-17T01:12:57.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.566+0000 7f023d7fa700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02280a0b20 0x7f02280a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:57.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.566+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02380cd0e0 con 0x7f0240102940 2023-10-17T01:12:57.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.566+0000 7f023ffff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02280a0b20 0x7f02280a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:57.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.567+0000 7f023ffff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02280a0b20 0x7f02280a2fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0240101e00 tx=0x7f0230009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:57.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.570+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0238096f00 con 0x7f0240102940 2023-10-17T01:12:57.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:57 smithi181 ceph-mon[134520]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:57.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:57 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2236202052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:57.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:57 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1638686009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:57 smithi045 ceph-mon[98474]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:57 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2236202052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:57.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:57 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1638686009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.803+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f024002d080 con 0x7f0240102940 2023-10-17T01:12:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.803+0000 7f023d7fa700 1 -- 172.21.15.45:0/3943929728 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f023809a520 con 0x7f0240102940 2023-10-17T01:12:57.805 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:12:57.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02280a0b20 msgr2=0x7f02280a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02280a0b20 0x7f02280a2fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0240101e00 tx=0x7f0230009260 comp rx=0 tx=0).stop 2023-10-17T01:12:57.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 msgr2=0x7f0240073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:57.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 0x7f0240073680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0238008eb0 tx=0x7f0238008ee0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 shutdown_connections 2023-10-17T01:12:57.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02280a0b20 0x7f02280a2fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0240102940 0x7f0240073680 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0240100db0 0x7f0240074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 --2- 172.21.15.45:0/3943929728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0240101fa0 0x7f0240073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:57.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 >> 172.21.15.45:0/3943929728 conn(0x7f02400fc320 msgr2=0x7f0240105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:57.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 shutdown_connections 2023-10-17T01:12:57.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:57.806+0000 7f02463ff700 1 -- 172.21.15.45:0/3943929728 wait complete. 2023-10-17T01:12:57.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-17T01:12:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:57 smithi060 ceph-mon[134136]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:57.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:57 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2236202052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:12:57.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:57 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1638686009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:12:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.045+0000 7f04e2bff700 1 -- 172.21.15.45:0/4045744828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 msgr2=0x7f04dc073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.045+0000 7f04e2bff700 1 --2- 172.21.15.45:0/4045744828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc073c30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04c80098f0 tx=0x7f04c8009c00 comp rx=0 tx=0).stop 2023-10-17T01:12:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 -- 172.21.15.45:0/4045744828 shutdown_connections 2023-10-17T01:12:58.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 --2- 172.21.15.45:0/4045744828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc108c10 0x7f04dc10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 --2- 172.21.15.45:0/4045744828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 0x7f04dc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 --2- 172.21.15.45:0/4045744828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc073c30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 -- 172.21.15.45:0/4045744828 >> 172.21.15.45:0/4045744828 conn(0x7f04dc0fc280 msgr2=0x7f04dc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:58.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 -- 172.21.15.45:0/4045744828 shutdown_connections 2023-10-17T01:12:58.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.046+0000 7f04e2bff700 1 -- 172.21.15.45:0/4045744828 wait complete. 2023-10-17T01:12:58.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 Processor -- start 2023-10-17T01:12:58.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 -- start start 2023-10-17T01:12:58.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 0x7f04dc199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc108c10 0x7f04dc1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a0830 con 0x7f04dc074d90 2023-10-17T01:12:58.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a09a0 con 0x7f04dc0737b0 2023-10-17T01:12:58.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.047+0000 7f04e2bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a0b60 con 0x7f04dc108c10 2023-10-17T01:12:58.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 0x7f04dc199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 0x7f04dc199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54640/0 (socket says 172.21.15.45:54640) 2023-10-17T01:12:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04dbfff700 1 -- 172.21.15.45:0/2484773406 learned_addr learned my addr 172.21.15.45:0/2484773406 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e119c700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc108c10 0x7f04dc1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 -- 172.21.15.45:0/2484773406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc108c10 msgr2=0x7f04dc1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc108c10 0x7f04dc1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 -- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 msgr2=0x7f04dc199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 0x7f04dc199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 -- 172.21.15.45:0/2484773406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4009c40 con 0x7f04dc0737b0 2023-10-17T01:12:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04e099b700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc198cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9c0 tx=0x7f04d000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.048+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000c9c0 con 0x7f04dc0737b0 2023-10-17T01:12:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.049+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d0010430 con 0x7f04dc0737b0 2023-10-17T01:12:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.049+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c8009520 con 0x7f04dc0737b0 2023-10-17T01:12:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.049+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0018680 con 0x7f04dc0737b0 2023-10-17T01:12:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.049+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc1a11c0 con 0x7f04dc0737b0 2023-10-17T01:12:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.050+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc1072c0 con 0x7f04dc0737b0 2023-10-17T01:12:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.052+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f04d0010860 con 0x7f04dc0737b0 2023-10-17T01:12:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.053+0000 7f04d9ffb700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04cc0a0bf0 0x7f04cc0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.053+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f04d0014070 con 0x7f04dc0737b0 2023-10-17T01:12:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.053+0000 7f04dbfff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04cc0a0bf0 0x7f04cc0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.054+0000 7f04dbfff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04cc0a0bf0 0x7f04cc0a30b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f04dc19a1c0 tx=0x7f04c8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.057+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f04d00520b0 con 0x7f04dc0737b0 2023-10-17T01:12:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.279+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f04dc02cc30 con 0x7f04dc0737b0 2023-10-17T01:12:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.280+0000 7f04d9ffb700 1 -- 172.21.15.45:0/2484773406 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f04d0095ae0 con 0x7f04dc0737b0 2023-10-17T01:12:58.281 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:12:58.282 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-17T01:12:58.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:12:58.282 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-17T01:12:58.282 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-17T01:12:58.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:12:58.283 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-17T01:12:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04cc0a0bf0 msgr2=0x7f04cc0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04cc0a0bf0 0x7f04cc0a30b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f04dc19a1c0 tx=0x7f04c8019040 comp rx=0 tx=0).stop 2023-10-17T01:12:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 msgr2=0x7f04dc198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc198cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9c0 tx=0x7f04d000ecd0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 shutdown_connections 2023-10-17T01:12:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f04cc0a0bf0 0x7f04cc0a30b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04dc108c10 0x7f04dc1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04dc0737b0 0x7f04dc198cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 --2- 172.21.15.45:0/2484773406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04dc074d90 0x7f04dc199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.282+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 >> 172.21.15.45:0/2484773406 conn(0x7f04dc0fc280 msgr2=0x7f04dc0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.283+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 shutdown_connections 2023-10-17T01:12:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.283+0000 7f04e2bff700 1 -- 172.21.15.45:0/2484773406 wait complete. 2023-10-17T01:12:58.304 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-17T01:12:58.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-17T01:12:58.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 -- 172.21.15.45:0/3856616135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 msgr2=0x7f65340731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 --2- 172.21.15.45:0/3856616135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 0x7f65340731e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6524009990 tx=0x7f6524009ca0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 -- 172.21.15.45:0/3856616135 shutdown_connections 2023-10-17T01:12:58.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 --2- 172.21.15.45:0/3856616135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653410d330 0x7f653410f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 --2- 172.21.15.45:0/3856616135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 0x7f65340731e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 --2- 172.21.15.45:0/3856616135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f6534073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.526+0000 7f65397ff700 1 -- 172.21.15.45:0/3856616135 >> 172.21.15.45:0/3856616135 conn(0x7f65340fc2d0 msgr2=0x7f65340fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:58.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.527+0000 7f65397ff700 1 -- 172.21.15.45:0/3856616135 shutdown_connections 2023-10-17T01:12:58.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.527+0000 7f65397ff700 1 -- 172.21.15.45:0/3856616135 wait complete. 2023-10-17T01:12:58.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 Processor -- start 2023-10-17T01:12:58.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 -- start start 2023-10-17T01:12:58.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f653410a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 0x7f653410ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653410d330 0x7f653410be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653410c450 con 0x7f6534074d90 2023-10-17T01:12:58.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653410c5c0 con 0x7f653410d330 2023-10-17T01:12:58.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f65397ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653410c780 con 0x7f65340737b0 2023-10-17T01:12:58.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.528+0000 7f6532ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f653410a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f653410a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46626/0 (socket says 172.21.15.45:46626) 2023-10-17T01:12:58.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f65337fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653410d330 0x7f653410be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 -- 172.21.15.45:0/2677520146 learned_addr learned my addr 172.21.15.45:0/2677520146 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:58.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f65327fc700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 0x7f653410ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 -- 172.21.15.45:0/2677520146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653410d330 msgr2=0x7f653410be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653410d330 0x7f653410be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 -- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 msgr2=0x7f653410ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:58.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 0x7f653410ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:58.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 -- 172.21.15.45:0/2677520146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240095f0 con 0x7f65340737b0 2023-10-17T01:12:58.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f6532ffd700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f653410a900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65240059d0 tx=0x7f6524004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:58.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652401d070 con 0x7f65340737b0 2023-10-17T01:12:58.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f652400fb90 con 0x7f65340737b0 2023-10-17T01:12:58.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653410c9a0 con 0x7f65340737b0 2023-10-17T01:12:58.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.529+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240173f0 con 0x7f65340737b0 2023-10-17T01:12:58.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.530+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653410ce50 con 0x7f65340737b0 2023-10-17T01:12:58.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.531+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6534102e80 con 0x7f65340737b0 2023-10-17T01:12:58.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.533+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f652400fd00 con 0x7f65340737b0 2023-10-17T01:12:58.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.534+0000 7f651bfff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f651c0a0ba0 0x7f651c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:58.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.534+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f65240cd4b0 con 0x7f65340737b0 2023-10-17T01:12:58.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.535+0000 7f65327fc700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f651c0a0ba0 0x7f651c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.535+0000 7f65327fc700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f651c0a0ba0 0x7f651c0a3060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6534074ab0 tx=0x7f6528009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.536+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f65240972d0 con 0x7f65340737b0 2023-10-17T01:12:58.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:58.758+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f653402d060 con 0x7f65340737b0 2023-10-17T01:12:58.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3943929728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:58.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2484773406' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-17T01:12:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:58 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3943929728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2484773406' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-17T01:12:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:58 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3943929728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:12:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2484773406' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-17T01:12:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:58.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:58 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.439+0000 7f651bfff700 1 -- 172.21.15.45:0/2677520146 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f652409a8f0 con 0x7f65340737b0 2023-10-17T01:12:59.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f651c0a0ba0 msgr2=0x7f651c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:59.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f651c0a0ba0 0x7f651c0a3060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6534074ab0 tx=0x7f6528009260 comp rx=0 tx=0).stop 2023-10-17T01:12:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 msgr2=0x7f653410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f653410a900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65240059d0 tx=0x7f6524004730 comp rx=0 tx=0).stop 2023-10-17T01:12:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 shutdown_connections 2023-10-17T01:12:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f651c0a0ba0 0x7f651c0a3060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653410d330 0x7f653410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340737b0 0x7f653410a900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 --2- 172.21.15.45:0/2677520146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534074d90 0x7f653410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.446+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 >> 172.21.15.45:0/2677520146 conn(0x7f65340fc2d0 msgr2=0x7f65340fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.447+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 shutdown_connections 2023-10-17T01:12:59.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.447+0000 7f65397ff700 1 -- 172.21.15.45:0/2677520146 wait complete. 2023-10-17T01:12:59.450 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-17T01:12:59.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-17T01:12:59.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.696+0000 7f2c719ff700 1 -- 172.21.15.45:0/2741671880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c0737b0 msgr2=0x7f2c6c073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:59.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.696+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2741671880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c073c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009990 tx=0x7f2c60009ca0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 -- 172.21.15.45:0/2741671880 shutdown_connections 2023-10-17T01:12:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2741671880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c10d310 0x7f2c6c10f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2741671880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c6c074d90 0x7f2c6c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2741671880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c073c30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 -- 172.21.15.45:0/2741671880 >> 172.21.15.45:0/2741671880 conn(0x7f2c6c0fc290 msgr2=0x7f2c6c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:12:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 -- 172.21.15.45:0/2741671880 shutdown_connections 2023-10-17T01:12:59.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.697+0000 7f2c719ff700 1 -- 172.21.15.45:0/2741671880 wait complete. 2023-10-17T01:12:59.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.698+0000 7f2c719ff700 1 Processor -- start 2023-10-17T01:12:59.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.698+0000 7f2c719ff700 1 -- start start 2023-10-17T01:12:59.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c719ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:59.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c719ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c6c074d90 0x7f2c6c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c719ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 0x7f2c6c1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c719ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1a4bc0 con 0x7f2c6c074d90 2023-10-17T01:12:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c719ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1a4d00 con 0x7f2c6c10d310 2023-10-17T01:12:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c719ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1a4e90 con 0x7f2c6c0737b0 2023-10-17T01:12:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 0x7f2c6c1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c6affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c6a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c6c074d90 0x7f2c6c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 0x7f2c6c1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33070/0 (socket says 172.21.15.45:33070) 2023-10-17T01:12:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.699+0000 7f2c6b7fe700 1 -- 172.21.15.45:0/2090889957 learned_addr learned my addr 172.21.15.45:0/2090889957 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:12:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6b7fe700 1 -- 172.21.15.45:0/2090889957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c0737b0 msgr2=0x7f2c6c19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6b7fe700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c19d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6b7fe700 1 -- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c6c074d90 msgr2=0x7f2c6c19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:12:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6b7fe700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c6c074d90 0x7f2c6c19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:12:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6b7fe700 1 -- 172.21.15.45:0/2090889957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600095f0 con 0x7f2c6c10d310 2023-10-17T01:12:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6affd700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:12:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c6b7fe700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 0x7f2c6c1a45f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400ef30 tx=0x7f2c6400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64018070 con 0x7f2c6c10d310 2023-10-17T01:12:59.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c6400f040 con 0x7f2c6c10d310 2023-10-17T01:12:59.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c1a5110 con 0x7f2c6c10d310 2023-10-17T01:12:59.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400cd00 con 0x7f2c6c10d310 2023-10-17T01:12:59.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.700+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c1a5670 con 0x7f2c6c10d310 2023-10-17T01:12:59.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.702+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c102e60 con 0x7f2c6c10d310 2023-10-17T01:12:59.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.704+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2c64003a00 con 0x7f2c6c10d310 2023-10-17T01:12:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.706+0000 7f2c709fd700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c540a0ba0 0x7f2c540a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:12:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.706+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f2c640cb9b0 con 0x7f2c6c10d310 2023-10-17T01:12:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.706+0000 7f2c6affd700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c540a0ba0 0x7f2c540a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:12:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.707+0000 7f2c6affd700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c540a0ba0 0x7f2c540a3060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011490 tx=0x7f2c5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:12:59.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.709+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2c64095660 con 0x7f2c6c10d310 2023-10-17T01:12:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2677520146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-17T01:12:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-17T01:12:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:59.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:12:59 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2677520146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-17T01:12:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-17T01:12:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:59.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:59.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:12:59 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:12:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2677520146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-17T01:12:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-17T01:12:59.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:12:59.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:12:59.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:12:59 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:12:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:12:59.927+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2c6c02d040 con 0x7f2c6c10d310 2023-10-17T01:13:00.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.474+0000 7f2c709fd700 1 -- 172.21.15.45:0/2090889957 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2c64098c80 con 0x7f2c6c10d310 2023-10-17T01:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.483+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c540a0ba0 msgr2=0x7f2c540a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.483+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c540a0ba0 0x7f2c540a3060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011490 tx=0x7f2c5c009260 comp rx=0 tx=0).stop 2023-10-17T01:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.483+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 msgr2=0x7f2c6c1a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.483+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 0x7f2c6c1a45f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400ef30 tx=0x7f2c6400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.484+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 shutdown_connections 2023-10-17T01:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.484+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c540a0ba0 0x7f2c540a3060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.484+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c6c10d310 0x7f2c6c1a45f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.484+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c6c0737b0 0x7f2c6c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.484+0000 7f2c719ff700 1 --2- 172.21.15.45:0/2090889957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c6c074d90 0x7f2c6c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.484+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 >> 172.21.15.45:0/2090889957 conn(0x7f2c6c0fc290 msgr2=0x7f2c6c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.485+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 shutdown_connections 2023-10-17T01:13:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.485+0000 7f2c719ff700 1 -- 172.21.15.45:0/2090889957 wait complete. 2023-10-17T01:13:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-17T01:13:00.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-17T01:13:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.734+0000 7f911d3ff700 1 -- 172.21.15.45:0/155301486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91181028f0 msgr2=0x7f91181091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.734+0000 7f911d3ff700 1 --2- 172.21.15.45:0/155301486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91181028f0 0x7f91181091a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f911000b0d0 tx=0x7f911000b3e0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.734+0000 7f911d3ff700 1 -- 172.21.15.45:0/155301486 shutdown_connections 2023-10-17T01:13:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.734+0000 7f911d3ff700 1 --2- 172.21.15.45:0/155301486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91181028f0 0x7f91181091a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.735+0000 7f911d3ff700 1 --2- 172.21.15.45:0/155301486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9118100d60 0x7f9118101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.735+0000 7f911d3ff700 1 --2- 172.21.15.45:0/155301486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9118101f50 0x7f91181023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.735+0000 7f911d3ff700 1 -- 172.21.15.45:0/155301486 >> 172.21.15.45:0/155301486 conn(0x7f91180fc2d0 msgr2=0x7f91180fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.735+0000 7f911d3ff700 1 -- 172.21.15.45:0/155301486 shutdown_connections 2023-10-17T01:13:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.735+0000 7f911d3ff700 1 -- 172.21.15.45:0/155301486 wait complete. 2023-10-17T01:13:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 Processor -- start 2023-10-17T01:13:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 -- start start 2023-10-17T01:13:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9118100d60 0x7f9118198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 0x7f9118199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91181028f0 0x7f91181a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181a0950 con 0x7f91181028f0 2023-10-17T01:13:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181a0a90 con 0x7f9118101f50 2023-10-17T01:13:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f911d3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181a0c20 con 0x7f9118100d60 2023-10-17T01:13:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.736+0000 7f91167fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 0x7f9118199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 0x7f9118199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33090/0 (socket says 172.21.15.45:33090) 2023-10-17T01:13:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 -- 172.21.15.45:0/3839535272 learned_addr learned my addr 172.21.15.45:0/3839535272 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91177fe700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91181028f0 0x7f91181a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f9116ffd700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9118100d60 0x7f9118198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 -- 172.21.15.45:0/3839535272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9118100d60 msgr2=0x7f9118198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9118100d60 0x7f9118198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 -- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91181028f0 msgr2=0x7f91181a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91181028f0 0x7f91181a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 -- 172.21.15.45:0/3839535272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009d00 con 0x7f9118101f50 2023-10-17T01:13:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f91167fc700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 0x7f9118199310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f910c00e9a0 tx=0x7f910c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c009790 con 0x7f9118101f50 2023-10-17T01:13:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f910c010430 con 0x7f9118101f50 2023-10-17T01:13:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91181a0f00 con 0x7f9118101f50 2023-10-17T01:13:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.737+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c018680 con 0x7f9118101f50 2023-10-17T01:13:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.738+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91181a13d0 con 0x7f9118101f50 2023-10-17T01:13:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.739+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911810c5d0 con 0x7f9118101f50 2023-10-17T01:13:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.740+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f910c0105a0 con 0x7f9118101f50 2023-10-17T01:13:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.742+0000 7f90fffff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f91000a0ba0 0x7f91000a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.742+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f910c014070 con 0x7f9118101f50 2023-10-17T01:13:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.742+0000 7f9116ffd700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f91000a0ba0 0x7f91000a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.743+0000 7f9116ffd700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f91000a0ba0 0x7f91000a3060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9118101db0 tx=0x7f9108009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:00.746+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f910c0520b0 con 0x7f9118101f50 2023-10-17T01:13:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:00 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-17T01:13:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:00 smithi045 ceph-mon[98474]: osdmap e80: 12 total, 12 up, 12 in 2023-10-17T01:13:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:00 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:13:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:00 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-17T01:13:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:00 smithi060 ceph-mon[134136]: osdmap e80: 12 total, 12 up, 12 in 2023-10-17T01:13:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:00 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:13:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:00 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-17T01:13:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:00 smithi181 ceph-mon[134520]: osdmap e80: 12 total, 12 up, 12 in 2023-10-17T01:13:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:00 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:13:01.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.035+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91181a1bb0 con 0x7f9118101f50 2023-10-17T01:13:01.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.048+0000 7f90fffff700 1 -- 172.21.15.45:0/3839535272 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f910c099660 con 0x7f9118101f50 2023-10-17T01:13:01.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.054+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f91000a0ba0 msgr2=0x7f91000a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.054+0000 7f911d3ff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f91000a0ba0 0x7f91000a3060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9118101db0 tx=0x7f9108009260 comp rx=0 tx=0).stop 2023-10-17T01:13:01.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.054+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 msgr2=0x7f9118199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.054+0000 7f911d3ff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 0x7f9118199310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f910c00e9a0 tx=0x7f910c00ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 shutdown_connections 2023-10-17T01:13:01.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f91000a0ba0 0x7f91000a3060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91181028f0 0x7f91181a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9118100d60 0x7f9118198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 --2- 172.21.15.45:0/3839535272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9118101f50 0x7f9118199310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 >> 172.21.15.45:0/3839535272 conn(0x7f91180fc2d0 msgr2=0x7f91181077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:01.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 shutdown_connections 2023-10-17T01:13:01.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.055+0000 7f911d3ff700 1 -- 172.21.15.45:0/3839535272 wait complete. 2023-10-17T01:13:01.066 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2023-10-17T01:13:01.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:13:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 -- 172.21.15.45:0/1755643720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4106a70 msgr2=0x7f04b4108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 --2- 172.21.15.45:0/1755643720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4106a70 0x7f04b4108e80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b0a0 tx=0x7f04ac00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 -- 172.21.15.45:0/1755643720 shutdown_connections 2023-10-17T01:13:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 --2- 172.21.15.45:0/1755643720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4106a70 0x7f04b4108e80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 --2- 172.21.15.45:0/1755643720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b4101810 0x7f04b4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 --2- 172.21.15.45:0/1755643720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4104140 0x7f04b4106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 -- 172.21.15.45:0/1755643720 >> 172.21.15.45:0/1755643720 conn(0x7f04b40fb180 msgr2=0x7f04b40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 -- 172.21.15.45:0/1755643720 shutdown_connections 2023-10-17T01:13:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.307+0000 7f04b9bff700 1 -- 172.21.15.45:0/1755643720 wait complete. 2023-10-17T01:13:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.308+0000 7f04b9bff700 1 Processor -- start 2023-10-17T01:13:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.308+0000 7f04b9bff700 1 -- start start 2023-10-17T01:13:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b9bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 0x7f04b4101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b9bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 0x7f04b40ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b9bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b4106a70 0x7f04b40ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b9bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b41004a0 con 0x7f04b4104140 2023-10-17T01:13:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b9bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b4100610 con 0x7f04b4101810 2023-10-17T01:13:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b9bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b41a0b40 con 0x7f04b4106a70 2023-10-17T01:13:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 0x7f04b40ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 0x7f04b4101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 0x7f04b4101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33104/0 (socket says 172.21.15.45:33104) 2023-10-17T01:13:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 0x7f04b40ff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54688/0 (socket says 172.21.15.45:54688) 2023-10-17T01:13:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 -- 172.21.15.45:0/3983696598 learned_addr learned my addr 172.21.15.45:0/3983696598 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b3fff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b4106a70 0x7f04b40ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 -- 172.21.15.45:0/3983696598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b4106a70 msgr2=0x7f04b40ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b4106a70 0x7f04b40ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 -- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 msgr2=0x7f04b40ff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 0x7f04b40ff7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b37fe700 1 -- 172.21.15.45:0/3983696598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac009d00 con 0x7f04b4101810 2023-10-17T01:13:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.309+0000 7f04b2ffd700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 0x7f04b40ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.310+0000 7f04b37fe700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 0x7f04b4101180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04a400e9a0 tx=0x7f04a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:01.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.310+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4009790 con 0x7f04b4101810 2023-10-17T01:13:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.310+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a4010430 con 0x7f04b4101810 2023-10-17T01:13:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.310+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4018680 con 0x7f04b4101810 2023-10-17T01:13:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.310+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b41a0e20 con 0x7f04b4101810 2023-10-17T01:13:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.310+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b41a11f0 con 0x7f04b4101810 2023-10-17T01:13:01.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.311+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b40fcd00 con 0x7f04b4101810 2023-10-17T01:13:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.313+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f04a40105a0 con 0x7f04b4101810 2023-10-17T01:13:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.314+0000 7f04b0ff9700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f049c0a0bf0 0x7f049c0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.314+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f04a4014070 con 0x7f04b4101810 2023-10-17T01:13:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.315+0000 7f04b2ffd700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f049c0a0bf0 0x7f049c0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.315+0000 7f04b2ffd700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f049c0a0bf0 0x7f049c0a30b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04a8011490 tx=0x7f04a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:01.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.318+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f04a40520b0 con 0x7f04b4101810 2023-10-17T01:13:01.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.548+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04b410c560 con 0x7f04b4101810 2023-10-17T01:13:01.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.550+0000 7f04b0ff9700 1 -- 172.21.15.45:0/3983696598 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22072 (secure 0 0 0) 0x7f04a4095c60 con 0x7f04b4101810 2023-10-17T01:13:01.551 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:01.552 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:01.037452+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:13:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f049c0a0bf0 msgr2=0x7f049c0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f049c0a0bf0 0x7f049c0a30b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04a8011490 tx=0x7f04a8009260 comp rx=0 tx=0).stop 2023-10-17T01:13:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 msgr2=0x7f04b4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 0x7f04b4101180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04a400e9a0 tx=0x7f04a400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 shutdown_connections 2023-10-17T01:13:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f049c0a0bf0 0x7f049c0a30b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b4106a70 0x7f04b40ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04b4101810 0x7f04b4101180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 --2- 172.21.15.45:0/3983696598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04b4104140 0x7f04b40ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 >> 172.21.15.45:0/3983696598 conn(0x7f04b40fb180 msgr2=0x7f04b4102480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 shutdown_connections 2023-10-17T01:13:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.552+0000 7f04b9bff700 1 -- 172.21.15.45:0/3983696598 wait complete. 2023-10-17T01:13:01.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:13:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.791+0000 7f971c7ff700 1 -- 172.21.15.45:0/294077688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714100d50 msgr2=0x7f9714101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.791+0000 7f971c7ff700 1 --2- 172.21.15.45:0/294077688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714100d50 0x7f9714101170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9710009990 tx=0x7f9710009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 -- 172.21.15.45:0/294077688 shutdown_connections 2023-10-17T01:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 --2- 172.21.15.45:0/294077688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 0x7f9714109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 --2- 172.21.15.45:0/294077688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714100d50 0x7f9714101170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 --2- 172.21.15.45:0/294077688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9714101f40 0x7f97141023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 -- 172.21.15.45:0/294077688 >> 172.21.15.45:0/294077688 conn(0x7f97140fc280 msgr2=0x7f97140fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 -- 172.21.15.45:0/294077688 shutdown_connections 2023-10-17T01:13:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.792+0000 7f971c7ff700 1 -- 172.21.15.45:0/294077688 wait complete. 2023-10-17T01:13:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 Processor -- start 2023-10-17T01:13:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 -- start start 2023-10-17T01:13:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9714100d50 0x7f9714198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 0x7f97141991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 0x7f97141a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97141a07e0 con 0x7f97141028e0 2023-10-17T01:13:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97141a0950 con 0x7f9714100d50 2023-10-17T01:13:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971c7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97141a0b10 con 0x7f9714101f40 2023-10-17T01:13:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f971ad9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 0x7f97141a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.793+0000 7f9719d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 0x7f97141991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f971a59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9714100d50 0x7f9714198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 0x7f97141991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46694/0 (socket says 172.21.15.45:46694) 2023-10-17T01:13:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 -- 172.21.15.45:0/1980320902 learned_addr learned my addr 172.21.15.45:0/1980320902 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 -- 172.21.15.45:0/1980320902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9714100d50 msgr2=0x7f9714198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9714100d50 0x7f9714198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 -- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 msgr2=0x7f97141a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:01.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 0x7f97141a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:01.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 -- 172.21.15.45:0/1980320902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9704009520 con 0x7f9714101f40 2023-10-17T01:13:01.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f971ad9c700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 0x7f97141a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:01.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f9719d9a700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 0x7f97141991b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f970400eb30 tx=0x7f970400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:01.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704009880 con 0x7f9714101f40 2023-10-17T01:13:01.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.794+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c009c40 con 0x7f9714101f40 2023-10-17T01:13:01.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.795+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97040044e0 con 0x7f9714101f40 2023-10-17T01:13:01.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.795+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704003d80 con 0x7f9714101f40 2023-10-17T01:13:01.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.795+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97100095f0 con 0x7f9714101f40 2023-10-17T01:13:01.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.796+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971410c5b0 con 0x7f9714101f40 2023-10-17T01:13:01.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.798+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9704003ee0 con 0x7f9714101f40 2023-10-17T01:13:01.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.799+0000 7f970b7fe700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97000a0b70 0x7f97000a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:01.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.799+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9704014070 con 0x7f9714101f40 2023-10-17T01:13:01.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.800+0000 7f971a59b700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97000a0b70 0x7f97000a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:01.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2090889957' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-17T01:13:01.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-17T01:13:01.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: pgmap v264: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:13:01.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-17T01:13:01.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: osdmap e81: 12 total, 12 up, 12 in 2023-10-17T01:13:01.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3839535272' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-17T01:13:01.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-17T01:13:01.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-17T01:13:01.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: osdmap e82: 12 total, 12 up, 12 in 2023-10-17T01:13:01.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-17T01:13:01.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-17T01:13:01.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:01 smithi045 ceph-mon[98474]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-17T01:13:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.801+0000 7f971a59b700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97000a0b70 0x7f97000a3030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c00 tx=0x7f9710005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:01.803+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f9704095f10 con 0x7f9714101f40 2023-10-17T01:13:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2090889957' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-17T01:13:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-17T01:13:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: pgmap v264: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:13:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-17T01:13:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: osdmap e81: 12 total, 12 up, 12 in 2023-10-17T01:13:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3839535272' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-17T01:13:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-17T01:13:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-17T01:13:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: osdmap e82: 12 total, 12 up, 12 in 2023-10-17T01:13:01.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-17T01:13:01.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-17T01:13:01.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:01 smithi060 ceph-mon[134136]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-17T01:13:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2090889957' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-17T01:13:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-17T01:13:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: pgmap v264: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:13:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-17T01:13:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: osdmap e81: 12 total, 12 up, 12 in 2023-10-17T01:13:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3839535272' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-17T01:13:01.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-17T01:13:01.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-17T01:13:01.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: osdmap e82: 12 total, 12 up, 12 in 2023-10-17T01:13:01.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-17T01:13:01.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-17T01:13:01.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:01 smithi181 ceph-mon[134520]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-17T01:13:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.050+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97141a1640 con 0x7f9714101f40 2023-10-17T01:13:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.052+0000 7f970b7fe700 1 -- 172.21.15.45:0/1980320902 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7f9704004190 con 0x7f9714101f40 2023-10-17T01:13:02.053 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:02.054 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:02.040707+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:13:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97000a0b70 msgr2=0x7f97000a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97000a0b70 0x7f97000a3030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c00 tx=0x7f9710005fb0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 msgr2=0x7f97141991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 0x7f97141991b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f970400eb30 tx=0x7f970400ee40 comp rx=0 tx=0).stop 2023-10-17T01:13:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 shutdown_connections 2023-10-17T01:13:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97000a0b70 0x7f97000a3030 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97141028e0 0x7f97141a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9714100d50 0x7f9714198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 --2- 172.21.15.45:0/1980320902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9714101f40 0x7f97141991b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 >> 172.21.15.45:0/1980320902 conn(0x7f97140fc280 msgr2=0x7f97140fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.054+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 shutdown_connections 2023-10-17T01:13:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.055+0000 7f971c7ff700 1 -- 172.21.15.45:0/1980320902 wait complete. 2023-10-17T01:13:02.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-17T01:13:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 -- 172.21.15.45:0/330377901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c100d40 msgr2=0x7f440c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 --2- 172.21.15.45:0/330377901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c100d40 0x7f440c101160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0099c0 tx=0x7f43fc009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 -- 172.21.15.45:0/330377901 shutdown_connections 2023-10-17T01:13:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 --2- 172.21.15.45:0/330377901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c1028d0 0x7f440c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 --2- 172.21.15.45:0/330377901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c100d40 0x7f440c101160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 --2- 172.21.15.45:0/330377901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440c101f30 0x7f440c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 -- 172.21.15.45:0/330377901 >> 172.21.15.45:0/330377901 conn(0x7f440c0fc290 msgr2=0x7f440c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 -- 172.21.15.45:0/330377901 shutdown_connections 2023-10-17T01:13:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.299+0000 7f44127ff700 1 -- 172.21.15.45:0/330377901 wait complete. 2023-10-17T01:13:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.300+0000 7f44127ff700 1 Processor -- start 2023-10-17T01:13:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 -- start start 2023-10-17T01:13:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440c100d40 0x7f440c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 0x7f440c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c1028d0 0x7f440c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1a0830 con 0x7f440c101f30 2023-10-17T01:13:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1a09a0 con 0x7f440c1028d0 2023-10-17T01:13:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f44127ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1a0b60 con 0x7f440c100d40 2023-10-17T01:13:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 0x7f440c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f440b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 0x7f440c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54722/0 (socket says 172.21.15.45:54722) 2023-10-17T01:13:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f440b7fe700 1 -- 172.21.15.45:0/1091683209 learned_addr learned my addr 172.21.15.45:0/1091683209 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f440bfff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440c100d40 0x7f440c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.301+0000 7f4410d9c700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c1028d0 0x7f440c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f440b7fe700 1 -- 172.21.15.45:0/1091683209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440c100d40 msgr2=0x7f440c198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f440b7fe700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440c100d40 0x7f440c198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f440b7fe700 1 -- 172.21.15.45:0/1091683209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c1028d0 msgr2=0x7f440c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f440b7fe700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c1028d0 0x7f440c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f440b7fe700 1 -- 172.21.15.45:0/1091683209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0095f0 con 0x7f440c101f30 2023-10-17T01:13:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f440b7fe700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 0x7f440c199200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f440000e9a0 tx=0x7f440000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400009790 con 0x7f440c101f30 2023-10-17T01:13:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4400010430 con 0x7f440c101f30 2023-10-17T01:13:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400018680 con 0x7f440c101f30 2023-10-17T01:13:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.302+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c1a0e40 con 0x7f440c101f30 2023-10-17T01:13:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.303+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c1a1210 con 0x7f440c101f30 2023-10-17T01:13:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.304+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c193390 con 0x7f440c101f30 2023-10-17T01:13:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.306+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f44000105a0 con 0x7f440c101f30 2023-10-17T01:13:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.307+0000 7f44097fa700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43f40a0ba0 0x7f43f40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.307+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4400014070 con 0x7f440c101f30 2023-10-17T01:13:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.308+0000 7f440bfff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43f40a0ba0 0x7f43f40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.308+0000 7f440bfff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43f40a0ba0 0x7f43f40a3060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009990 tx=0x7f43fc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.311+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f4400095e90 con 0x7f440c101f30 2023-10-17T01:13:02.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:02.580+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f440c02ce80 con 0x7f440c101f30 2023-10-17T01:13:02.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3983696598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:02.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] up:active 2023-10-17T01:13:02.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:02.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: osdmap e83: 12 total, 12 up, 12 in 2023-10-17T01:13:02.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:02.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1980320902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:02.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:02 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3983696598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] up:active 2023-10-17T01:13:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:02.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: osdmap e83: 12 total, 12 up, 12 in 2023-10-17T01:13:02.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:02.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1980320902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:02.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:02 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3983696598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] up:active 2023-10-17T01:13:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:02.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: osdmap e83: 12 total, 12 up, 12 in 2023-10-17T01:13:02.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:02.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1980320902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:02.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:02.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:02 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.472+0000 7f44097fa700 1 -- 172.21.15.45:0/1091683209 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4400026020 con 0x7f440c101f30 2023-10-17T01:13:03.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43f40a0ba0 msgr2=0x7f43f40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:03.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43f40a0ba0 0x7f43f40a3060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009990 tx=0x7f43fc01a040 comp rx=0 tx=0).stop 2023-10-17T01:13:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 msgr2=0x7f440c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 0x7f440c199200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f440000e9a0 tx=0x7f440000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 shutdown_connections 2023-10-17T01:13:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43f40a0ba0 0x7f43f40a3060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c1028d0 0x7f440c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f440c100d40 0x7f440c198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 --2- 172.21.15.45:0/1091683209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c101f30 0x7f440c199200 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 >> 172.21.15.45:0/1091683209 conn(0x7f440c0fc290 msgr2=0x7f440c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 shutdown_connections 2023-10-17T01:13:03.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.475+0000 7f44127ff700 1 -- 172.21.15.45:0/1091683209 wait complete. 2023-10-17T01:13:03.496 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-10-17T01:13:03.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-17T01:13:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 -- 172.21.15.45:0/2622500219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4101f30 msgr2=0x7f3cf4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/2622500219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4101f30 0x7f3cf4102390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0099c0 tx=0x7f3cdc009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 -- 172.21.15.45:0/2622500219 shutdown_connections 2023-10-17T01:13:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/2622500219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf41028d0 0x7f3cf4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/2622500219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4100d40 0x7f3cf4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/2622500219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4101f30 0x7f3cf4102390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 -- 172.21.15.45:0/2622500219 >> 172.21.15.45:0/2622500219 conn(0x7f3cf40fc290 msgr2=0x7f3cf40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:03.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 -- 172.21.15.45:0/2622500219 shutdown_connections 2023-10-17T01:13:03.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.705+0000 7f3cf95ff700 1 -- 172.21.15.45:0/2622500219 wait complete. 2023-10-17T01:13:03.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 Processor -- start 2023-10-17T01:13:03.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 -- start start 2023-10-17T01:13:03.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 0x7f3cf4074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4101f30 0x7f3cf4073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf41028d0 0x7f3cf4073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4073e20 con 0x7f3cf4100d40 2023-10-17T01:13:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4073f90 con 0x7f3cf41028d0 2023-10-17T01:13:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.706+0000 7f3cf95ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41a0af0 con 0x7f3cf4101f30 2023-10-17T01:13:03.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 0x7f3cf4074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:03.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 0x7f3cf4074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54732/0 (socket says 172.21.15.45:54732) 2023-10-17T01:13:03.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 -- 172.21.15.45:0/1861388003 learned_addr learned my addr 172.21.15.45:0/1861388003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:03.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf37fe700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf41028d0 0x7f3cf4073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:03.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cebfff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4101f30 0x7f3cf4073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:03.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 -- 172.21.15.45:0/1861388003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4101f30 msgr2=0x7f3cf4073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:03.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4101f30 0x7f3cf4073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 -- 172.21.15.45:0/1861388003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf41028d0 msgr2=0x7f3cf4073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:03.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf41028d0 0x7f3cf4073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:03.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 -- 172.21.15.45:0/1861388003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0095f0 con 0x7f3cf4100d40 2023-10-17T01:13:03.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cebfff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4101f30 0x7f3cf4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:03.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.707+0000 7f3cf2ffd700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 0x7f3cf4074b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400e970 tx=0x7f3ce400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.708+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4009790 con 0x7f3cf4100d40 2023-10-17T01:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.708+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf41a0dd0 con 0x7f3cf4100d40 2023-10-17T01:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.708+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce400eec0 con 0x7f3cf4100d40 2023-10-17T01:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.708+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4003cd0 con 0x7f3cf4100d40 2023-10-17T01:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.708+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf41a1200 con 0x7f3cf4100d40 2023-10-17T01:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.709+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf410c610 con 0x7f3cf4100d40 2023-10-17T01:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.711+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f3ce400ccd0 con 0x7f3cf4100d40 2023-10-17T01:13:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.712+0000 7f3cf0ff9700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ce00a0b20 0x7f3ce00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.712+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3ce4014070 con 0x7f3cf4100d40 2023-10-17T01:13:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.712+0000 7f3cebfff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ce00a0b20 0x7f3ce00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.713+0000 7f3cebfff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ce00a0b20 0x7f3ce00a2fe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00b580 tx=0x7f3cdc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:03.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.716+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3ce4095d30 con 0x7f3cf4100d40 2023-10-17T01:13:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: pgmap v268: 97 pgs: 9 creating+peering, 4 creating+activating, 32 active+clean, 52 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-10-17T01:13:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1091683209' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-17T01:13:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:13:03.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:13:03.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:03 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: pgmap v268: 97 pgs: 9 creating+peering, 4 creating+activating, 32 active+clean, 52 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-10-17T01:13:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1091683209' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-17T01:13:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:13:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:13:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:03 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: pgmap v268: 97 pgs: 9 creating+peering, 4 creating+activating, 32 active+clean, 52 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-10-17T01:13:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1091683209' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-17T01:13:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:13:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:13:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:03 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:13:03.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:03.976+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f3cf41a1af0 con 0x7f3cf4100d40 2023-10-17T01:13:04.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.413+0000 7f3cf0ff9700 1 -- 172.21.15.45:0/1861388003 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3ce4099350 con 0x7f3cf4100d40 2023-10-17T01:13:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ce00a0b20 msgr2=0x7f3ce00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ce00a0b20 0x7f3ce00a2fe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00b580 tx=0x7f3cdc01a040 comp rx=0 tx=0).stop 2023-10-17T01:13:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 msgr2=0x7f3cf4074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 0x7f3cf4074b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400e970 tx=0x7f3ce400ec80 comp rx=0 tx=0).stop 2023-10-17T01:13:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 shutdown_connections 2023-10-17T01:13:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ce00a0b20 0x7f3ce00a2fe0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf41028d0 0x7f3cf4073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4100d40 0x7f3cf4074b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 --2- 172.21.15.45:0/1861388003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cf4101f30 0x7f3cf4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.416+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 >> 172.21.15.45:0/1861388003 conn(0x7f3cf40fc290 msgr2=0x7f3cf40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.417+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 shutdown_connections 2023-10-17T01:13:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.417+0000 7f3cf95ff700 1 -- 172.21.15.45:0/1861388003 wait complete. 2023-10-17T01:13:04.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-17T01:13:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 -- 172.21.15.45:0/4077853357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb410074d90 msgr2=0x7fb4100731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 --2- 172.21.15.45:0/4077853357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb410074d90 0x7fb4100731e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb400009990 tx=0x7fb400009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 -- 172.21.15.45:0/4077853357 shutdown_connections 2023-10-17T01:13:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 --2- 172.21.15.45:0/4077853357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb410108c00 0x7fb41010b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 --2- 172.21.15.45:0/4077853357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb410074d90 0x7fb4100731e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 --2- 172.21.15.45:0/4077853357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4100737b0 0x7fb410073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 -- 172.21.15.45:0/4077853357 >> 172.21.15.45:0/4077853357 conn(0x7fb4100fc290 msgr2=0x7fb4100fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 -- 172.21.15.45:0/4077853357 shutdown_connections 2023-10-17T01:13:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.651+0000 7fb4163ff700 1 -- 172.21.15.45:0/4077853357 wait complete. 2023-10-17T01:13:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.653+0000 7fb4163ff700 1 Processor -- start 2023-10-17T01:13:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.653+0000 7fb4163ff700 1 -- start start 2023-10-17T01:13:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.653+0000 7fb4163ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 0x7fb410198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.653+0000 7fb4163ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb410074d90 0x7fb410199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb4163ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb410108c00 0x7fb4101a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb4163ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101a0840 con 0x7fb4100737b0 2023-10-17T01:13:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb4163ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101a09b0 con 0x7fb410074d90 2023-10-17T01:13:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb4163ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101a0b70 con 0x7fb410108c00 2023-10-17T01:13:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 0x7fb410198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 0x7fb410198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54762/0 (socket says 172.21.15.45:54762) 2023-10-17T01:13:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 -- 172.21.15.45:0/3463502413 learned_addr learned my addr 172.21.15.45:0/3463502413 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40f7fe700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb410074d90 0x7fb410199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb41499c700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb410108c00 0x7fb4101a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 -- 172.21.15.45:0/3463502413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb410108c00 msgr2=0x7fb4101a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb410108c00 0x7fb4101a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 -- 172.21.15.45:0/3463502413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb410074d90 msgr2=0x7fb410199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb410074d90 0x7fb410199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 -- 172.21.15.45:0/3463502413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000095f0 con 0x7fb4100737b0 2023-10-17T01:13:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.654+0000 7fb40ffff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 0x7fb410198d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb400004730 tx=0x7fb400004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.655+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40001d070 con 0x7fb4100737b0 2023-10-17T01:13:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.655+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4101a0df0 con 0x7fb4100737b0 2023-10-17T01:13:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.655+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb400004c90 con 0x7fb4100737b0 2023-10-17T01:13:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.655+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400005570 con 0x7fb4100737b0 2023-10-17T01:13:04.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.655+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4101a11c0 con 0x7fb4100737b0 2023-10-17T01:13:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.656+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4101072b0 con 0x7fb4100737b0 2023-10-17T01:13:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.658+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb40000fc70 con 0x7fb4100737b0 2023-10-17T01:13:04.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.659+0000 7fb40d7fa700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3f80a0b20 0x7fb3f80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.659+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4000cd490 con 0x7fb4100737b0 2023-10-17T01:13:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.659+0000 7fb40f7fe700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3f80a0b20 0x7fb3f80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:04.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.660+0000 7fb40f7fe700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3f80a0b20 0x7fb3f80a2fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb410074ab0 tx=0x7fb404009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.663+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb40005b0b0 con 0x7fb4100737b0 2023-10-17T01:13:04.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1091683209' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-17T01:13:04.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:04.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1861388003' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-17T01:13:04.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-17T01:13:04.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-17T01:13:04.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1861388003' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-17T01:13:04.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-17T01:13:04.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:04 smithi045 ceph-mon[98474]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-17T01:13:04.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1091683209' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-17T01:13:04.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1861388003' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-17T01:13:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-17T01:13:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-17T01:13:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1861388003' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-17T01:13:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-17T01:13:04.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:04 smithi060 ceph-mon[134136]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-17T01:13:04.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1091683209' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-17T01:13:04.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-17T01:13:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1861388003' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-17T01:13:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-17T01:13:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-17T01:13:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1861388003' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-17T01:13:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-17T01:13:04.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:04 smithi181 ceph-mon[134520]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-17T01:13:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:04.922+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb410061a00 con 0x7fb4100737b0 2023-10-17T01:13:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.423+0000 7fb40d7fa700 1 -- 172.21.15.45:0/3463502413 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb400096f60 con 0x7fb4100737b0 2023-10-17T01:13:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3f80a0b20 msgr2=0x7fb3f80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3f80a0b20 0x7fb3f80a2fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb410074ab0 tx=0x7fb404009260 comp rx=0 tx=0).stop 2023-10-17T01:13:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 msgr2=0x7fb410198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 0x7fb410198d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb400004730 tx=0x7fb400004760 comp rx=0 tx=0).stop 2023-10-17T01:13:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 shutdown_connections 2023-10-17T01:13:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3f80a0b20 0x7fb3f80a2fe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb410108c00 0x7fb4101a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4100737b0 0x7fb410198d00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 --2- 172.21.15.45:0/3463502413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb410074d90 0x7fb410199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.427+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 >> 172.21.15.45:0/3463502413 conn(0x7fb4100fc290 msgr2=0x7fb4100fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.428+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 shutdown_connections 2023-10-17T01:13:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.428+0000 7fb4163ff700 1 -- 172.21.15.45:0/3463502413 wait complete. 2023-10-17T01:13:05.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-17T01:13:05.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 -- 172.21.15.45:0/1570194672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 msgr2=0x7f52b0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 --2- 172.21.15.45:0/1570194672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0102390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52a40099c0 tx=0x7f52a4009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 -- 172.21.15.45:0/1570194672 shutdown_connections 2023-10-17T01:13:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 --2- 172.21.15.45:0/1570194672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52b01028d0 0x7f52b0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 --2- 172.21.15.45:0/1570194672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b0100d40 0x7f52b0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 --2- 172.21.15.45:0/1570194672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0102390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 -- 172.21.15.45:0/1570194672 >> 172.21.15.45:0/1570194672 conn(0x7f52b00fc290 msgr2=0x7f52b00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 -- 172.21.15.45:0/1570194672 shutdown_connections 2023-10-17T01:13:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.669+0000 7f52b63ff700 1 -- 172.21.15.45:0/1570194672 wait complete. 2023-10-17T01:13:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.671+0000 7f52b63ff700 1 Processor -- start 2023-10-17T01:13:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.671+0000 7f52b63ff700 1 -- start start 2023-10-17T01:13:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.671+0000 7f52b63ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52b0100d40 0x7f52b0074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:05.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.671+0000 7f52b63ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.671+0000 7f52b63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b01028d0 0x7f52b0073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52b63ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0073e20 con 0x7f52b0101f30 2023-10-17T01:13:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52b63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0073f90 con 0x7f52b01028d0 2023-10-17T01:13:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52b63ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01a0af0 con 0x7f52b0100d40 2023-10-17T01:13:05.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52b499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b01028d0 0x7f52b0073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54770/0 (socket says 172.21.15.45:54770) 2023-10-17T01:13:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 -- 172.21.15.45:0/526212144 learned_addr learned my addr 172.21.15.45:0/526212144 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52affff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52b0100d40 0x7f52b0074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 -- 172.21.15.45:0/526212144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52b0100d40 msgr2=0x7f52b0074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52b0100d40 0x7f52b0074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 -- 172.21.15.45:0/526212144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b01028d0 msgr2=0x7f52b0073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b01028d0 0x7f52b0073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:05.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 -- 172.21.15.45:0/526212144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40095f0 con 0x7f52b0101f30 2023-10-17T01:13:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52b499c700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b01028d0 0x7f52b0073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.672+0000 7f52af7fe700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0073140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009990 tx=0x7f52a400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.673+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a401d070 con 0x7f52b0101f30 2023-10-17T01:13:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.673+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a4005030 con 0x7f52b0101f30 2023-10-17T01:13:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.673+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b01a0d70 con 0x7f52b0101f30 2023-10-17T01:13:05.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.673+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004a70 con 0x7f52b0101f30 2023-10-17T01:13:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.673+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b01a1190 con 0x7f52b0101f30 2023-10-17T01:13:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.674+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b0107cc0 con 0x7f52b0101f30 2023-10-17T01:13:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.676+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f52a4005410 con 0x7f52b0101f30 2023-10-17T01:13:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.678+0000 7f52ad7fa700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f52980a0b20 0x7f52980a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.678+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52a40cce80 con 0x7f52b0101f30 2023-10-17T01:13:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.678+0000 7f52affff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f52980a0b20 0x7f52980a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.679+0000 7f52affff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f52980a0b20 0x7f52980a2fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52b0101d90 tx=0x7f52a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:05.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.682+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f52a4096950 con 0x7f52b0101f30 2023-10-17T01:13:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: pgmap v269: 97 pgs: 9 creating+peering, 4 creating+activating, 84 active+clean; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2023-10-17T01:13:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3463502413' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-17T01:13:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] up:active 2023-10-17T01:13:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3463502413' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-17T01:13:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-17T01:13:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-17T01:13:05.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-17T01:13:05.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:05 smithi045 ceph-mon[98474]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-17T01:13:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: pgmap v269: 97 pgs: 9 creating+peering, 4 creating+activating, 84 active+clean; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2023-10-17T01:13:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3463502413' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-17T01:13:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] up:active 2023-10-17T01:13:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3463502413' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-17T01:13:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-17T01:13:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-17T01:13:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-17T01:13:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:05 smithi060 ceph-mon[134136]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-17T01:13:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: pgmap v269: 97 pgs: 9 creating+peering, 4 creating+activating, 84 active+clean; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2023-10-17T01:13:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3463502413' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-17T01:13:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] up:active 2023-10-17T01:13:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3463502413' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-17T01:13:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-17T01:13:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-17T01:13:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-17T01:13:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:05 smithi181 ceph-mon[134520]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-17T01:13:05.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:05.943+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f52b002d050 con 0x7f52b0101f30 2023-10-17T01:13:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.436+0000 7f52ad7fa700 1 -- 172.21.15.45:0/526212144 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f52a4083e60 con 0x7f52b0101f30 2023-10-17T01:13:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.439+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f52980a0b20 msgr2=0x7f52980a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.439+0000 7f52b63ff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f52980a0b20 0x7f52980a2fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52b0101d90 tx=0x7f52a0009260 comp rx=0 tx=0).stop 2023-10-17T01:13:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.439+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 msgr2=0x7f52b0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.439+0000 7f52b63ff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0073140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009990 tx=0x7f52a400f690 comp rx=0 tx=0).stop 2023-10-17T01:13:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 shutdown_connections 2023-10-17T01:13:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f52980a0b20 0x7f52980a2fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52b01028d0 0x7f52b0073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52b0100d40 0x7f52b0074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 --2- 172.21.15.45:0/526212144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0101f30 0x7f52b0073140 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 >> 172.21.15.45:0/526212144 conn(0x7f52b00fc290 msgr2=0x7f52b00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 shutdown_connections 2023-10-17T01:13:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.440+0000 7f52b63ff700 1 -- 172.21.15.45:0/526212144 wait complete. 2023-10-17T01:13:06.467 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-10-17T01:13:06.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-17T01:13:06.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/526212144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-17T01:13:06.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] up:active 2023-10-17T01:13:06.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/526212144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-17T01:13:06.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-17T01:13:06.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-10-17T01:13:06.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-10-17T01:13:06.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-mon[134136]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-10-17T01:13:06.580 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: 2023-10-17T01:13:06.249+0000 7fb246f5d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-17T01:13:06.580 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:13:06 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: 2023-10-17T01:13:06.249+0000 7fb246f5d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-17T01:13:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.678+0000 7f8098fff700 1 -- 172.21.15.45:0/1146981548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80941028d0 msgr2=0x7f8094109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.678+0000 7f8098fff700 1 --2- 172.21.15.45:0/1146981548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80941028d0 0x7f8094109180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b0d0 tx=0x7f808c00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 -- 172.21.15.45:0/1146981548 shutdown_connections 2023-10-17T01:13:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 --2- 172.21.15.45:0/1146981548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80941028d0 0x7f8094109180 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 --2- 172.21.15.45:0/1146981548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8094100d40 0x7f8094101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 --2- 172.21.15.45:0/1146981548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8094101f30 0x7f8094102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 -- 172.21.15.45:0/1146981548 >> 172.21.15.45:0/1146981548 conn(0x7f80940fc290 msgr2=0x7f80940fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 -- 172.21.15.45:0/1146981548 shutdown_connections 2023-10-17T01:13:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.679+0000 7f8098fff700 1 -- 172.21.15.45:0/1146981548 wait complete. 2023-10-17T01:13:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 Processor -- start 2023-10-17T01:13:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 -- start start 2023-10-17T01:13:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8094100d40 0x7f8094074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 0x7f8094073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80941028d0 0x7f8094073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094073e20 con 0x7f8094101f30 2023-10-17T01:13:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094073f90 con 0x7f8094100d40 2023-10-17T01:13:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8098fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941a0bb0 con 0x7f80941028d0 2023-10-17T01:13:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8091d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 0x7f8094073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8091d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 0x7f8094073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54776/0 (socket says 172.21.15.45:54776) 2023-10-17T01:13:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f8091d9b700 1 -- 172.21.15.45:0/3424519241 learned_addr learned my addr 172.21.15.45:0/3424519241 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.680+0000 7f809259c700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8094100d40 0x7f8094074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8092d9d700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80941028d0 0x7f8094073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8091d9b700 1 -- 172.21.15.45:0/3424519241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80941028d0 msgr2=0x7f8094073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8091d9b700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80941028d0 0x7f8094073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8091d9b700 1 -- 172.21.15.45:0/3424519241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8094100d40 msgr2=0x7f8094074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8091d9b700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8094100d40 0x7f8094074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8091d9b700 1 -- 172.21.15.45:0/3424519241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c009d00 con 0x7f8094101f30 2023-10-17T01:13:06.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8091d9b700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 0x7f8094073140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f808800d950 tx=0x7f808800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:06.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800f890 con 0x7f8094101f30 2023-10-17T01:13:06.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8088010460 con 0x7f8094101f30 2023-10-17T01:13:06.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80941a0e90 con 0x7f8094101f30 2023-10-17T01:13:06.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808800e610 con 0x7f8094101f30 2023-10-17T01:13:06.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.681+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941a1360 con 0x7f8094101f30 2023-10-17T01:13:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.682+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8094108290 con 0x7f8094101f30 2023-10-17T01:13:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.684+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f80880108d0 con 0x7f8094101f30 2023-10-17T01:13:06.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.686+0000 7f80837fe700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f807c0a0ba0 0x7f807c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:06.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.686+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f80880cbdf0 con 0x7f8094101f30 2023-10-17T01:13:06.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.686+0000 7f809259c700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f807c0a0ba0 0x7f807c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:06.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.687+0000 7f809259c700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f807c0a0ba0 0x7f807c0a3060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8094101d90 tx=0x7f8084009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.690+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f8088095830 con 0x7f8094101f30 2023-10-17T01:13:06.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/526212144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-17T01:13:06.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] up:active 2023-10-17T01:13:06.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/526212144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-17T01:13:06.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-17T01:13:06.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-10-17T01:13:06.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-10-17T01:13:06.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-mon[98474]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-10-17T01:13:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/526212144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-17T01:13:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] up:active 2023-10-17T01:13:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/526212144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-17T01:13:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-17T01:13:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-10-17T01:13:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-10-17T01:13:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:06 smithi181 ceph-mon[134520]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-10-17T01:13:06.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:06.941+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f8094061190 con 0x7f807c0a0ba0 2023-10-17T01:13:07.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:13:06.947+0000 7f30308c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-17T01:13:07.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:13:06.947+0000 7f30308c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-17T01:13:07.301 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:13:06.947+0000 7f30308c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-17T01:13:07.302 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:13:06.947+0000 7f30308c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-17T01:13:07.302 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:13:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:13:06.947+0000 7f30308c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-17T01:13:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.487+0000 7f80837fe700 1 -- 172.21.15.45:0/3424519241 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8094061190 con 0x7f807c0a0ba0 2023-10-17T01:13:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f807c0a0ba0 msgr2=0x7f807c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f807c0a0ba0 0x7f807c0a3060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8094101d90 tx=0x7f8084009260 comp rx=0 tx=0).stop 2023-10-17T01:13:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 msgr2=0x7f8094073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 0x7f8094073140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f808800d950 tx=0x7f808800dc60 comp rx=0 tx=0).stop 2023-10-17T01:13:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 shutdown_connections 2023-10-17T01:13:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f807c0a0ba0 0x7f807c0a3060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80941028d0 0x7f8094073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8094100d40 0x7f8094074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 --2- 172.21.15.45:0/3424519241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094101f30 0x7f8094073140 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 >> 172.21.15.45:0/3424519241 conn(0x7f80940fc290 msgr2=0x7f80941053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 shutdown_connections 2023-10-17T01:13:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.490+0000 7f8098fff700 1 -- 172.21.15.45:0/3424519241 wait complete. 2023-10-17T01:13:07.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-17T01:13:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 -- 172.21.15.45:0/2226937330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8530101f30 msgr2=0x7f8530102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 --2- 172.21.15.45:0/2226937330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8530101f30 0x7f8530102390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85240099c0 tx=0x7f8524009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 -- 172.21.15.45:0/2226937330 shutdown_connections 2023-10-17T01:13:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 --2- 172.21.15.45:0/2226937330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85301028d0 0x7f8530109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 --2- 172.21.15.45:0/2226937330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8530100d40 0x7f8530101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 --2- 172.21.15.45:0/2226937330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8530101f30 0x7f8530102390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 -- 172.21.15.45:0/2226937330 >> 172.21.15.45:0/2226937330 conn(0x7f85300fc290 msgr2=0x7f85300fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 -- 172.21.15.45:0/2226937330 shutdown_connections 2023-10-17T01:13:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.720+0000 7f85361ff700 1 -- 172.21.15.45:0/2226937330 wait complete. 2023-10-17T01:13:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.721+0000 7f85361ff700 1 Processor -- start 2023-10-17T01:13:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 -- start start 2023-10-17T01:13:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8530100d40 0x7f8530198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101f30 0x7f85301991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 0x7f85301a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85301a0820 con 0x7f85301028d0 2023-10-17T01:13:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85301a0990 con 0x7f8530100d40 2023-10-17T01:13:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f85361ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85301a0b50 con 0x7f8530101f30 2023-10-17T01:13:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f852ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 0x7f85301a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f852f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8530100d40 0x7f8530198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f852ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 0x7f85301a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54800/0 (socket says 172.21.15.45:54800) 2023-10-17T01:13:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f852ffff700 1 -- 172.21.15.45:0/449178640 learned_addr learned my addr 172.21.15.45:0/449178640 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.722+0000 7f852effd700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101f30 0x7f85301991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852ffff700 1 -- 172.21.15.45:0/449178640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101f30 msgr2=0x7f85301991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852ffff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101f30 0x7f85301991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852ffff700 1 -- 172.21.15.45:0/449178640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8530100d40 msgr2=0x7f8530198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852ffff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8530100d40 0x7f8530198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852ffff700 1 -- 172.21.15.45:0/449178640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240095f0 con 0x7f85301028d0 2023-10-17T01:13:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852ffff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 0x7f85301a0250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f852800ef30 tx=0x7f852800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528018070 con 0x7f85301028d0 2023-10-17T01:13:07.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852800f040 con 0x7f85301028d0 2023-10-17T01:13:07.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85301a0dd0 con 0x7f85301028d0 2023-10-17T01:13:07.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800cd00 con 0x7f85301028d0 2023-10-17T01:13:07.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.723+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85301a1250 con 0x7f85301028d0 2023-10-17T01:13:07.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.724+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853010c5a0 con 0x7f85301028d0 2023-10-17T01:13:07.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.726+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8528003a00 con 0x7f85301028d0 2023-10-17T01:13:07.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.728+0000 7f852cff9700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85180a0b20 0x7f85180a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.728+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f85280cbb50 con 0x7f85301028d0 2023-10-17T01:13:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.728+0000 7f852f7fe700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85180a0b20 0x7f85180a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.728+0000 7f852f7fe700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85180a0b20 0x7f85180a2fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8530199b80 tx=0x7f8520007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.731+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f8528095620 con 0x7f85301028d0 2023-10-17T01:13:07.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: pgmap v270: 97 pgs: 97 active+clean; 581 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s wr, 13 op/s 2023-10-17T01:13:07.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] up:active 2023-10-17T01:13:07.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-17T01:13:07.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-17T01:13:07.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: Cluster is now healthy 2023-10-17T01:13:07.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-10-17T01:13:07.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-17T01:13:07.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:07 smithi045 ceph-mon[98474]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-10-17T01:13:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: pgmap v270: 97 pgs: 97 active+clean; 581 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s wr, 13 op/s 2023-10-17T01:13:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] up:active 2023-10-17T01:13:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-17T01:13:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-17T01:13:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: Cluster is now healthy 2023-10-17T01:13:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-10-17T01:13:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-17T01:13:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:07 smithi060 ceph-mon[134136]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-10-17T01:13:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: pgmap v270: 97 pgs: 97 active+clean; 581 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s wr, 13 op/s 2023-10-17T01:13:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] up:active 2023-10-17T01:13:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-17T01:13:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-17T01:13:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: Cluster is now healthy 2023-10-17T01:13:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-10-17T01:13:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-17T01:13:07.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:07 smithi181 ceph-mon[134520]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-10-17T01:13:07.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.979+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f85301088c0 con 0x7f85180a0b20 2023-10-17T01:13:07.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.984+0000 7f852cff9700 1 -- 172.21.15.45:0/449178640 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f85301088c0 con 0x7f85180a0b20 2023-10-17T01:13:07.985 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/_nogroup/sv_0/e7238776-7516-41a5-aa32-1e4c0d896191 2023-10-17T01:13:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85180a0b20 msgr2=0x7f85180a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85180a0b20 0x7f85180a2fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8530199b80 tx=0x7f8520007400 comp rx=0 tx=0).stop 2023-10-17T01:13:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 msgr2=0x7f85301a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 0x7f85301a0250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f852800ef30 tx=0x7f852800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 shutdown_connections 2023-10-17T01:13:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85180a0b20 0x7f85180a2fe0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85301028d0 0x7f85301a0250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8530100d40 0x7f8530198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 --2- 172.21.15.45:0/449178640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8530101f30 0x7f85301991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.986+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 >> 172.21.15.45:0/449178640 conn(0x7f85300fc290 msgr2=0x7f85300fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.987+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 shutdown_connections 2023-10-17T01:13:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:07.987+0000 7f85361ff700 1 -- 172.21.15.45:0/449178640 wait complete. 2023-10-17T01:13:08.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-17T01:13:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 -- 172.21.15.45:0/1169573223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 msgr2=0x7f60fc101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 --2- 172.21.15.45:0/1169573223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc101160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009990 tx=0x7f60ec009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 -- 172.21.15.45:0/1169573223 shutdown_connections 2023-10-17T01:13:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 --2- 172.21.15.45:0/1169573223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc1028d0 0x7f60fc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 --2- 172.21.15.45:0/1169573223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc101160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 --2- 172.21.15.45:0/1169573223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc101f30 0x7f60fc102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 -- 172.21.15.45:0/1169573223 >> 172.21.15.45:0/1169573223 conn(0x7f60fc0fc290 msgr2=0x7f60fc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 -- 172.21.15.45:0/1169573223 shutdown_connections 2023-10-17T01:13:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.215+0000 7f61013ff700 1 -- 172.21.15.45:0/1169573223 wait complete. 2023-10-17T01:13:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.217+0000 7f61013ff700 1 Processor -- start 2023-10-17T01:13:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.217+0000 7f61013ff700 1 -- start start 2023-10-17T01:13:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.217+0000 7f61013ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.217+0000 7f61013ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 0x7f60fc197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f61013ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc1028d0 0x7f60fc19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f61013ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc19e640 con 0x7f60fc101f30 2023-10-17T01:13:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f61013ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc19e780 con 0x7f60fc1028d0 2023-10-17T01:13:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f61013ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc19e910 con 0x7f60fc100d40 2023-10-17T01:13:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60fa7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 0x7f60fc197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60fa7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 0x7f60fc197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55620/0 (socket says 172.21.15.45:55620) 2023-10-17T01:13:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60fa7fc700 1 -- 172.21.15.45:0/419082620 learned_addr learned my addr 172.21.15.45:0/419082620 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47316/0 (socket says 172.21.15.45:47316) 2023-10-17T01:13:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60fb7fe700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc1028d0 0x7f60fc19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 -- 172.21.15.45:0/419082620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc1028d0 msgr2=0x7f60fc19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc1028d0 0x7f60fc19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 -- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 msgr2=0x7f60fc197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 0x7f60fc197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 -- 172.21.15.45:0/419082620 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0095f0 con 0x7f60fc100d40 2023-10-17T01:13:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60fb7fe700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc1028d0 0x7f60fc19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60fa7fc700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 0x7f60fc197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.218+0000 7f60faffd700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc196ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b680 tx=0x7f60ec009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.219+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec01d070 con 0x7f60fc100d40 2023-10-17T01:13:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.219+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc19eb90 con 0x7f60fc100d40 2023-10-17T01:13:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.219+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60ec021410 con 0x7f60fc100d40 2023-10-17T01:13:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.219+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc19f010 con 0x7f60fc100d40 2023-10-17T01:13:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.219+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec021ea0 con 0x7f60fc100d40 2023-10-17T01:13:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.220+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc10c5a0 con 0x7f60fc100d40 2023-10-17T01:13:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.222+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f60ec021830 con 0x7f60fc100d40 2023-10-17T01:13:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.223+0000 7f60e3fff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60e40a0ba0 0x7f60e40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.224+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f60ec0cd710 con 0x7f60fc100d40 2023-10-17T01:13:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.224+0000 7f60fa7fc700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60e40a0ba0 0x7f60e40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.224+0000 7f60fa7fc700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60e40a0ba0 0x7f60e40a3060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60f0011550 tx=0x7f60f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.225+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f60ec0971e0 con 0x7f60fc100d40 2023-10-17T01:13:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.476+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f60fc061190 con 0x7f60e40a0ba0 2023-10-17T01:13:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.506+0000 7f60e3fff700 1 -- 172.21.15.45:0/419082620 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f60fc061190 con 0x7f60e40a0ba0 2023-10-17T01:13:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60e40a0ba0 msgr2=0x7f60e40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60e40a0ba0 0x7f60e40a3060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60f0011550 tx=0x7f60f0009260 comp rx=0 tx=0).stop 2023-10-17T01:13:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 msgr2=0x7f60fc196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc196ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b680 tx=0x7f60ec009580 comp rx=0 tx=0).stop 2023-10-17T01:13:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 shutdown_connections 2023-10-17T01:13:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60e40a0ba0 0x7f60e40a3060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60fc1028d0 0x7f60fc19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60fc100d40 0x7f60fc196ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 --2- 172.21.15.45:0/419082620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60fc101f30 0x7f60fc197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 >> 172.21.15.45:0/419082620 conn(0x7f60fc0fc290 msgr2=0x7f60fc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 shutdown_connections 2023-10-17T01:13:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.508+0000 7f61013ff700 1 -- 172.21.15.45:0/419082620 wait complete. 2023-10-17T01:13:08.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-17T01:13:08.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.735+0000 7f3949dff700 1 -- 172.21.15.45:0/1085214690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944108c00 msgr2=0x7f394410b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.735+0000 7f3949dff700 1 --2- 172.21.15.45:0/1085214690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944108c00 0x7f394410b090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f393c00b0a0 tx=0x7f393c00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 -- 172.21.15.45:0/1085214690 shutdown_connections 2023-10-17T01:13:08.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 --2- 172.21.15.45:0/1085214690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944108c00 0x7f394410b090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 --2- 172.21.15.45:0/1085214690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3944074d90 0x7f39440731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 --2- 172.21.15.45:0/1085214690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39440737b0 0x7f3944073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 -- 172.21.15.45:0/1085214690 >> 172.21.15.45:0/1085214690 conn(0x7f39440fc290 msgr2=0x7f39440fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:08.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 -- 172.21.15.45:0/1085214690 shutdown_connections 2023-10-17T01:13:08.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.736+0000 7f3949dff700 1 -- 172.21.15.45:0/1085214690 wait complete. 2023-10-17T01:13:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 Processor -- start 2023-10-17T01:13:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 -- start start 2023-10-17T01:13:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39440737b0 0x7f3944198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 0x7f3944199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3944108c00 0x7f39441a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441a0940 con 0x7f3944074d90 2023-10-17T01:13:08.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441a0a80 con 0x7f3944108c00 2023-10-17T01:13:08.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3949dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441a0c10 con 0x7f39440737b0 2023-10-17T01:13:08.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3942ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 0x7f3944199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3942ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 0x7f3944199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55640/0 (socket says 172.21.15.45:55640) 2023-10-17T01:13:08.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3942ffd700 1 -- 172.21.15.45:0/1477007483 learned_addr learned my addr 172.21.15.45:0/1477007483 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:08.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f39437fe700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39440737b0 0x7f3944198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.737+0000 7f3943fff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3944108c00 0x7f39441a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3942ffd700 1 -- 172.21.15.45:0/1477007483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39440737b0 msgr2=0x7f3944198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3942ffd700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39440737b0 0x7f3944198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3942ffd700 1 -- 172.21.15.45:0/1477007483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3944108c00 msgr2=0x7f39441a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3942ffd700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3944108c00 0x7f39441a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3942ffd700 1 -- 172.21.15.45:0/1477007483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c009d00 con 0x7f3944074d90 2023-10-17T01:13:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3942ffd700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 0x7f3944199300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f393800e970 tx=0x7f393800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938009790 con 0x7f3944074d90 2023-10-17T01:13:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f393800c7e0 con 0x7f3944074d90 2023-10-17T01:13:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938003c70 con 0x7f3944074d90 2023-10-17T01:13:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39441a0ef0 con 0x7f3944074d90 2023-10-17T01:13:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.738+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39441a1320 con 0x7f3944074d90 2023-10-17T01:13:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.740+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39441072b0 con 0x7f3944074d90 2023-10-17T01:13:08.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.742+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f393800cc50 con 0x7f3944074d90 2023-10-17T01:13:08.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.743+0000 7f3940ff9700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f392c0a0b20 0x7f392c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.743+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3938014070 con 0x7f3944074d90 2023-10-17T01:13:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.743+0000 7f39437fe700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f392c0a0b20 0x7f392c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.744+0000 7f39437fe700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f392c0a0b20 0x7f392c0a2fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3934009c60 tx=0x7f3934009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.747+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3938095cd0 con 0x7f3944074d90 2023-10-17T01:13:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:08 smithi045 ceph-mon[98474]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:08 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] up:active 2023-10-17T01:13:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:08 smithi045 ceph-mon[98474]: fsmap cephfs:5 7 up:standby 5 up:active 2023-10-17T01:13:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:08 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:13:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:08 smithi060 ceph-mon[134136]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:08 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] up:active 2023-10-17T01:13:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:08 smithi060 ceph-mon[134136]: fsmap cephfs:5 7 up:standby 5 up:active 2023-10-17T01:13:08.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:08 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:13:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:08 smithi181 ceph-mon[134520]: from='client.14649 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:08 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] up:active 2023-10-17T01:13:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:08 smithi181 ceph-mon[134520]: fsmap cephfs:5 7 up:standby 5 up:active 2023-10-17T01:13:08.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:08 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:13:08.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:08.997+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3944061190 con 0x7f392c0a0b20 2023-10-17T01:13:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.001+0000 7f3940ff9700 1 -- 172.21.15.45:0/1477007483 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f3944061190 con 0x7f392c0a0b20 2023-10-17T01:13:09.003 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/_nogroup/sv_1/c18bfcb3-8f74-497f-901a-c6d13f726359 2023-10-17T01:13:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f392c0a0b20 msgr2=0x7f392c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f392c0a0b20 0x7f392c0a2fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3934009c60 tx=0x7f3934009260 comp rx=0 tx=0).stop 2023-10-17T01:13:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 msgr2=0x7f3944199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 0x7f3944199300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f393800e970 tx=0x7f393800ec80 comp rx=0 tx=0).stop 2023-10-17T01:13:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 shutdown_connections 2023-10-17T01:13:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f392c0a0b20 0x7f392c0a2fe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3944108c00 0x7f39441a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39440737b0 0x7f3944198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 --2- 172.21.15.45:0/1477007483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3944074d90 0x7f3944199300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 >> 172.21.15.45:0/1477007483 conn(0x7f39440fc290 msgr2=0x7f39440fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 shutdown_connections 2023-10-17T01:13:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.004+0000 7f3949dff700 1 -- 172.21.15.45:0/1477007483 wait complete. 2023-10-17T01:13:09.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:13:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 -- 172.21.15.45:0/1050364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500108c10 msgr2=0x7f550010b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 --2- 172.21.15.45:0/1050364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500108c10 0x7f550010b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54f800b0a0 tx=0x7f54f800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 -- 172.21.15.45:0/1050364 shutdown_connections 2023-10-17T01:13:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 --2- 172.21.15.45:0/1050364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500108c10 0x7f550010b0a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 --2- 172.21.15.45:0/1050364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500074d90 0x7f55000731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 --2- 172.21.15.45:0/1050364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55000737b0 0x7f5500073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 -- 172.21.15.45:0/1050364 >> 172.21.15.45:0/1050364 conn(0x7f55000fc280 msgr2=0x7f55000fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 -- 172.21.15.45:0/1050364 shutdown_connections 2023-10-17T01:13:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.230+0000 7f55063ff700 1 -- 172.21.15.45:0/1050364 wait complete. 2023-10-17T01:13:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.231+0000 7f55063ff700 1 Processor -- start 2023-10-17T01:13:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 -- start start 2023-10-17T01:13:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 0x7f5500198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500074d90 0x7f55001991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500108c10 0x7f55001a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55001a07f0 con 0x7f55000737b0 2023-10-17T01:13:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55001a0960 con 0x7f5500108c10 2023-10-17T01:13:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f55063ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55001a0b20 con 0x7f5500074d90 2023-10-17T01:13:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f54fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 0x7f5500198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f54fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 0x7f5500198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55656/0 (socket says 172.21.15.45:55656) 2023-10-17T01:13:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f54fffff700 1 -- 172.21.15.45:0/1730384086 learned_addr learned my addr 172.21.15.45:0/1730384086 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f54ff7fe700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500074d90 0x7f55001991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.232+0000 7f550499c700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500108c10 0x7f55001a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fffff700 1 -- 172.21.15.45:0/1730384086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500074d90 msgr2=0x7f55001991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fffff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500074d90 0x7f55001991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fffff700 1 -- 172.21.15.45:0/1730384086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500108c10 msgr2=0x7f55001a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fffff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500108c10 0x7f55001a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fffff700 1 -- 172.21.15.45:0/1730384086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f8009d00 con 0x7f55000737b0 2023-10-17T01:13:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54ff7fe700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500074d90 0x7f55001991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f550499c700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500108c10 0x7f55001a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fffff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 0x7f5500198c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e9a0 tx=0x7f54f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0009790 con 0x7f55000737b0 2023-10-17T01:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f000cbd0 con 0x7f55000737b0 2023-10-17T01:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55001a0e00 con 0x7f55000737b0 2023-10-17T01:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0018680 con 0x7f55000737b0 2023-10-17T01:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.233+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55001a1280 con 0x7f55000737b0 2023-10-17T01:13:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.234+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55001079d0 con 0x7f55000737b0 2023-10-17T01:13:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.236+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f54f000cd40 con 0x7f55000737b0 2023-10-17T01:13:09.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.238+0000 7f54fd7fa700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f54e80a0bf0 0x7f54e80a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.239+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54f0014070 con 0x7f55000737b0 2023-10-17T01:13:09.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.239+0000 7f54ff7fe700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f54e80a0bf0 0x7f54e80a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.239+0000 7f54ff7fe700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f54e80a0bf0 0x7f54e80a30b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f550019a180 tx=0x7f54f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.242+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f54f00520b0 con 0x7f55000737b0 2023-10-17T01:13:09.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.504+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f54f00960e0 con 0x7f55000737b0 2023-10-17T01:13:09.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.505+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f550002cc30 con 0x7f55000737b0 2023-10-17T01:13:09.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.506+0000 7f54fd7fa700 1 -- 172.21.15.45:0/1730384086 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f54f0095d10 con 0x7f55000737b0 2023-10-17T01:13:09.508 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:09.508 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:13:09.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f54e80a0bf0 msgr2=0x7f54e80a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f54e80a0bf0 0x7f54e80a30b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f550019a180 tx=0x7f54f4009260 comp rx=0 tx=0).stop 2023-10-17T01:13:09.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 msgr2=0x7f5500198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 0x7f5500198c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e9a0 tx=0x7f54f000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 shutdown_connections 2023-10-17T01:13:09.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f54e80a0bf0 0x7f54e80a30b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5500108c10 0x7f55001a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000737b0 0x7f5500198c80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 --2- 172.21.15.45:0/1730384086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500074d90 0x7f55001991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.510+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 >> 172.21.15.45:0/1730384086 conn(0x7f55000fc280 msgr2=0x7f5500104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:09.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.511+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 shutdown_connections 2023-10-17T01:13:09.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.511+0000 7f55063ff700 1 -- 172.21.15.45:0/1730384086 wait complete. 2023-10-17T01:13:09.515 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:13:09.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:13:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 -- 172.21.15.45:0/4189433405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78100d60 msgr2=0x7f1b78101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 --2- 172.21.15.45:0/4189433405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78100d60 0x7f1b78101180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1b740099c0 tx=0x7f1b74009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 -- 172.21.15.45:0/4189433405 shutdown_connections 2023-10-17T01:13:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 --2- 172.21.15.45:0/4189433405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b781028f0 0x7f1b781091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 --2- 172.21.15.45:0/4189433405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78100d60 0x7f1b78101180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 --2- 172.21.15.45:0/4189433405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b78101f50 0x7f1b781023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 -- 172.21.15.45:0/4189433405 >> 172.21.15.45:0/4189433405 conn(0x7f1b780fc290 msgr2=0x7f1b780fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 -- 172.21.15.45:0/4189433405 shutdown_connections 2023-10-17T01:13:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.733+0000 7f1b801ff700 1 -- 172.21.15.45:0/4189433405 wait complete. 2023-10-17T01:13:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.734+0000 7f1b801ff700 1 Processor -- start 2023-10-17T01:13:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.734+0000 7f1b801ff700 1 -- start start 2023-10-17T01:13:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b801ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b78100d60 0x7f1b78198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b801ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 0x7f1b781991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b801ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b781028f0 0x7f1b781a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b801ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a07e0 con 0x7f1b78101f50 2023-10-17T01:13:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b801ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a0950 con 0x7f1b78100d60 2023-10-17T01:13:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b801ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a0b10 con 0x7f1b781028f0 2023-10-17T01:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 0x7f1b781991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7df9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b78100d60 0x7f1b78198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7e79c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b781028f0 0x7f1b781a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7df9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b78100d60 0x7f1b78198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43100/0 (socket says 172.21.15.45:43100) 2023-10-17T01:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 0x7f1b781991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55670/0 (socket says 172.21.15.45:55670) 2023-10-17T01:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7e79c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b781028f0 0x7f1b781a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47378/0 (socket says 172.21.15.45:47378) 2023-10-17T01:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7df9b700 1 -- 172.21.15.45:0/480878845 learned_addr learned my addr 172.21.15.45:0/480878845 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 -- 172.21.15.45:0/480878845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b781028f0 msgr2=0x7f1b781a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b781028f0 0x7f1b781a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 -- 172.21.15.45:0/480878845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b78100d60 msgr2=0x7f1b78198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b78100d60 0x7f1b78198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.735+0000 7f1b7d79a700 1 -- 172.21.15.45:0/480878845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b70009c40 con 0x7f1b78101f50 2023-10-17T01:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.736+0000 7f1b7d79a700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 0x7f1b781991b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9f0 tx=0x7f1b6800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.736+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800c9c0 con 0x7f1b78101f50 2023-10-17T01:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.736+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b740095f0 con 0x7f1b78101f50 2023-10-17T01:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.736+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b68010430 con 0x7f1b78101f50 2023-10-17T01:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.736+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68018680 con 0x7f1b78101f50 2023-10-17T01:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.736+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b781a1170 con 0x7f1b78101f50 2023-10-17T01:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.737+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b781933b0 con 0x7f1b78101f50 2023-10-17T01:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.739+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f1b68010860 con 0x7f1b78101f50 2023-10-17T01:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.740+0000 7f1b6effd700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b640a0bf0 0x7f1b640a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.740+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b68014070 con 0x7f1b78101f50 2023-10-17T01:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.741+0000 7f1b7df9b700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b640a0bf0 0x7f1b640a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.741+0000 7f1b7df9b700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b640a0bf0 0x7f1b640a30b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b74000c00 tx=0x7f1b74005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.744+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1b680520b0 con 0x7f1b78101f50 2023-10-17T01:13:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:09 smithi045 ceph-mon[98474]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:09 smithi045 ceph-mon[98474]: pgmap v271: 97 pgs: 97 active+clean; 582 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 14 op/s 2023-10-17T01:13:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:09 smithi045 ceph-mon[98474]: from='client.24685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:09 smithi060 ceph-mon[134136]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:09 smithi060 ceph-mon[134136]: pgmap v271: 97 pgs: 97 active+clean; 582 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 14 op/s 2023-10-17T01:13:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:09 smithi060 ceph-mon[134136]: from='client.24685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:09 smithi181 ceph-mon[134520]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:09 smithi181 ceph-mon[134520]: pgmap v271: 97 pgs: 97 active+clean; 582 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 14 op/s 2023-10-17T01:13:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:09 smithi181 ceph-mon[134520]: from='client.24685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:09.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.959+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b7802ce80 con 0x7f1b78101f50 2023-10-17T01:13:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.961+0000 7f1b6effd700 1 -- 172.21.15.45:0/480878845 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7f1b68095e80 con 0x7f1b78101f50 2023-10-17T01:13:09.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:09.963 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:02.040707+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:13:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b640a0bf0 msgr2=0x7f1b640a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b640a0bf0 0x7f1b640a30b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b74000c00 tx=0x7f1b74005fb0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 msgr2=0x7f1b781991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 0x7f1b781991b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9f0 tx=0x7f1b6800ed00 comp rx=0 tx=0).stop 2023-10-17T01:13:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 shutdown_connections 2023-10-17T01:13:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1b640a0bf0 0x7f1b640a30b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b781028f0 0x7f1b781a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b78100d60 0x7f1b78198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 --2- 172.21.15.45:0/480878845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b78101f50 0x7f1b781991b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.963+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 >> 172.21.15.45:0/480878845 conn(0x7f1b780fc290 msgr2=0x7f1b78107830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.964+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 shutdown_connections 2023-10-17T01:13:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:09.964+0000 7f1b801ff700 1 -- 172.21.15.45:0/480878845 wait complete. 2023-10-17T01:13:09.984 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-17T01:13:09.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-17T01:13:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.187+0000 7f17a43ff700 1 -- 172.21.15.45:0/1463784473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1028f0 msgr2=0x7f179c1091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.187+0000 7f17a43ff700 1 --2- 172.21.15.45:0/1463784473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1028f0 0x7f179c1091a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f179400b0d0 tx=0x7f179400b3e0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 -- 172.21.15.45:0/1463784473 shutdown_connections 2023-10-17T01:13:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 --2- 172.21.15.45:0/1463784473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1028f0 0x7f179c1091a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 --2- 172.21.15.45:0/1463784473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c100d60 0x7f179c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 --2- 172.21.15.45:0/1463784473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f179c101f50 0x7f179c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 -- 172.21.15.45:0/1463784473 >> 172.21.15.45:0/1463784473 conn(0x7f179c0fc290 msgr2=0x7f179c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 -- 172.21.15.45:0/1463784473 shutdown_connections 2023-10-17T01:13:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.188+0000 7f17a43ff700 1 -- 172.21.15.45:0/1463784473 wait complete. 2023-10-17T01:13:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 Processor -- start 2023-10-17T01:13:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 -- start start 2023-10-17T01:13:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 0x7f179c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c101f50 0x7f179c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f179c1028f0 0x7f179c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1a08e0 con 0x7f179c100d60 2023-10-17T01:13:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1a0a20 con 0x7f179c101f50 2023-10-17T01:13:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.189+0000 7f17a43ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1a0bb0 con 0x7f179c1028f0 2023-10-17T01:13:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 0x7f179c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 0x7f179c198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55686/0 (socket says 172.21.15.45:55686) 2023-10-17T01:13:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 -- 172.21.15.45:0/2315923572 learned_addr learned my addr 172.21.15.45:0/2315923572 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a299c700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f179c1028f0 0x7f179c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a199a700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c101f50 0x7f179c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 -- 172.21.15.45:0/2315923572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f179c1028f0 msgr2=0x7f179c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f179c1028f0 0x7f179c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 -- 172.21.15.45:0/2315923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c101f50 msgr2=0x7f179c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c101f50 0x7f179c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 -- 172.21.15.45:0/2315923572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1798009520 con 0x7f179c100d60 2023-10-17T01:13:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a219b700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 0x7f179c198d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f179800ea50 tx=0x7f179800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179800cbd0 con 0x7f179c100d60 2023-10-17T01:13:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179800cd30 con 0x7f179c100d60 2023-10-17T01:13:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.190+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1794009d00 con 0x7f179c100d60 2023-10-17T01:13:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.191+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798018680 con 0x7f179c100d60 2023-10-17T01:13:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.191+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c1a11b0 con 0x7f179c100d60 2023-10-17T01:13:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.192+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c10c640 con 0x7f179c100d60 2023-10-17T01:13:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.193+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f1798010430 con 0x7f179c100d60 2023-10-17T01:13:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.195+0000 7f17937fe700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17880a0cf0 0x7f17880a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.195+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1798014070 con 0x7f179c100d60 2023-10-17T01:13:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.195+0000 7f17a199a700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17880a0cf0 0x7f17880a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.196+0000 7f17a199a700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17880a0cf0 0x7f17880a31b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f178c009be0 tx=0x7f178c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.199+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1798095fa0 con 0x7f179c100d60 2023-10-17T01:13:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.406+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f179c108e90 con 0x7f17880a0cf0 2023-10-17T01:13:10.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.420+0000 7f17937fe700 1 -- 172.21.15.45:0/2315923572 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336118 (secure 0 0 0) 0x7f179c108e90 con 0x7f17880a0cf0 2023-10-17T01:13:10.425 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:10.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17880a0cf0 msgr2=0x7f17880a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17880a0cf0 0x7f17880a31b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f178c009be0 tx=0x7f178c009260 comp rx=0 tx=0).stop 2023-10-17T01:13:10.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 msgr2=0x7f179c198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 0x7f179c198d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f179800ea50 tx=0x7f179800ed60 comp rx=0 tx=0).stop 2023-10-17T01:13:10.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 shutdown_connections 2023-10-17T01:13:10.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17880a0cf0 0x7f17880a31b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f179c1028f0 0x7f179c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c100d60 0x7f179c198d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 --2- 172.21.15.45:0/2315923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c101f50 0x7f179c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 >> 172.21.15.45:0/2315923572 conn(0x7f179c0fc290 msgr2=0x7f179c107770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 shutdown_connections 2023-10-17T01:13:10.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.425+0000 7f17a43ff700 1 -- 172.21.15.45:0/2315923572 wait complete. 2023-10-17T01:13:10.431 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:13:10.446 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2023-10-17T01:13:08.432650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":297,"num_write_kb":2970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334160,"kb_used_data":10736,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607664,"statfs":{"total":1151940427776,"available":1151598247936,"internally_reserved":0,"allocated":10993664,"data_stored":6884628,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5540,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.961392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849116+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:13:02.849116+0000","last_peered":"2023-10-17T01:13:02.849116+0000","last_clean":"2023-10-17T01:13:02.849116+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:13:02.849116+0000","last_undegraded":"2023-10-17T01:13:02.849116+0000","last_fullsized":"2023-10-17T01:13:02.849116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033441+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:13:02.033441+0000","last_peered":"2023-10-17T01:13:02.033441+0000","last_clean":"2023-10-17T01:13:02.033441+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:13:02.033441+0000","last_undegraded":"2023-10-17T01:13:02.033441+0000","last_fullsized":"2023-10-17T01:13:02.033441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033424+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:13:02.033424+0000","last_peered":"2023-10-17T01:13:02.033424+0000","last_clean":"2023-10-17T01:13:02.033424+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:13:02.033424+0000","last_undegraded":"2023-10-17T01:13:02.033424+0000","last_fullsized":"2023-10-17T01:13:02.033424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.073086+0000","last_change":"2023-10-17T01:13:01.071138+0000","last_active":"2023-10-17T01:13:01.073086+0000","last_peered":"2023-10-17T01:13:01.073086+0000","last_clean":"2023-10-17T01:13:01.073086+0000","last_became_active":"2023-10-17T01:13:01.070813+0000","last_became_peered":"2023-10-17T01:13:01.070813+0000","last_unstale":"2023-10-17T01:13:01.073086+0000","last_undegraded":"2023-10-17T01:13:01.073086+0000","last_fullsized":"2023-10-17T01:13:01.073086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:13:32.387439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847607+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:13:02.847607+0000","last_peered":"2023-10-17T01:13:02.847607+0000","last_clean":"2023-10-17T01:13:02.847607+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:13:02.847607+0000","last_undegraded":"2023-10-17T01:13:02.847607+0000","last_fullsized":"2023-10-17T01:13:02.847607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940585+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:13:01.940585+0000","last_peered":"2023-10-17T01:13:01.940585+0000","last_clean":"2023-10-17T01:13:01.940585+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:13:01.940585+0000","last_undegraded":"2023-10-17T01:13:01.940585+0000","last_fullsized":"2023-10-17T01:13:01.940585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347542+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:13:02.347542+0000","last_peered":"2023-10-17T01:13:02.347542+0000","last_clean":"2023-10-17T01:13:02.347542+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:13:02.347542+0000","last_undegraded":"2023-10-17T01:13:02.347542+0000","last_fullsized":"2023-10-17T01:13:02.347542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630206+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:13:02.630206+0000","last_peered":"2023-10-17T01:13:02.630206+0000","last_clean":"2023-10-17T01:13:02.630206+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:13:02.630206+0000","last_undegraded":"2023-10-17T01:13:02.630206+0000","last_fullsized":"2023-10-17T01:13:02.630206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.635096+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:13:02.635096+0000","last_peered":"2023-10-17T01:13:02.635096+0000","last_clean":"2023-10-17T01:13:02.635096+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:13:02.635096+0000","last_undegraded":"2023-10-17T01:13:02.635096+0000","last_fullsized":"2023-10-17T01:13:02.635096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940636+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:13:01.940636+0000","last_peered":"2023-10-17T01:13:01.940636+0000","last_clean":"2023-10-17T01:13:01.940636+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:13:01.940636+0000","last_undegraded":"2023-10-17T01:13:01.940636+0000","last_fullsized":"2023-10-17T01:13:01.940636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347859+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:13:02.347859+0000","last_peered":"2023-10-17T01:13:02.347859+0000","last_clean":"2023-10-17T01:13:02.347859+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:13:02.347859+0000","last_undegraded":"2023-10-17T01:13:02.347859+0000","last_fullsized":"2023-10-17T01:13:02.347859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.073105+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:13:01.073105+0000","last_peered":"2023-10-17T01:13:01.073105+0000","last_clean":"2023-10-17T01:13:01.073105+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:13:01.073105+0000","last_undegraded":"2023-10-17T01:13:01.073105+0000","last_fullsized":"2023-10-17T01:13:01.073105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430997+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:13:01.430997+0000","last_peered":"2023-10-17T01:13:01.430997+0000","last_clean":"2023-10-17T01:13:01.430997+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:13:01.430997+0000","last_undegraded":"2023-10-17T01:13:01.430997+0000","last_fullsized":"2023-10-17T01:13:01.430997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033393+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:13:02.033393+0000","last_peered":"2023-10-17T01:13:02.033393+0000","last_clean":"2023-10-17T01:13:02.033393+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:13:02.033393+0000","last_undegraded":"2023-10-17T01:13:02.033393+0000","last_fullsized":"2023-10-17T01:13:02.033393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033373+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:13:02.033373+0000","last_peered":"2023-10-17T01:13:02.033373+0000","last_clean":"2023-10-17T01:13:02.033373+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:13:02.033373+0000","last_undegraded":"2023-10-17T01:13:02.033373+0000","last_fullsized":"2023-10-17T01:13:02.033373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634596+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:13:02.634596+0000","last_peered":"2023-10-17T01:13:02.634596+0000","last_clean":"2023-10-17T01:13:02.634596+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:13:02.634596+0000","last_undegraded":"2023-10-17T01:13:02.634596+0000","last_fullsized":"2023-10-17T01:13:02.634596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849181+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:13:02.849181+0000","last_peered":"2023-10-17T01:13:02.849181+0000","last_clean":"2023-10-17T01:13:02.849181+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:13:02.849181+0000","last_undegraded":"2023-10-17T01:13:02.849181+0000","last_fullsized":"2023-10-17T01:13:02.849181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.068097+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:13:01.068097+0000","last_peered":"2023-10-17T01:13:01.068097+0000","last_clean":"2023-10-17T01:13:01.068097+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:13:01.068097+0000","last_undegraded":"2023-10-17T01:13:01.068097+0000","last_fullsized":"2023-10-17T01:13:01.068097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047824+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:13:02.047824+0000","last_peered":"2023-10-17T01:13:02.047824+0000","last_clean":"2023-10-17T01:13:02.047824+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:13:02.047824+0000","last_undegraded":"2023-10-17T01:13:02.047824+0000","last_fullsized":"2023-10-17T01:13:02.047824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.438783+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:13:05.438783+0000","last_peered":"2023-10-17T01:13:05.438783+0000","last_clean":"2023-10-17T01:13:05.438783+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:13:05.438783+0000","last_undegraded":"2023-10-17T01:13:05.438783+0000","last_fullsized":"2023-10-17T01:13:05.438783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630743+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:13:02.630743+0000","last_peered":"2023-10-17T01:13:02.630743+0000","last_clean":"2023-10-17T01:13:02.630743+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:13:02.630743+0000","last_undegraded":"2023-10-17T01:13:02.630743+0000","last_fullsized":"2023-10-17T01:13:02.630743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630839+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:02.630839+0000","last_peered":"2023-10-17T01:13:02.630839+0000","last_clean":"2023-10-17T01:13:02.630839+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:02.630839+0000","last_undegraded":"2023-10-17T01:13:02.630839+0000","last_fullsized":"2023-10-17T01:13:02.630839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033491+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:13:02.033491+0000","last_peered":"2023-10-17T01:13:02.033491+0000","last_clean":"2023-10-17T01:13:02.033491+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:13:02.033491+0000","last_undegraded":"2023-10-17T01:13:02.033491+0000","last_fullsized":"2023-10-17T01:13:02.033491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848099+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:02.848099+0000","last_peered":"2023-10-17T01:13:02.848099+0000","last_clean":"2023-10-17T01:13:02.848099+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:02.848099+0000","last_undegraded":"2023-10-17T01:13:02.848099+0000","last_fullsized":"2023-10-17T01:13:02.848099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634755+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:13:02.634755+0000","last_peered":"2023-10-17T01:13:02.634755+0000","last_clean":"2023-10-17T01:13:02.634755+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:13:02.634755+0000","last_undegraded":"2023-10-17T01:13:02.634755+0000","last_fullsized":"2023-10-17T01:13:02.634755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847900+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:02.847900+0000","last_peered":"2023-10-17T01:13:02.847900+0000","last_clean":"2023-10-17T01:13:02.847900+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:02.847900+0000","last_undegraded":"2023-10-17T01:13:02.847900+0000","last_fullsized":"2023-10-17T01:13:02.847900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065218+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:13:01.065218+0000","last_peered":"2023-10-17T01:13:01.065218+0000","last_clean":"2023-10-17T01:13:01.065218+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:13:01.065218+0000","last_undegraded":"2023-10-17T01:13:01.065218+0000","last_fullsized":"2023-10-17T01:13:01.065218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424752+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:13:04.424752+0000","last_peered":"2023-10-17T01:13:04.424752+0000","last_clean":"2023-10-17T01:13:04.424752+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:13:04.424752+0000","last_undegraded":"2023-10-17T01:13:04.424752+0000","last_fullsized":"2023-10-17T01:13:04.424752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847600+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:13:02.847600+0000","last_peered":"2023-10-17T01:13:02.847600+0000","last_clean":"2023-10-17T01:13:02.847600+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:13:02.847600+0000","last_undegraded":"2023-10-17T01:13:02.847600+0000","last_fullsized":"2023-10-17T01:13:02.847600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:06.449131+0000","last_change":"2023-10-17T01:13:00.490073+0000","last_active":"2023-10-17T01:13:06.449131+0000","last_peered":"2023-10-17T01:13:06.449131+0000","last_clean":"2023-10-17T01:13:06.449131+0000","last_became_active":"2023-10-17T01:13:00.489744+0000","last_became_peered":"2023-10-17T01:13:00.489744+0000","last_unstale":"2023-10-17T01:13:06.449131+0000","last_undegraded":"2023-10-17T01:13:06.449131+0000","last_fullsized":"2023-10-17T01:13:06.449131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:10:35.405750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065298+0000","last_change":"2023-10-17T01:13:01.062844+0000","last_active":"2023-10-17T01:13:01.065298+0000","last_peered":"2023-10-17T01:13:01.065298+0000","last_clean":"2023-10-17T01:13:01.065298+0000","last_became_active":"2023-10-17T01:13:01.062570+0000","last_became_peered":"2023-10-17T01:13:01.062570+0000","last_unstale":"2023-10-17T01:13:01.065298+0000","last_undegraded":"2023-10-17T01:13:01.065298+0000","last_fullsized":"2023-10-17T01:13:01.065298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:30:51.242720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.439391+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:05.439391+0000","last_peered":"2023-10-17T01:13:05.439391+0000","last_clean":"2023-10-17T01:13:05.439391+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:05.439391+0000","last_undegraded":"2023-10-17T01:13:05.439391+0000","last_fullsized":"2023-10-17T01:13:05.439391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630287+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:13:02.630287+0000","last_peered":"2023-10-17T01:13:02.630287+0000","last_clean":"2023-10-17T01:13:02.630287+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:13:02.630287+0000","last_undegraded":"2023-10-17T01:13:02.630287+0000","last_fullsized":"2023-10-17T01:13:02.630287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848557+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:02.848557+0000","last_peered":"2023-10-17T01:13:02.848557+0000","last_clean":"2023-10-17T01:13:02.848557+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:02.848557+0000","last_undegraded":"2023-10-17T01:13:02.848557+0000","last_fullsized":"2023-10-17T01:13:02.848557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.071404+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:13:01.071404+0000","last_peered":"2023-10-17T01:13:01.071404+0000","last_clean":"2023-10-17T01:13:01.071404+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:13:01.071404+0000","last_undegraded":"2023-10-17T01:13:01.071404+0000","last_fullsized":"2023-10-17T01:13:01.071404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424451+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:13:04.424451+0000","last_peered":"2023-10-17T01:13:04.424451+0000","last_clean":"2023-10-17T01:13:04.424451+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:13:04.424451+0000","last_undegraded":"2023-10-17T01:13:04.424451+0000","last_fullsized":"2023-10-17T01:13:04.424451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.635032+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:13:02.635032+0000","last_peered":"2023-10-17T01:13:02.635032+0000","last_clean":"2023-10-17T01:13:02.635032+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:13:02.635032+0000","last_undegraded":"2023-10-17T01:13:02.635032+0000","last_fullsized":"2023-10-17T01:13:02.635032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347855+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:02.347855+0000","last_peered":"2023-10-17T01:13:02.347855+0000","last_clean":"2023-10-17T01:13:02.347855+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:02.347855+0000","last_undegraded":"2023-10-17T01:13:02.347855+0000","last_fullsized":"2023-10-17T01:13:02.347855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033477+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:13:02.033477+0000","last_peered":"2023-10-17T01:13:02.033477+0000","last_clean":"2023-10-17T01:13:02.033477+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:13:02.033477+0000","last_undegraded":"2023-10-17T01:13:02.033477+0000","last_fullsized":"2023-10-17T01:13:02.033477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849181+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:02.849181+0000","last_peered":"2023-10-17T01:13:02.849181+0000","last_clean":"2023-10-17T01:13:02.849181+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:02.849181+0000","last_undegraded":"2023-10-17T01:13:02.849181+0000","last_fullsized":"2023-10-17T01:13:02.849181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634878+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:13:02.634878+0000","last_peered":"2023-10-17T01:13:02.634878+0000","last_clean":"2023-10-17T01:13:02.634878+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:13:02.634878+0000","last_undegraded":"2023-10-17T01:13:02.634878+0000","last_fullsized":"2023-10-17T01:13:02.634878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.423127+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:13:04.423127+0000","last_peered":"2023-10-17T01:13:04.423127+0000","last_clean":"2023-10-17T01:13:04.423127+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:13:04.423127+0000","last_undegraded":"2023-10-17T01:13:04.423127+0000","last_fullsized":"2023-10-17T01:13:04.423127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847842+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:13:02.847842+0000","last_peered":"2023-10-17T01:13:02.847842+0000","last_clean":"2023-10-17T01:13:02.847842+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:13:02.847842+0000","last_undegraded":"2023-10-17T01:13:02.847842+0000","last_fullsized":"2023-10-17T01:13:02.847842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.459288+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:07.459288+0000","last_peered":"2023-10-17T01:13:07.459288+0000","last_clean":"2023-10-17T01:13:07.459288+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:07.459288+0000","last_undegraded":"2023-10-17T01:13:07.459288+0000","last_fullsized":"2023-10-17T01:13:07.459288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430960+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:13:01.430960+0000","last_peered":"2023-10-17T01:13:01.430960+0000","last_clean":"2023-10-17T01:13:01.430960+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:13:01.430960+0000","last_undegraded":"2023-10-17T01:13:01.430960+0000","last_fullsized":"2023-10-17T01:13:01.430960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.423024+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:04.423024+0000","last_peered":"2023-10-17T01:13:04.423024+0000","last_clean":"2023-10-17T01:13:04.423024+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:04.423024+0000","last_undegraded":"2023-10-17T01:13:04.423024+0000","last_fullsized":"2023-10-17T01:13:04.423024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347776+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:13:02.347776+0000","last_peered":"2023-10-17T01:13:02.347776+0000","last_clean":"2023-10-17T01:13:02.347776+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:13:02.347776+0000","last_undegraded":"2023-10-17T01:13:02.347776+0000","last_fullsized":"2023-10-17T01:13:02.347776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.436200+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:13:05.436200+0000","last_peered":"2023-10-17T01:13:05.436200+0000","last_clean":"2023-10-17T01:13:05.436200+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:13:05.436200+0000","last_undegraded":"2023-10-17T01:13:05.436200+0000","last_fullsized":"2023-10-17T01:13:05.436200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347516+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:13:02.347516+0000","last_peered":"2023-10-17T01:13:02.347516+0000","last_clean":"2023-10-17T01:13:02.347516+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:13:02.347516+0000","last_undegraded":"2023-10-17T01:13:02.347516+0000","last_fullsized":"2023-10-17T01:13:02.347516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'10","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.421341+0000","last_change":"2023-10-17T01:13:00.500551+0000","last_active":"2023-10-17T01:13:04.421341+0000","last_peered":"2023-10-17T01:13:04.421341+0000","last_clean":"2023-10-17T01:13:04.421341+0000","last_became_active":"2023-10-17T01:13:00.500211+0000","last_became_peered":"2023-10-17T01:13:00.500211+0000","last_unstale":"2023-10-17T01:13:04.421341+0000","last_undegraded":"2023-10-17T01:13:04.421341+0000","last_fullsized":"2023-10-17T01:13:04.421341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:21:14.217786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.068374+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:13:01.068374+0000","last_peered":"2023-10-17T01:13:01.068374+0000","last_clean":"2023-10-17T01:13:01.068374+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:13:01.068374+0000","last_undegraded":"2023-10-17T01:13:01.068374+0000","last_fullsized":"2023-10-17T01:13:01.068374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847871+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:02.847871+0000","last_peered":"2023-10-17T01:13:02.847871+0000","last_clean":"2023-10-17T01:13:02.847871+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:02.847871+0000","last_undegraded":"2023-10-17T01:13:02.847871+0000","last_fullsized":"2023-10-17T01:13:02.847871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848843+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:13:02.848843+0000","last_peered":"2023-10-17T01:13:02.848843+0000","last_clean":"2023-10-17T01:13:02.848843+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:13:02.848843+0000","last_undegraded":"2023-10-17T01:13:02.848843+0000","last_fullsized":"2023-10-17T01:13:02.848843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.046704+0000","last_change":"2023-10-17T01:13:00.491886+0000","last_active":"2023-10-17T01:13:01.046704+0000","last_peered":"2023-10-17T01:13:01.046704+0000","last_clean":"2023-10-17T01:13:01.046704+0000","last_became_active":"2023-10-17T01:13:00.491435+0000","last_became_peered":"2023-10-17T01:13:00.491435+0000","last_unstale":"2023-10-17T01:13:01.046704+0000","last_undegraded":"2023-10-17T01:13:01.046704+0000","last_fullsized":"2023-10-17T01:13:01.046704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:37:29.052376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940625+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:13:01.940625+0000","last_peered":"2023-10-17T01:13:01.940625+0000","last_clean":"2023-10-17T01:13:01.940625+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:13:01.940625+0000","last_undegraded":"2023-10-17T01:13:01.940625+0000","last_fullsized":"2023-10-17T01:13:01.940625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347703+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:13:02.347703+0000","last_peered":"2023-10-17T01:13:02.347703+0000","last_clean":"2023-10-17T01:13:02.347703+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:13:02.347703+0000","last_undegraded":"2023-10-17T01:13:02.347703+0000","last_fullsized":"2023-10-17T01:13:02.347703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940599+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:13:01.940599+0000","last_peered":"2023-10-17T01:13:01.940599+0000","last_clean":"2023-10-17T01:13:01.940599+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:13:01.940599+0000","last_undegraded":"2023-10-17T01:13:01.940599+0000","last_fullsized":"2023-10-17T01:13:01.940599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065060+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:13:01.065060+0000","last_peered":"2023-10-17T01:13:01.065060+0000","last_clean":"2023-10-17T01:13:01.065060+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:13:01.065060+0000","last_undegraded":"2023-10-17T01:13:01.065060+0000","last_fullsized":"2023-10-17T01:13:01.065060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.441014+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:13:05.441014+0000","last_peered":"2023-10-17T01:13:05.441014+0000","last_clean":"2023-10-17T01:13:05.441014+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:13:05.441014+0000","last_undegraded":"2023-10-17T01:13:05.441014+0000","last_fullsized":"2023-10-17T01:13:05.441014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630580+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:13:02.630580+0000","last_peered":"2023-10-17T01:13:02.630580+0000","last_clean":"2023-10-17T01:13:02.630580+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:13:02.630580+0000","last_undegraded":"2023-10-17T01:13:02.630580+0000","last_fullsized":"2023-10-17T01:13:02.630580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424907+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:04.424907+0000","last_peered":"2023-10-17T01:13:04.424907+0000","last_clean":"2023-10-17T01:13:04.424907+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:04.424907+0000","last_undegraded":"2023-10-17T01:13:04.424907+0000","last_fullsized":"2023-10-17T01:13:04.424907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.057643+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:01.057643+0000","last_peered":"2023-10-17T01:13:01.057643+0000","last_clean":"2023-10-17T01:13:01.057643+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:01.057643+0000","last_undegraded":"2023-10-17T01:13:01.057643+0000","last_fullsized":"2023-10-17T01:13:01.057643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430869+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:13:01.430869+0000","last_peered":"2023-10-17T01:13:01.430869+0000","last_clean":"2023-10-17T01:13:01.430869+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:13:01.430869+0000","last_undegraded":"2023-10-17T01:13:01.430869+0000","last_fullsized":"2023-10-17T01:13:01.430869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347643+0000","last_change":"2023-10-17T01:13:00.488729+0000","last_active":"2023-10-17T01:13:02.347643+0000","last_peered":"2023-10-17T01:13:02.347643+0000","last_clean":"2023-10-17T01:13:02.347643+0000","last_became_active":"2023-10-17T01:13:00.487735+0000","last_became_peered":"2023-10-17T01:13:00.487735+0000","last_unstale":"2023-10-17T01:13:02.347643+0000","last_undegraded":"2023-10-17T01:13:02.347643+0000","last_fullsized":"2023-10-17T01:13:02.347643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:53.287559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.421406+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:04.421406+0000","last_peered":"2023-10-17T01:13:04.421406+0000","last_clean":"2023-10-17T01:13:04.421406+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:04.421406+0000","last_undegraded":"2023-10-17T01:13:04.421406+0000","last_fullsized":"2023-10-17T01:13:04.421406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.431026+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:13:01.431026+0000","last_peered":"2023-10-17T01:13:01.431026+0000","last_clean":"2023-10-17T01:13:01.431026+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:13:01.431026+0000","last_undegraded":"2023-10-17T01:13:01.431026+0000","last_fullsized":"2023-10-17T01:13:01.431026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.440581+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:13:05.440581+0000","last_peered":"2023-10-17T01:13:05.440581+0000","last_clean":"2023-10-17T01:13:05.440581+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:13:05.440581+0000","last_undegraded":"2023-10-17T01:13:05.440581+0000","last_fullsized":"2023-10-17T01:13:05.440581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630805+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:13:02.630805+0000","last_peered":"2023-10-17T01:13:02.630805+0000","last_clean":"2023-10-17T01:13:02.630805+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:13:02.630805+0000","last_undegraded":"2023-10-17T01:13:02.630805+0000","last_fullsized":"2023-10-17T01:13:02.630805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.422918+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:04.422918+0000","last_peered":"2023-10-17T01:13:04.422918+0000","last_clean":"2023-10-17T01:13:04.422918+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:04.422918+0000","last_undegraded":"2023-10-17T01:13:04.422918+0000","last_fullsized":"2023-10-17T01:13:04.422918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033457+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:13:02.033457+0000","last_peered":"2023-10-17T01:13:02.033457+0000","last_clean":"2023-10-17T01:13:02.033457+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:13:02.033457+0000","last_undegraded":"2023-10-17T01:13:02.033457+0000","last_fullsized":"2023-10-17T01:13:02.033457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634655+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:02.634655+0000","last_peered":"2023-10-17T01:13:02.634655+0000","last_clean":"2023-10-17T01:13:02.634655+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:02.634655+0000","last_undegraded":"2023-10-17T01:13:02.634655+0000","last_fullsized":"2023-10-17T01:13:02.634655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848108+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:13:02.848108+0000","last_peered":"2023-10-17T01:13:02.848108+0000","last_clean":"2023-10-17T01:13:02.848108+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:13:02.848108+0000","last_undegraded":"2023-10-17T01:13:02.848108+0000","last_fullsized":"2023-10-17T01:13:02.848108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630284+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:02.630284+0000","last_peered":"2023-10-17T01:13:02.630284+0000","last_clean":"2023-10-17T01:13:02.630284+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:02.630284+0000","last_undegraded":"2023-10-17T01:13:02.630284+0000","last_fullsized":"2023-10-17T01:13:02.630284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848588+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:13:02.848588+0000","last_peered":"2023-10-17T01:13:02.848588+0000","last_clean":"2023-10-17T01:13:02.848588+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:13:02.848588+0000","last_undegraded":"2023-10-17T01:13:02.848588+0000","last_fullsized":"2023-10-17T01:13:02.848588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:06.448727+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:06.448727+0000","last_peered":"2023-10-17T01:13:06.448727+0000","last_clean":"2023-10-17T01:13:06.448727+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:06.448727+0000","last_undegraded":"2023-10-17T01:13:06.448727+0000","last_fullsized":"2023-10-17T01:13:06.448727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630439+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:13:02.630439+0000","last_peered":"2023-10-17T01:13:02.630439+0000","last_clean":"2023-10-17T01:13:02.630439+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:13:02.630439+0000","last_undegraded":"2023-10-17T01:13:02.630439+0000","last_fullsized":"2023-10-17T01:13:02.630439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.423613+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:04.423613+0000","last_peered":"2023-10-17T01:13:04.423613+0000","last_clean":"2023-10-17T01:13:04.423613+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:04.423613+0000","last_undegraded":"2023-10-17T01:13:04.423613+0000","last_fullsized":"2023-10-17T01:13:04.423613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347733+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:13:02.347733+0000","last_peered":"2023-10-17T01:13:02.347733+0000","last_clean":"2023-10-17T01:13:02.347733+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:13:02.347733+0000","last_undegraded":"2023-10-17T01:13:02.347733+0000","last_fullsized":"2023-10-17T01:13:02.347733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:06.449334+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:13:06.449334+0000","last_peered":"2023-10-17T01:13:06.449334+0000","last_clean":"2023-10-17T01:13:06.449334+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:13:06.449334+0000","last_undegraded":"2023-10-17T01:13:06.449334+0000","last_fullsized":"2023-10-17T01:13:06.449334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347959+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:13:02.347959+0000","last_peered":"2023-10-17T01:13:02.347959+0000","last_clean":"2023-10-17T01:13:02.347959+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:13:02.347959+0000","last_undegraded":"2023-10-17T01:13:02.347959+0000","last_fullsized":"2023-10-17T01:13:02.347959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.069637+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:01.069637+0000","last_peered":"2023-10-17T01:13:01.069637+0000","last_clean":"2023-10-17T01:13:01.069637+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:01.069637+0000","last_undegraded":"2023-10-17T01:13:01.069637+0000","last_fullsized":"2023-10-17T01:13:01.069637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347735+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:13:02.347735+0000","last_peered":"2023-10-17T01:13:02.347735+0000","last_clean":"2023-10-17T01:13:02.347735+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:13:02.347735+0000","last_undegraded":"2023-10-17T01:13:02.347735+0000","last_fullsized":"2023-10-17T01:13:02.347735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.431065+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:01.431065+0000","last_peered":"2023-10-17T01:13:01.431065+0000","last_clean":"2023-10-17T01:13:01.431065+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:01.431065+0000","last_undegraded":"2023-10-17T01:13:01.431065+0000","last_fullsized":"2023-10-17T01:13:01.431065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347920+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:13:02.347920+0000","last_peered":"2023-10-17T01:13:02.347920+0000","last_clean":"2023-10-17T01:13:02.347920+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:13:02.347920+0000","last_undegraded":"2023-10-17T01:13:02.347920+0000","last_fullsized":"2023-10-17T01:13:02.347920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848662+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:13:02.848662+0000","last_peered":"2023-10-17T01:13:02.848662+0000","last_clean":"2023-10-17T01:13:02.848662+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:13:02.848662+0000","last_undegraded":"2023-10-17T01:13:02.848662+0000","last_fullsized":"2023-10-17T01:13:02.848662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047916+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:13:02.047916+0000","last_peered":"2023-10-17T01:13:02.047916+0000","last_clean":"2023-10-17T01:13:02.047916+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:13:02.047916+0000","last_undegraded":"2023-10-17T01:13:02.047916+0000","last_fullsized":"2023-10-17T01:13:02.047916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047635+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:13:02.047635+0000","last_peered":"2023-10-17T01:13:02.047635+0000","last_clean":"2023-10-17T01:13:02.047635+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:13:02.047635+0000","last_undegraded":"2023-10-17T01:13:02.047635+0000","last_fullsized":"2023-10-17T01:13:02.047635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033413+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:13:02.033413+0000","last_peered":"2023-10-17T01:13:02.033413+0000","last_clean":"2023-10-17T01:13:02.033413+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:13:02.033413+0000","last_undegraded":"2023-10-17T01:13:02.033413+0000","last_fullsized":"2023-10-17T01:13:02.033413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.431045+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:13:01.431045+0000","last_peered":"2023-10-17T01:13:01.431045+0000","last_clean":"2023-10-17T01:13:01.431045+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:13:01.431045+0000","last_undegraded":"2023-10-17T01:13:01.431045+0000","last_fullsized":"2023-10-17T01:13:01.431045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849011+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:13:02.849011+0000","last_peered":"2023-10-17T01:13:02.849011+0000","last_clean":"2023-10-17T01:13:02.849011+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:13:02.849011+0000","last_undegraded":"2023-10-17T01:13:02.849011+0000","last_fullsized":"2023-10-17T01:13:02.849011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047951+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:13:02.047951+0000","last_peered":"2023-10-17T01:13:02.047951+0000","last_clean":"2023-10-17T01:13:02.047951+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:13:02.047951+0000","last_undegraded":"2023-10-17T01:13:02.047951+0000","last_fullsized":"2023-10-17T01:13:02.047951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630796+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:13:02.630796+0000","last_peered":"2023-10-17T01:13:02.630796+0000","last_clean":"2023-10-17T01:13:02.630796+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:13:02.630796+0000","last_undegraded":"2023-10-17T01:13:02.630796+0000","last_fullsized":"2023-10-17T01:13:02.630796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065243+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:13:01.065243+0000","last_peered":"2023-10-17T01:13:01.065243+0000","last_clean":"2023-10-17T01:13:01.065243+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:13:01.065243+0000","last_undegraded":"2023-10-17T01:13:01.065243+0000","last_fullsized":"2023-10-17T01:13:01.065243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630228+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:13:02.630228+0000","last_peered":"2023-10-17T01:13:02.630228+0000","last_clean":"2023-10-17T01:13:02.630228+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:13:02.630228+0000","last_undegraded":"2023-10-17T01:13:02.630228+0000","last_fullsized":"2023-10-17T01:13:02.630228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634809+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:13:02.634809+0000","last_peered":"2023-10-17T01:13:02.634809+0000","last_clean":"2023-10-17T01:13:02.634809+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:13:02.634809+0000","last_undegraded":"2023-10-17T01:13:02.634809+0000","last_fullsized":"2023-10-17T01:13:02.634809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430980+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:13:01.430980+0000","last_peered":"2023-10-17T01:13:01.430980+0000","last_clean":"2023-10-17T01:13:01.430980+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:13:01.430980+0000","last_undegraded":"2023-10-17T01:13:01.430980+0000","last_fullsized":"2023-10-17T01:13:01.430980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848866+0000","last_change":"2023-10-17T01:13:01.066367+0000","last_active":"2023-10-17T01:13:02.848866+0000","last_peered":"2023-10-17T01:13:02.848866+0000","last_clean":"2023-10-17T01:13:02.848866+0000","last_became_active":"2023-10-17T01:13:01.065975+0000","last_became_peered":"2023-10-17T01:13:01.065975+0000","last_unstale":"2023-10-17T01:13:02.848866+0000","last_undegraded":"2023-10-17T01:13:02.848866+0000","last_fullsized":"2023-10-17T01:13:02.848866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:29.094472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5540,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":32243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":417195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678033,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1019227,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.81699999999999995}]},{"osd":3,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.84499999999999997}]},{"osd":6,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.875},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.83699999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.93100000000000005}]},{"osd":9,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.97699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":426549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.67400000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.751}]},{"osd":7,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.60099999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.879}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.80400000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":426038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.56499999999999995}]},{"osd":1,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.626},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66100000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.73999999999999999}]},{"osd":5,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41299999999999998}]},{"osd":6,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.72699999999999998}]},{"osd":9,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68200000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.70999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":1016384,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76000000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59899999999999998,"15min":0.58599999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.54300000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.79500000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71399999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.72899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.38100000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.46800000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95199999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78300000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.52100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.64600000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Oct 17 01:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.80900000000000005,"15min":0.80200000000000005},"min":{"1min":0.63500000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3899999999999999,"5min":1.532,"15min":1.532},"last":0.85399999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.0580000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":1.417,"15min":1.417},"last":0.97099999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.018}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.91600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":427056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.505},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.53000000000000003},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Oct 17 01:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.45400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.50900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Oct 17 01:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.45200000000000001,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.47099999999999997,"15min":0.45800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.373}]},{"osd":5,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.59599999999999997},"min":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66000000000000003,"15min":0.64100000000000001},"min":{"1min":0.441,"5min":0.432,"15min":0.432},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Oct 17 01:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Oct 17 01:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61799999999999999}]},{"osd":8,"last update":"Tue Oct 17 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78600000000000003}]},{"osd":10,"last update":"Tue Oct 17 01:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":427783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.01,"5min":1.02,"15min":1.02},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54800000000000004,"15min":0.51900000000000002},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68799999999999994}]},{"osd":2,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.63900000000000001,"15min":0.59099999999999997},"min":{"1min":0.499,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.79600000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.313,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89100000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.36899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Oct 17 01:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68100000000000005,"15min":0.64900000000000002},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":0.997,"15min":0.997},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68700000000000006,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":0.91000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3340000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.997}]},{"osd":9,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.83799999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":429324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.52200000000000002},"min":{"1min":0.376,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77800000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.40100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72399999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.749}]},{"osd":3,"last update":"Tue Oct 17 01:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Oct 17 01:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.60599999999999998,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":6.133,"5min":6.133,"15min":6.133},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.66700000000000004,"15min":0.624},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"last":0.621}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.437},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.495},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.81999999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.84299999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.80500000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.2060000000000004,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":425869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.34999999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.42199999999999999,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":1,"last update":"Tue Oct 17 01:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.63800000000000001,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.752}]},{"osd":2,"last update":"Tue Oct 17 01:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65200000000000002,"15min":0.63300000000000001},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99099999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.66300000000000003,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.71499999999999997}]},{"osd":4,"last update":"Tue Oct 17 01:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.58999999999999997},"min":{"1min":0.59699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.81799999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0860000000000001,"5min":1.161,"15min":1.161},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.57299999999999995,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.752,"15min":0.73099999999999998},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.87},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.81899999999999995,"15min":0.80700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0940000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":1.018},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.80300000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.33100000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51600000000000001},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"min":{"1min":0.50900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64500000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.745},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79800000000000004}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.47999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65600000000000003,"15min":0.63900000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.78100000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69499999999999995}]},{"osd":9,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.85199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757287,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":1017561,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73999999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.81100000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.84199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.54100000000000004,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.755}]},{"osd":2,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40500000000000003,"15min":0.38900000000000001},"min":{"1min":0.28399999999999997,"5min":0.223,"15min":0.223},"max":{"1min":0.76600000000000001,"5min":1.24,"15min":1.24},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27100000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.5}]},{"osd":3,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.76500000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Oct 17 01:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.67500000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71899999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82099999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":21199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:13:10.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-17T01:13:10.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 -- 172.21.15.45:0/3566851754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 msgr2=0x7f47981023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 --2- 172.21.15.45:0/3566851754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f47981023a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f478c0099c0 tx=0x7f478c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 -- 172.21.15.45:0/3566851754 shutdown_connections 2023-10-17T01:13:10.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 --2- 172.21.15.45:0/3566851754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47981028e0 0x7f4798109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 --2- 172.21.15.45:0/3566851754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4798100d50 0x7f4798101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 --2- 172.21.15.45:0/3566851754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f47981023a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 -- 172.21.15.45:0/3566851754 >> 172.21.15.45:0/3566851754 conn(0x7f47980fc280 msgr2=0x7f47980fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:10.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 -- 172.21.15.45:0/3566851754 shutdown_connections 2023-10-17T01:13:10.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.690+0000 7f479ddff700 1 -- 172.21.15.45:0/3566851754 wait complete. 2023-10-17T01:13:10.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.691+0000 7f479ddff700 1 Processor -- start 2023-10-17T01:13:10.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 -- start start 2023-10-17T01:13:10.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4798100d50 0x7f4798198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f4798199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47981028e0 0x7f47981a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981a0770 con 0x7f4798101f40 2023-10-17T01:13:10.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981a08e0 con 0x7f4798100d50 2023-10-17T01:13:10.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f479ddff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981a0aa0 con 0x7f47981028e0 2023-10-17T01:13:10.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f4796ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f4798199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f4796ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f4798199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55704/0 (socket says 172.21.15.45:55704) 2023-10-17T01:13:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f4796ffd700 1 -- 172.21.15.45:0/1151638432 learned_addr learned my addr 172.21.15.45:0/1151638432 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f47977fe700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4798100d50 0x7f4798198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.692+0000 7f4797fff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47981028e0 0x7f47981a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4796ffd700 1 -- 172.21.15.45:0/1151638432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47981028e0 msgr2=0x7f47981a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4796ffd700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47981028e0 0x7f47981a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4796ffd700 1 -- 172.21.15.45:0/1151638432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4798100d50 msgr2=0x7f4798198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4796ffd700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4798100d50 0x7f4798198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4796ffd700 1 -- 172.21.15.45:0/1151638432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0095f0 con 0x7f4798101f40 2023-10-17T01:13:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f47977fe700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4798100d50 0x7f4798198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4796ffd700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f4798199170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f478c009fd0 tx=0x7f478c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c01d070 con 0x7f4798101f40 2023-10-17T01:13:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478c005030 con 0x7f4798101f40 2023-10-17T01:13:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c004a90 con 0x7f4798101f40 2023-10-17T01:13:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47981a0d20 con 0x7f4798101f40 2023-10-17T01:13:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.693+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47981a10f0 con 0x7f4798101f40 2023-10-17T01:13:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.694+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479810c560 con 0x7f4798101f40 2023-10-17T01:13:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.696+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f478c0051a0 con 0x7f4798101f40 2023-10-17T01:13:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.698+0000 7f4794ff9700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47800a0bf0 0x7f47800a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.698+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f478c0ccfe0 con 0x7f4798101f40 2023-10-17T01:13:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.698+0000 7f47977fe700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47800a0bf0 0x7f47800a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.698+0000 7f47977fe700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47800a0bf0 0x7f47800a30b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f479819a100 tx=0x7f4788009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.701+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f478c0969b0 con 0x7f4798101f40 2023-10-17T01:13:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:10 smithi045 ceph-mon[98474]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:10 smithi045 ceph-mon[98474]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-17T01:13:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:10 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1730384086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:13:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:10 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/480878845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:10 smithi060 ceph-mon[134136]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:10 smithi060 ceph-mon[134136]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-17T01:13:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:10 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1730384086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:13:10.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:10 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/480878845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:10 smithi181 ceph-mon[134520]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:13:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:10 smithi181 ceph-mon[134520]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-17T01:13:10.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:10 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1730384086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:13:10.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:10 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/480878845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:10.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.910+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4798108890 con 0x7f47800a0bf0 2023-10-17T01:13:10.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.921+0000 7f4794ff9700 1 -- 172.21.15.45:0/1151638432 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336163 (secure 0 0 0) 0x7f4798108890 con 0x7f47800a0bf0 2023-10-17T01:13:10.926 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:10.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.925+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47800a0bf0 msgr2=0x7f47800a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47800a0bf0 0x7f47800a30b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f479819a100 tx=0x7f4788009260 comp rx=0 tx=0).stop 2023-10-17T01:13:10.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 msgr2=0x7f4798199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:10.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f4798199170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f478c009fd0 tx=0x7f478c009580 comp rx=0 tx=0).stop 2023-10-17T01:13:10.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 shutdown_connections 2023-10-17T01:13:10.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47800a0bf0 0x7f47800a30b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47981028e0 0x7f47981a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4798100d50 0x7f4798198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 --2- 172.21.15.45:0/1151638432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4798101f40 0x7f4798199170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 >> 172.21.15.45:0/1151638432 conn(0x7f47980fc280 msgr2=0x7f47980fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 shutdown_connections 2023-10-17T01:13:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:10.926+0000 7f479ddff700 1 -- 172.21.15.45:0/1151638432 wait complete. 2023-10-17T01:13:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:13:10.946 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-10-17T01:13:10.433206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597445,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":349,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334292,"kb_used_data":10868,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607532,"statfs":{"total":1151940427776,"available":1151598112768,"internally_reserved":0,"allocated":11128832,"data_stored":6917689,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7077,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":168,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.391857"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849116+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:13:02.849116+0000","last_peered":"2023-10-17T01:13:02.849116+0000","last_clean":"2023-10-17T01:13:02.849116+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:13:02.849116+0000","last_undegraded":"2023-10-17T01:13:02.849116+0000","last_fullsized":"2023-10-17T01:13:02.849116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033441+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:13:02.033441+0000","last_peered":"2023-10-17T01:13:02.033441+0000","last_clean":"2023-10-17T01:13:02.033441+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:13:02.033441+0000","last_undegraded":"2023-10-17T01:13:02.033441+0000","last_fullsized":"2023-10-17T01:13:02.033441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033424+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:13:02.033424+0000","last_peered":"2023-10-17T01:13:02.033424+0000","last_clean":"2023-10-17T01:13:02.033424+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:13:02.033424+0000","last_undegraded":"2023-10-17T01:13:02.033424+0000","last_fullsized":"2023-10-17T01:13:02.033424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.073086+0000","last_change":"2023-10-17T01:13:01.071138+0000","last_active":"2023-10-17T01:13:01.073086+0000","last_peered":"2023-10-17T01:13:01.073086+0000","last_clean":"2023-10-17T01:13:01.073086+0000","last_became_active":"2023-10-17T01:13:01.070813+0000","last_became_peered":"2023-10-17T01:13:01.070813+0000","last_unstale":"2023-10-17T01:13:01.073086+0000","last_undegraded":"2023-10-17T01:13:01.073086+0000","last_fullsized":"2023-10-17T01:13:01.073086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:13:32.387439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847607+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:13:02.847607+0000","last_peered":"2023-10-17T01:13:02.847607+0000","last_clean":"2023-10-17T01:13:02.847607+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:13:02.847607+0000","last_undegraded":"2023-10-17T01:13:02.847607+0000","last_fullsized":"2023-10-17T01:13:02.847607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940585+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:13:01.940585+0000","last_peered":"2023-10-17T01:13:01.940585+0000","last_clean":"2023-10-17T01:13:01.940585+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:13:01.940585+0000","last_undegraded":"2023-10-17T01:13:01.940585+0000","last_fullsized":"2023-10-17T01:13:01.940585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347542+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:13:02.347542+0000","last_peered":"2023-10-17T01:13:02.347542+0000","last_clean":"2023-10-17T01:13:02.347542+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:13:02.347542+0000","last_undegraded":"2023-10-17T01:13:02.347542+0000","last_fullsized":"2023-10-17T01:13:02.347542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630206+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:13:02.630206+0000","last_peered":"2023-10-17T01:13:02.630206+0000","last_clean":"2023-10-17T01:13:02.630206+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:13:02.630206+0000","last_undegraded":"2023-10-17T01:13:02.630206+0000","last_fullsized":"2023-10-17T01:13:02.630206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.635096+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:13:02.635096+0000","last_peered":"2023-10-17T01:13:02.635096+0000","last_clean":"2023-10-17T01:13:02.635096+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:13:02.635096+0000","last_undegraded":"2023-10-17T01:13:02.635096+0000","last_fullsized":"2023-10-17T01:13:02.635096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940636+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:13:01.940636+0000","last_peered":"2023-10-17T01:13:01.940636+0000","last_clean":"2023-10-17T01:13:01.940636+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:13:01.940636+0000","last_undegraded":"2023-10-17T01:13:01.940636+0000","last_fullsized":"2023-10-17T01:13:01.940636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347859+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:13:02.347859+0000","last_peered":"2023-10-17T01:13:02.347859+0000","last_clean":"2023-10-17T01:13:02.347859+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:13:02.347859+0000","last_undegraded":"2023-10-17T01:13:02.347859+0000","last_fullsized":"2023-10-17T01:13:02.347859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.073105+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:13:01.073105+0000","last_peered":"2023-10-17T01:13:01.073105+0000","last_clean":"2023-10-17T01:13:01.073105+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:13:01.073105+0000","last_undegraded":"2023-10-17T01:13:01.073105+0000","last_fullsized":"2023-10-17T01:13:01.073105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430997+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:13:01.430997+0000","last_peered":"2023-10-17T01:13:01.430997+0000","last_clean":"2023-10-17T01:13:01.430997+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:13:01.430997+0000","last_undegraded":"2023-10-17T01:13:01.430997+0000","last_fullsized":"2023-10-17T01:13:01.430997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033393+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:13:02.033393+0000","last_peered":"2023-10-17T01:13:02.033393+0000","last_clean":"2023-10-17T01:13:02.033393+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:13:02.033393+0000","last_undegraded":"2023-10-17T01:13:02.033393+0000","last_fullsized":"2023-10-17T01:13:02.033393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033373+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:13:02.033373+0000","last_peered":"2023-10-17T01:13:02.033373+0000","last_clean":"2023-10-17T01:13:02.033373+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:13:02.033373+0000","last_undegraded":"2023-10-17T01:13:02.033373+0000","last_fullsized":"2023-10-17T01:13:02.033373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634596+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:13:02.634596+0000","last_peered":"2023-10-17T01:13:02.634596+0000","last_clean":"2023-10-17T01:13:02.634596+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:13:02.634596+0000","last_undegraded":"2023-10-17T01:13:02.634596+0000","last_fullsized":"2023-10-17T01:13:02.634596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849181+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:13:02.849181+0000","last_peered":"2023-10-17T01:13:02.849181+0000","last_clean":"2023-10-17T01:13:02.849181+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:13:02.849181+0000","last_undegraded":"2023-10-17T01:13:02.849181+0000","last_fullsized":"2023-10-17T01:13:02.849181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.068097+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:13:01.068097+0000","last_peered":"2023-10-17T01:13:01.068097+0000","last_clean":"2023-10-17T01:13:01.068097+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:13:01.068097+0000","last_undegraded":"2023-10-17T01:13:01.068097+0000","last_fullsized":"2023-10-17T01:13:01.068097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047824+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:13:02.047824+0000","last_peered":"2023-10-17T01:13:02.047824+0000","last_clean":"2023-10-17T01:13:02.047824+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:13:02.047824+0000","last_undegraded":"2023-10-17T01:13:02.047824+0000","last_fullsized":"2023-10-17T01:13:02.047824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.438783+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:13:05.438783+0000","last_peered":"2023-10-17T01:13:05.438783+0000","last_clean":"2023-10-17T01:13:05.438783+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:13:05.438783+0000","last_undegraded":"2023-10-17T01:13:05.438783+0000","last_fullsized":"2023-10-17T01:13:05.438783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630743+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:13:02.630743+0000","last_peered":"2023-10-17T01:13:02.630743+0000","last_clean":"2023-10-17T01:13:02.630743+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:13:02.630743+0000","last_undegraded":"2023-10-17T01:13:02.630743+0000","last_fullsized":"2023-10-17T01:13:02.630743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:03.652454+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:03.652454+0000","last_peered":"2023-10-17T01:13:03.652454+0000","last_clean":"2023-10-17T01:13:03.652454+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:03.652454+0000","last_undegraded":"2023-10-17T01:13:03.652454+0000","last_fullsized":"2023-10-17T01:13:03.652454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033491+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:13:02.033491+0000","last_peered":"2023-10-17T01:13:02.033491+0000","last_clean":"2023-10-17T01:13:02.033491+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:13:02.033491+0000","last_undegraded":"2023-10-17T01:13:02.033491+0000","last_fullsized":"2023-10-17T01:13:02.033491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'11","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.460575+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:07.460575+0000","last_peered":"2023-10-17T01:13:07.460575+0000","last_clean":"2023-10-17T01:13:07.460575+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:07.460575+0000","last_undegraded":"2023-10-17T01:13:07.460575+0000","last_fullsized":"2023-10-17T01:13:07.460575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634755+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:13:02.634755+0000","last_peered":"2023-10-17T01:13:02.634755+0000","last_clean":"2023-10-17T01:13:02.634755+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:13:02.634755+0000","last_undegraded":"2023-10-17T01:13:02.634755+0000","last_fullsized":"2023-10-17T01:13:02.634755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.459343+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:07.459343+0000","last_peered":"2023-10-17T01:13:07.459343+0000","last_clean":"2023-10-17T01:13:07.459343+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:07.459343+0000","last_undegraded":"2023-10-17T01:13:07.459343+0000","last_fullsized":"2023-10-17T01:13:07.459343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065218+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:13:01.065218+0000","last_peered":"2023-10-17T01:13:01.065218+0000","last_clean":"2023-10-17T01:13:01.065218+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:13:01.065218+0000","last_undegraded":"2023-10-17T01:13:01.065218+0000","last_fullsized":"2023-10-17T01:13:01.065218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424752+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:13:04.424752+0000","last_peered":"2023-10-17T01:13:04.424752+0000","last_clean":"2023-10-17T01:13:04.424752+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:13:04.424752+0000","last_undegraded":"2023-10-17T01:13:04.424752+0000","last_fullsized":"2023-10-17T01:13:04.424752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847600+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:13:02.847600+0000","last_peered":"2023-10-17T01:13:02.847600+0000","last_clean":"2023-10-17T01:13:02.847600+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:13:02.847600+0000","last_undegraded":"2023-10-17T01:13:02.847600+0000","last_fullsized":"2023-10-17T01:13:02.847600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:06.449131+0000","last_change":"2023-10-17T01:13:00.490073+0000","last_active":"2023-10-17T01:13:06.449131+0000","last_peered":"2023-10-17T01:13:06.449131+0000","last_clean":"2023-10-17T01:13:06.449131+0000","last_became_active":"2023-10-17T01:13:00.489744+0000","last_became_peered":"2023-10-17T01:13:00.489744+0000","last_unstale":"2023-10-17T01:13:06.449131+0000","last_undegraded":"2023-10-17T01:13:06.449131+0000","last_fullsized":"2023-10-17T01:13:06.449131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:10:35.405750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065298+0000","last_change":"2023-10-17T01:13:01.062844+0000","last_active":"2023-10-17T01:13:01.065298+0000","last_peered":"2023-10-17T01:13:01.065298+0000","last_clean":"2023-10-17T01:13:01.065298+0000","last_became_active":"2023-10-17T01:13:01.062570+0000","last_became_peered":"2023-10-17T01:13:01.062570+0000","last_unstale":"2023-10-17T01:13:01.065298+0000","last_undegraded":"2023-10-17T01:13:01.065298+0000","last_fullsized":"2023-10-17T01:13:01.065298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:30:51.242720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.439391+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:05.439391+0000","last_peered":"2023-10-17T01:13:05.439391+0000","last_clean":"2023-10-17T01:13:05.439391+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:05.439391+0000","last_undegraded":"2023-10-17T01:13:05.439391+0000","last_fullsized":"2023-10-17T01:13:05.439391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630287+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:13:02.630287+0000","last_peered":"2023-10-17T01:13:02.630287+0000","last_clean":"2023-10-17T01:13:02.630287+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:13:02.630287+0000","last_undegraded":"2023-10-17T01:13:02.630287+0000","last_fullsized":"2023-10-17T01:13:02.630287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848557+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:02.848557+0000","last_peered":"2023-10-17T01:13:02.848557+0000","last_clean":"2023-10-17T01:13:02.848557+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:02.848557+0000","last_undegraded":"2023-10-17T01:13:02.848557+0000","last_fullsized":"2023-10-17T01:13:02.848557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.071404+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:13:01.071404+0000","last_peered":"2023-10-17T01:13:01.071404+0000","last_clean":"2023-10-17T01:13:01.071404+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:13:01.071404+0000","last_undegraded":"2023-10-17T01:13:01.071404+0000","last_fullsized":"2023-10-17T01:13:01.071404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424451+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:13:04.424451+0000","last_peered":"2023-10-17T01:13:04.424451+0000","last_clean":"2023-10-17T01:13:04.424451+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:13:04.424451+0000","last_undegraded":"2023-10-17T01:13:04.424451+0000","last_fullsized":"2023-10-17T01:13:04.424451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.635032+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:13:02.635032+0000","last_peered":"2023-10-17T01:13:02.635032+0000","last_clean":"2023-10-17T01:13:02.635032+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:13:02.635032+0000","last_undegraded":"2023-10-17T01:13:02.635032+0000","last_fullsized":"2023-10-17T01:13:02.635032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.458850+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:07.458850+0000","last_peered":"2023-10-17T01:13:07.458850+0000","last_clean":"2023-10-17T01:13:07.458850+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:07.458850+0000","last_undegraded":"2023-10-17T01:13:07.458850+0000","last_fullsized":"2023-10-17T01:13:07.458850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033477+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:13:02.033477+0000","last_peered":"2023-10-17T01:13:02.033477+0000","last_clean":"2023-10-17T01:13:02.033477+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:13:02.033477+0000","last_undegraded":"2023-10-17T01:13:02.033477+0000","last_fullsized":"2023-10-17T01:13:02.033477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849181+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:02.849181+0000","last_peered":"2023-10-17T01:13:02.849181+0000","last_clean":"2023-10-17T01:13:02.849181+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:02.849181+0000","last_undegraded":"2023-10-17T01:13:02.849181+0000","last_fullsized":"2023-10-17T01:13:02.849181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634878+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:13:02.634878+0000","last_peered":"2023-10-17T01:13:02.634878+0000","last_clean":"2023-10-17T01:13:02.634878+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:13:02.634878+0000","last_undegraded":"2023-10-17T01:13:02.634878+0000","last_fullsized":"2023-10-17T01:13:02.634878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.423127+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:13:04.423127+0000","last_peered":"2023-10-17T01:13:04.423127+0000","last_clean":"2023-10-17T01:13:04.423127+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:13:04.423127+0000","last_undegraded":"2023-10-17T01:13:04.423127+0000","last_fullsized":"2023-10-17T01:13:04.423127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.847842+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:13:02.847842+0000","last_peered":"2023-10-17T01:13:02.847842+0000","last_clean":"2023-10-17T01:13:02.847842+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:13:02.847842+0000","last_undegraded":"2023-10-17T01:13:02.847842+0000","last_fullsized":"2023-10-17T01:13:02.847842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.459288+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:07.459288+0000","last_peered":"2023-10-17T01:13:07.459288+0000","last_clean":"2023-10-17T01:13:07.459288+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:07.459288+0000","last_undegraded":"2023-10-17T01:13:07.459288+0000","last_fullsized":"2023-10-17T01:13:07.459288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430960+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:13:01.430960+0000","last_peered":"2023-10-17T01:13:01.430960+0000","last_clean":"2023-10-17T01:13:01.430960+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:13:01.430960+0000","last_undegraded":"2023-10-17T01:13:01.430960+0000","last_fullsized":"2023-10-17T01:13:01.430960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.423024+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:04.423024+0000","last_peered":"2023-10-17T01:13:04.423024+0000","last_clean":"2023-10-17T01:13:04.423024+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:04.423024+0000","last_undegraded":"2023-10-17T01:13:04.423024+0000","last_fullsized":"2023-10-17T01:13:04.423024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347776+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:13:02.347776+0000","last_peered":"2023-10-17T01:13:02.347776+0000","last_clean":"2023-10-17T01:13:02.347776+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:13:02.347776+0000","last_undegraded":"2023-10-17T01:13:02.347776+0000","last_fullsized":"2023-10-17T01:13:02.347776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.436200+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:13:05.436200+0000","last_peered":"2023-10-17T01:13:05.436200+0000","last_clean":"2023-10-17T01:13:05.436200+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:13:05.436200+0000","last_undegraded":"2023-10-17T01:13:05.436200+0000","last_fullsized":"2023-10-17T01:13:05.436200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347516+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:13:02.347516+0000","last_peered":"2023-10-17T01:13:02.347516+0000","last_clean":"2023-10-17T01:13:02.347516+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:13:02.347516+0000","last_undegraded":"2023-10-17T01:13:02.347516+0000","last_fullsized":"2023-10-17T01:13:02.347516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.458638+0000","last_change":"2023-10-17T01:13:00.500551+0000","last_active":"2023-10-17T01:13:07.458638+0000","last_peered":"2023-10-17T01:13:07.458638+0000","last_clean":"2023-10-17T01:13:07.458638+0000","last_became_active":"2023-10-17T01:13:00.500211+0000","last_became_peered":"2023-10-17T01:13:00.500211+0000","last_unstale":"2023-10-17T01:13:07.458638+0000","last_undegraded":"2023-10-17T01:13:07.458638+0000","last_fullsized":"2023-10-17T01:13:07.458638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:21:14.217786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.068374+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:13:01.068374+0000","last_peered":"2023-10-17T01:13:01.068374+0000","last_clean":"2023-10-17T01:13:01.068374+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:13:01.068374+0000","last_undegraded":"2023-10-17T01:13:01.068374+0000","last_fullsized":"2023-10-17T01:13:01.068374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424617+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:04.424617+0000","last_peered":"2023-10-17T01:13:04.424617+0000","last_clean":"2023-10-17T01:13:04.424617+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:04.424617+0000","last_undegraded":"2023-10-17T01:13:04.424617+0000","last_fullsized":"2023-10-17T01:13:04.424617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848843+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:13:02.848843+0000","last_peered":"2023-10-17T01:13:02.848843+0000","last_clean":"2023-10-17T01:13:02.848843+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:13:02.848843+0000","last_undegraded":"2023-10-17T01:13:02.848843+0000","last_fullsized":"2023-10-17T01:13:02.848843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.046704+0000","last_change":"2023-10-17T01:13:00.491886+0000","last_active":"2023-10-17T01:13:01.046704+0000","last_peered":"2023-10-17T01:13:01.046704+0000","last_clean":"2023-10-17T01:13:01.046704+0000","last_became_active":"2023-10-17T01:13:00.491435+0000","last_became_peered":"2023-10-17T01:13:00.491435+0000","last_unstale":"2023-10-17T01:13:01.046704+0000","last_undegraded":"2023-10-17T01:13:01.046704+0000","last_fullsized":"2023-10-17T01:13:01.046704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:37:29.052376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940625+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:13:01.940625+0000","last_peered":"2023-10-17T01:13:01.940625+0000","last_clean":"2023-10-17T01:13:01.940625+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:13:01.940625+0000","last_undegraded":"2023-10-17T01:13:01.940625+0000","last_fullsized":"2023-10-17T01:13:01.940625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:08.460339+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:13:08.460339+0000","last_peered":"2023-10-17T01:13:08.460339+0000","last_clean":"2023-10-17T01:13:08.460339+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:13:08.460339+0000","last_undegraded":"2023-10-17T01:13:08.460339+0000","last_fullsized":"2023-10-17T01:13:08.460339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.940599+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:13:01.940599+0000","last_peered":"2023-10-17T01:13:01.940599+0000","last_clean":"2023-10-17T01:13:01.940599+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:13:01.940599+0000","last_undegraded":"2023-10-17T01:13:01.940599+0000","last_fullsized":"2023-10-17T01:13:01.940599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065060+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:13:01.065060+0000","last_peered":"2023-10-17T01:13:01.065060+0000","last_clean":"2023-10-17T01:13:01.065060+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:13:01.065060+0000","last_undegraded":"2023-10-17T01:13:01.065060+0000","last_fullsized":"2023-10-17T01:13:01.065060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.441014+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:13:05.441014+0000","last_peered":"2023-10-17T01:13:05.441014+0000","last_clean":"2023-10-17T01:13:05.441014+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:13:05.441014+0000","last_undegraded":"2023-10-17T01:13:05.441014+0000","last_fullsized":"2023-10-17T01:13:05.441014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630580+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:13:02.630580+0000","last_peered":"2023-10-17T01:13:02.630580+0000","last_clean":"2023-10-17T01:13:02.630580+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:13:02.630580+0000","last_undegraded":"2023-10-17T01:13:02.630580+0000","last_fullsized":"2023-10-17T01:13:02.630580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424907+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:04.424907+0000","last_peered":"2023-10-17T01:13:04.424907+0000","last_clean":"2023-10-17T01:13:04.424907+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:04.424907+0000","last_undegraded":"2023-10-17T01:13:04.424907+0000","last_fullsized":"2023-10-17T01:13:04.424907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.057643+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:01.057643+0000","last_peered":"2023-10-17T01:13:01.057643+0000","last_clean":"2023-10-17T01:13:01.057643+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:01.057643+0000","last_undegraded":"2023-10-17T01:13:01.057643+0000","last_fullsized":"2023-10-17T01:13:01.057643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430869+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:13:01.430869+0000","last_peered":"2023-10-17T01:13:01.430869+0000","last_clean":"2023-10-17T01:13:01.430869+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:13:01.430869+0000","last_undegraded":"2023-10-17T01:13:01.430869+0000","last_fullsized":"2023-10-17T01:13:01.430869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.435606+0000","last_change":"2023-10-17T01:13:00.488729+0000","last_active":"2023-10-17T01:13:05.435606+0000","last_peered":"2023-10-17T01:13:05.435606+0000","last_clean":"2023-10-17T01:13:05.435606+0000","last_became_active":"2023-10-17T01:13:00.487735+0000","last_became_peered":"2023-10-17T01:13:00.487735+0000","last_unstale":"2023-10-17T01:13:05.435606+0000","last_undegraded":"2023-10-17T01:13:05.435606+0000","last_fullsized":"2023-10-17T01:13:05.435606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:53.287559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.459749+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:07.459749+0000","last_peered":"2023-10-17T01:13:07.459749+0000","last_clean":"2023-10-17T01:13:07.459749+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:07.459749+0000","last_undegraded":"2023-10-17T01:13:07.459749+0000","last_fullsized":"2023-10-17T01:13:07.459749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.431026+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:13:01.431026+0000","last_peered":"2023-10-17T01:13:01.431026+0000","last_clean":"2023-10-17T01:13:01.431026+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:13:01.431026+0000","last_undegraded":"2023-10-17T01:13:01.431026+0000","last_fullsized":"2023-10-17T01:13:01.431026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:05.440581+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:13:05.440581+0000","last_peered":"2023-10-17T01:13:05.440581+0000","last_clean":"2023-10-17T01:13:05.440581+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:13:05.440581+0000","last_undegraded":"2023-10-17T01:13:05.440581+0000","last_fullsized":"2023-10-17T01:13:05.440581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630805+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:13:02.630805+0000","last_peered":"2023-10-17T01:13:02.630805+0000","last_clean":"2023-10-17T01:13:02.630805+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:13:02.630805+0000","last_undegraded":"2023-10-17T01:13:02.630805+0000","last_fullsized":"2023-10-17T01:13:02.630805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.422918+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:04.422918+0000","last_peered":"2023-10-17T01:13:04.422918+0000","last_clean":"2023-10-17T01:13:04.422918+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:04.422918+0000","last_undegraded":"2023-10-17T01:13:04.422918+0000","last_fullsized":"2023-10-17T01:13:04.422918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033457+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:13:02.033457+0000","last_peered":"2023-10-17T01:13:02.033457+0000","last_clean":"2023-10-17T01:13:02.033457+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:13:02.033457+0000","last_undegraded":"2023-10-17T01:13:02.033457+0000","last_fullsized":"2023-10-17T01:13:02.033457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.423700+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:04.423700+0000","last_peered":"2023-10-17T01:13:04.423700+0000","last_clean":"2023-10-17T01:13:04.423700+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:04.423700+0000","last_undegraded":"2023-10-17T01:13:04.423700+0000","last_fullsized":"2023-10-17T01:13:04.423700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848108+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:13:02.848108+0000","last_peered":"2023-10-17T01:13:02.848108+0000","last_clean":"2023-10-17T01:13:02.848108+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:13:02.848108+0000","last_undegraded":"2023-10-17T01:13:02.848108+0000","last_fullsized":"2023-10-17T01:13:02.848108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:07.458808+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:07.458808+0000","last_peered":"2023-10-17T01:13:07.458808+0000","last_clean":"2023-10-17T01:13:07.458808+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:07.458808+0000","last_undegraded":"2023-10-17T01:13:07.458808+0000","last_fullsized":"2023-10-17T01:13:07.458808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848588+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:13:02.848588+0000","last_peered":"2023-10-17T01:13:02.848588+0000","last_clean":"2023-10-17T01:13:02.848588+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:13:02.848588+0000","last_undegraded":"2023-10-17T01:13:02.848588+0000","last_fullsized":"2023-10-17T01:13:02.848588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:06.448727+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:06.448727+0000","last_peered":"2023-10-17T01:13:06.448727+0000","last_clean":"2023-10-17T01:13:06.448727+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:06.448727+0000","last_undegraded":"2023-10-17T01:13:06.448727+0000","last_fullsized":"2023-10-17T01:13:06.448727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630439+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:13:02.630439+0000","last_peered":"2023-10-17T01:13:02.630439+0000","last_clean":"2023-10-17T01:13:02.630439+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:13:02.630439+0000","last_undegraded":"2023-10-17T01:13:02.630439+0000","last_fullsized":"2023-10-17T01:13:02.630439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.424491+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:04.424491+0000","last_peered":"2023-10-17T01:13:04.424491+0000","last_clean":"2023-10-17T01:13:04.424491+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:04.424491+0000","last_undegraded":"2023-10-17T01:13:04.424491+0000","last_fullsized":"2023-10-17T01:13:04.424491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347733+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:13:02.347733+0000","last_peered":"2023-10-17T01:13:02.347733+0000","last_clean":"2023-10-17T01:13:02.347733+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:13:02.347733+0000","last_undegraded":"2023-10-17T01:13:02.347733+0000","last_fullsized":"2023-10-17T01:13:02.347733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:06.449334+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:13:06.449334+0000","last_peered":"2023-10-17T01:13:06.449334+0000","last_clean":"2023-10-17T01:13:06.449334+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:13:06.449334+0000","last_undegraded":"2023-10-17T01:13:06.449334+0000","last_fullsized":"2023-10-17T01:13:06.449334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347959+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:13:02.347959+0000","last_peered":"2023-10-17T01:13:02.347959+0000","last_clean":"2023-10-17T01:13:02.347959+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:13:02.347959+0000","last_undegraded":"2023-10-17T01:13:02.347959+0000","last_fullsized":"2023-10-17T01:13:02.347959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.069637+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:01.069637+0000","last_peered":"2023-10-17T01:13:01.069637+0000","last_clean":"2023-10-17T01:13:01.069637+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:01.069637+0000","last_undegraded":"2023-10-17T01:13:01.069637+0000","last_fullsized":"2023-10-17T01:13:01.069637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:04.005711+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:13:04.005711+0000","last_peered":"2023-10-17T01:13:04.005711+0000","last_clean":"2023-10-17T01:13:04.005711+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:13:04.005711+0000","last_undegraded":"2023-10-17T01:13:04.005711+0000","last_fullsized":"2023-10-17T01:13:04.005711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.431065+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:01.431065+0000","last_peered":"2023-10-17T01:13:01.431065+0000","last_clean":"2023-10-17T01:13:01.431065+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:01.431065+0000","last_undegraded":"2023-10-17T01:13:01.431065+0000","last_fullsized":"2023-10-17T01:13:01.431065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.347920+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:13:02.347920+0000","last_peered":"2023-10-17T01:13:02.347920+0000","last_clean":"2023-10-17T01:13:02.347920+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:13:02.347920+0000","last_undegraded":"2023-10-17T01:13:02.347920+0000","last_fullsized":"2023-10-17T01:13:02.347920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848662+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:13:02.848662+0000","last_peered":"2023-10-17T01:13:02.848662+0000","last_clean":"2023-10-17T01:13:02.848662+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:13:02.848662+0000","last_undegraded":"2023-10-17T01:13:02.848662+0000","last_fullsized":"2023-10-17T01:13:02.848662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047916+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:13:02.047916+0000","last_peered":"2023-10-17T01:13:02.047916+0000","last_clean":"2023-10-17T01:13:02.047916+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:13:02.047916+0000","last_undegraded":"2023-10-17T01:13:02.047916+0000","last_fullsized":"2023-10-17T01:13:02.047916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047635+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:13:02.047635+0000","last_peered":"2023-10-17T01:13:02.047635+0000","last_clean":"2023-10-17T01:13:02.047635+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:13:02.047635+0000","last_undegraded":"2023-10-17T01:13:02.047635+0000","last_fullsized":"2023-10-17T01:13:02.047635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.033413+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:13:02.033413+0000","last_peered":"2023-10-17T01:13:02.033413+0000","last_clean":"2023-10-17T01:13:02.033413+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:13:02.033413+0000","last_undegraded":"2023-10-17T01:13:02.033413+0000","last_fullsized":"2023-10-17T01:13:02.033413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.431045+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:13:01.431045+0000","last_peered":"2023-10-17T01:13:01.431045+0000","last_clean":"2023-10-17T01:13:01.431045+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:13:01.431045+0000","last_undegraded":"2023-10-17T01:13:01.431045+0000","last_fullsized":"2023-10-17T01:13:01.431045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.849011+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:13:02.849011+0000","last_peered":"2023-10-17T01:13:02.849011+0000","last_clean":"2023-10-17T01:13:02.849011+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:13:02.849011+0000","last_undegraded":"2023-10-17T01:13:02.849011+0000","last_fullsized":"2023-10-17T01:13:02.849011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.047951+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:13:02.047951+0000","last_peered":"2023-10-17T01:13:02.047951+0000","last_clean":"2023-10-17T01:13:02.047951+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:13:02.047951+0000","last_undegraded":"2023-10-17T01:13:02.047951+0000","last_fullsized":"2023-10-17T01:13:02.047951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630796+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:13:02.630796+0000","last_peered":"2023-10-17T01:13:02.630796+0000","last_clean":"2023-10-17T01:13:02.630796+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:13:02.630796+0000","last_undegraded":"2023-10-17T01:13:02.630796+0000","last_fullsized":"2023-10-17T01:13:02.630796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.065243+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:13:01.065243+0000","last_peered":"2023-10-17T01:13:01.065243+0000","last_clean":"2023-10-17T01:13:01.065243+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:13:01.065243+0000","last_undegraded":"2023-10-17T01:13:01.065243+0000","last_fullsized":"2023-10-17T01:13:01.065243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.630228+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:13:02.630228+0000","last_peered":"2023-10-17T01:13:02.630228+0000","last_clean":"2023-10-17T01:13:02.630228+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:13:02.630228+0000","last_undegraded":"2023-10-17T01:13:02.630228+0000","last_fullsized":"2023-10-17T01:13:02.630228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.634809+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:13:02.634809+0000","last_peered":"2023-10-17T01:13:02.634809+0000","last_clean":"2023-10-17T01:13:02.634809+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:13:02.634809+0000","last_undegraded":"2023-10-17T01:13:02.634809+0000","last_fullsized":"2023-10-17T01:13:02.634809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-17T01:13:01.430980+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:13:01.430980+0000","last_peered":"2023-10-17T01:13:01.430980+0000","last_clean":"2023-10-17T01:13:01.430980+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:13:01.430980+0000","last_undegraded":"2023-10-17T01:13:01.430980+0000","last_fullsized":"2023-10-17T01:13:01.430980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-17T01:13:02.848866+0000","last_change":"2023-10-17T01:13:01.066367+0000","last_active":"2023-10-17T01:13:02.848866+0000","last_peered":"2023-10-17T01:13:02.848866+0000","last_clean":"2023-10-17T01:13:02.848866+0000","last_became_active":"2023-10-17T01:13:01.065975+0000","last_became_peered":"2023-10-17T01:13:01.065975+0000","last_unstale":"2023-10-17T01:13:02.848866+0000","last_undegraded":"2023-10-17T01:13:02.848866+0000","last_fullsized":"2023-10-17T01:13:02.848866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:29.094472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6954,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":167,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":73774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":426909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678033,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1019227,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.81699999999999995}]},{"osd":3,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.84499999999999997}]},{"osd":6,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.875},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.83699999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.93100000000000005}]},{"osd":9,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.97699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":426549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.67400000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.751}]},{"osd":7,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.60099999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.879}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.80400000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103194,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":426934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.76600000000000001}]},{"osd":3,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.749}]},{"osd":6,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.80800000000000005}]},{"osd":9,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.90900000000000003}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332127,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":1019315,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76000000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.505}]},{"osd":2,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65300000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59899999999999998,"15min":0.58599999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.626}]},{"osd":5,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.86299999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.54300000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.84499999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.04,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92800000000000005}]},{"osd":9,"last update":"Tue Oct 17 01:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.878}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.91500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.38100000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.46800000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95199999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78300000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.52100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.64600000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Oct 17 01:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.80900000000000005,"15min":0.80200000000000005},"min":{"1min":0.63500000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3899999999999999,"5min":1.532,"15min":1.532},"last":0.85399999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.0580000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":1.417,"15min":1.417},"last":0.97099999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.018}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.91600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":446127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.498},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.505},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.53000000000000003},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69199999999999995}]},{"osd":3,"last update":"Tue Oct 17 01:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.45400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.50900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Oct 17 01:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.45200000000000001,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.47099999999999997,"15min":0.45800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.59599999999999997},"min":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66000000000000003,"15min":0.64100000000000001},"min":{"1min":0.441,"5min":0.432,"15min":0.432},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.748}]},{"osd":6,"last update":"Tue Oct 17 01:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Oct 17 01:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.72599999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84399999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.875}]},{"osd":10,"last update":"Tue Oct 17 01:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":427783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.01,"5min":1.02,"15min":1.02},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54800000000000004,"15min":0.51900000000000002},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68799999999999994}]},{"osd":2,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.63900000000000001,"15min":0.59099999999999997},"min":{"1min":0.499,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.79600000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.313,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89100000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.36899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Oct 17 01:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68100000000000005,"15min":0.64900000000000002},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":0.997,"15min":0.997},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.56200000000000006},"min":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68700000000000006,"15min":0.66100000000000003},"min":{"1min":0.48499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":0.91000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3340000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.997}]},{"osd":9,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.83799999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":429324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.52200000000000002},"min":{"1min":0.376,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77800000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.40100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72399999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.749}]},{"osd":3,"last update":"Tue Oct 17 01:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Oct 17 01:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.60599999999999998,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":6.133,"5min":6.133,"15min":6.133},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.66700000000000004,"15min":0.624},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"last":0.621}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.437},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.495},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.81999999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.84299999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.80500000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.2060000000000004,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":426318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.42399999999999999,"15min":0.40200000000000002},"min":{"1min":0.34999999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.42199999999999999,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.63800000000000001,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.626},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65200000000000002,"15min":0.63300000000000001},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99099999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.66300000000000003,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Oct 17 01:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.58999999999999997},"min":{"1min":0.59699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0860000000000001,"5min":1.161,"15min":1.161},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.57299999999999995,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.752,"15min":0.73099999999999998},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.81899999999999995,"15min":0.80700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.65100000000000002}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.47699999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.33100000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51600000000000001},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"min":{"1min":0.50900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64500000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.745},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79800000000000004}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.47999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65600000000000003,"15min":0.63900000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.78100000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69499999999999995}]},{"osd":9,"last update":"Tue Oct 17 01:12:35 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.85199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757287,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":1017561,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73999999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.81100000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.84199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.54100000000000004,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.755}]},{"osd":2,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40500000000000003,"15min":0.38900000000000001},"min":{"1min":0.28399999999999997,"5min":0.223,"15min":0.223},"max":{"1min":0.76600000000000001,"5min":1.24,"15min":1.24},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27100000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.5}]},{"osd":3,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.76500000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Oct 17 01:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.67500000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71899999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82099999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":21199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:13:10.954 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-17T01:13:10.955 INFO:tasks.cephadm:Setting up client nodes... 2023-10-17T01:13:10.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-17T01:13:11.516 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:13:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: 2023-10-17T01:13:11.249+0000 7fb246f5d700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-10-17T01:13:11.516 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:13:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: 2023-10-17T01:13:11.249+0000 7fb246f5d700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-10-17T01:13:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:11 smithi045 ceph-mon[98474]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:11 smithi045 ceph-mon[98474]: pgmap v272: 97 pgs: 97 active+clean; 583 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 4.8 KiB/s wr, 18 op/s 2023-10-17T01:13:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:11 smithi060 ceph-mon[134136]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:11 smithi060 ceph-mon[134136]: pgmap v272: 97 pgs: 97 active+clean; 583 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 4.8 KiB/s wr, 18 op/s 2023-10-17T01:13:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:11 smithi181 ceph-mon[134520]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:11 smithi181 ceph-mon[134520]: pgmap v272: 97 pgs: 97 active+clean; 583 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 4.8 KiB/s wr, 18 op/s 2023-10-17T01:13:12.676 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:12 smithi045 ceph-mon[98474]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:12 smithi060 ceph-mon[134136]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:12.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:12 smithi181 ceph-mon[134520]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:13.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 -- 172.21.15.45:0/4140997943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a0074d80 msgr2=0x7f36a00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:13.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/4140997943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a0074d80 0x7f36a00731e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f369c009990 tx=0x7f369c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 -- 172.21.15.45:0/4140997943 shutdown_connections 2023-10-17T01:13:13.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/4140997943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0108b10 0x7f36a010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/4140997943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a0074d80 0x7f36a00731e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/4140997943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a00737b0 0x7f36a0073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 -- 172.21.15.45:0/4140997943 >> 172.21.15.45:0/4140997943 conn(0x7f36a00fc200 msgr2=0x7f36a00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:13.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 -- 172.21.15.45:0/4140997943 shutdown_connections 2023-10-17T01:13:13.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.479+0000 7f36a5cb4700 1 -- 172.21.15.45:0/4140997943 wait complete. 2023-10-17T01:13:13.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.480+0000 7f36a5cb4700 1 Processor -- start 2023-10-17T01:13:13.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.480+0000 7f36a5cb4700 1 -- start start 2023-10-17T01:13:13.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a5cb4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 0x7f36a0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:13.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a5cb4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0074d80 0x7f36a0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:13.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a5cb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a0108b10 0x7f36a01a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:13.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a5cb4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01a06c0 con 0x7f36a00737b0 2023-10-17T01:13:13.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a5cb4700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01a0800 con 0x7f36a0074d80 2023-10-17T01:13:13.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a5cb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01a0990 con 0x7f36a0108b10 2023-10-17T01:13:13.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a4cb2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 0x7f36a0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:13.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a4cb2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 0x7f36a0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55718/0 (socket says 172.21.15.45:55718) 2023-10-17T01:13:13.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a4cb2700 1 -- 172.21.15.45:0/3204506723 learned_addr learned my addr 172.21.15.45:0/3204506723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:13.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f3697fff700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0074d80 0x7f36a0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:13.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a54b3700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a0108b10 0x7f36a01a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:13.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.481+0000 7f36a4cb2700 1 -- 172.21.15.45:0/3204506723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a0108b10 msgr2=0x7f36a01a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:13.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a4cb2700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a0108b10 0x7f36a01a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a4cb2700 1 -- 172.21.15.45:0/3204506723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0074d80 msgr2=0x7f36a0199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:13.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a4cb2700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0074d80 0x7f36a0199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a4cb2700 1 -- 172.21.15.45:0/3204506723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0095f0 con 0x7f36a00737b0 2023-10-17T01:13:13.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f3697fff700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0074d80 0x7f36a0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:13.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a4cb2700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 0x7f36a0198b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f369c004740 tx=0x7f369c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:13.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c01d070 con 0x7f36a00737b0 2023-10-17T01:13:13.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f369c021410 con 0x7f36a00737b0 2023-10-17T01:13:13.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c017650 con 0x7f36a00737b0 2023-10-17T01:13:13.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a01a0c10 con 0x7f36a00737b0 2023-10-17T01:13:13.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.482+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a01a1020 con 0x7f36a00737b0 2023-10-17T01:13:13.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.483+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a01071c0 con 0x7f36a00737b0 2023-10-17T01:13:13.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.485+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f369c021580 con 0x7f36a00737b0 2023-10-17T01:13:13.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.486+0000 7f3695ffb700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36880a0c30 0x7f36880a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:13.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.487+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f369c0cd630 con 0x7f36a00737b0 2023-10-17T01:13:13.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.487+0000 7f3697fff700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36880a0c30 0x7f36880a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:13.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.487+0000 7f3697fff700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36880a0c30 0x7f36880a30f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36a0074ab0 tx=0x7f3690009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:13.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.490+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f369c096fa0 con 0x7f36a00737b0 2023-10-17T01:13:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.785+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f36a002ce80 con 0x7f36a00737b0 2023-10-17T01:13:13.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.789+0000 7f3695ffb700 1 -- 172.21.15.45:0/3204506723 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f369c09a5c0 con 0x7f36a00737b0 2023-10-17T01:13:13.791 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-10-17T01:13:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCp3y1l6pXmLhAAHe3prDAN0XVRXq4SMaVuzQ== 2023-10-17T01:13:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36880a0c30 msgr2=0x7f36880a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36880a0c30 0x7f36880a30f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36a0074ab0 tx=0x7f3690009260 comp rx=0 tx=0).stop 2023-10-17T01:13:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 msgr2=0x7f36a0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 0x7f36a0198b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f369c004740 tx=0x7f369c009580 comp rx=0 tx=0).stop 2023-10-17T01:13:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 shutdown_connections 2023-10-17T01:13:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36880a0c30 0x7f36880a30f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a0108b10 0x7f36a01a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00737b0 0x7f36a0198b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 --2- 172.21.15.45:0/3204506723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0074d80 0x7f36a0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 >> 172.21.15.45:0/3204506723 conn(0x7f36a00fc200 msgr2=0x7f36a00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 shutdown_connections 2023-10-17T01:13:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:13.792+0000 7f36a5cb4700 1 -- 172.21.15.45:0/3204506723 wait complete. 2023-10-17T01:13:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:13 smithi045 ceph-mon[98474]: pgmap v273: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 KiB/s wr, 25 op/s 2023-10-17T01:13:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:13 smithi060 ceph-mon[134136]: pgmap v273: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 KiB/s wr, 25 op/s 2023-10-17T01:13:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:13 smithi181 ceph-mon[134520]: pgmap v273: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 KiB/s wr, 25 op/s 2023-10-17T01:13:14.316 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:13:14.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-17T01:13:14.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-17T01:13:14.424 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-17T01:13:14.425 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-17T01:13:14.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-17T01:13:14.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:14 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3204506723' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-17T01:13:14.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:14 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3204506723' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-17T01:13:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:14 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3204506723' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-17T01:13:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:14 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3204506723' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-17T01:13:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:14 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3204506723' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-17T01:13:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:14 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3204506723' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-17T01:13:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:15.753 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:15 smithi045 ceph-mon[98474]: pgmap v274: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2023-10-17T01:13:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:15 smithi060 ceph-mon[134136]: pgmap v274: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2023-10-17T01:13:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:15 smithi181 ceph-mon[134520]: pgmap v274: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2023-10-17T01:13:16.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 -- 172.21.15.45:0/148647098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74241060b0 msgr2=0x7f742410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 --2- 172.21.15.45:0/148647098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74241060b0 0x7f742410dc80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f741c00b0a0 tx=0x7f741c00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 -- 172.21.15.45:0/148647098 shutdown_connections 2023-10-17T01:13:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 --2- 172.21.15.45:0/148647098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74241060b0 0x7f742410dc80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 --2- 172.21.15.45:0/148647098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7424100fd0 0x7f74241013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 --2- 172.21.15.45:0/148647098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7424101980 0x7f74241059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 -- 172.21.15.45:0/148647098 >> 172.21.15.45:0/148647098 conn(0x7f74240fc840 msgr2=0x7f74240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 -- 172.21.15.45:0/148647098 shutdown_connections 2023-10-17T01:13:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.760+0000 7f742aff1700 1 -- 172.21.15.45:0/148647098 wait complete. 2023-10-17T01:13:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 Processor -- start 2023-10-17T01:13:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 -- start start 2023-10-17T01:13:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7424100fd0 0x7f7424196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 0x7f74241974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74241060b0 0x7f742419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742419bf50 con 0x7f7424101980 2023-10-17T01:13:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742419c0c0 con 0x7f74241060b0 2023-10-17T01:13:16.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.761+0000 7f742aff1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742419c280 con 0x7f7424100fd0 2023-10-17T01:13:16.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 0x7f74241974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:16.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7428d8d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7424100fd0 0x7f7424196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:16.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 0x7f74241974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55742/0 (socket says 172.21.15.45:55742) 2023-10-17T01:13:16.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 -- 172.21.15.45:0/3387008595 learned_addr learned my addr 172.21.15.45:0/3387008595 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:16.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f742958e700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74241060b0 0x7f742419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:16.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 -- 172.21.15.45:0/3387008595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7424100fd0 msgr2=0x7f7424196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:16.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7424100fd0 0x7f7424196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:16.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 -- 172.21.15.45:0/3387008595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74241060b0 msgr2=0x7f742419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:16.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74241060b0 0x7f742419b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:16.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 -- 172.21.15.45:0/3387008595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c009d00 con 0x7f7424101980 2023-10-17T01:13:16.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.762+0000 7f7423fff700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 0x7f74241974d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f741000e970 tx=0x7f741000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:16.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.763+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410009790 con 0x7f7424101980 2023-10-17T01:13:16.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.763+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f741000c7e0 con 0x7f7424101980 2023-10-17T01:13:16.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.763+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742419c560 con 0x7f7424101980 2023-10-17T01:13:16.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.763+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74100054e0 con 0x7f7424101980 2023-10-17T01:13:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.763+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74241a3dc0 con 0x7f7424101980 2023-10-17T01:13:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.764+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742404ea50 con 0x7f7424101980 2023-10-17T01:13:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.766+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f741000c950 con 0x7f7424101980 2023-10-17T01:13:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.767+0000 7f7421ffb700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f74140a0c30 0x7f74140a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.767+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7410014070 con 0x7f7424101980 2023-10-17T01:13:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.767+0000 7f7428d8d700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f74140a0c30 0x7f74140a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.768+0000 7f7428d8d700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f74140a0c30 0x7f74140a30f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7418009c60 tx=0x7f7418009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:16.771+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7410095c80 con 0x7f7424101980 2023-10-17T01:13:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.040+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f74241a3fb0 con 0x7f7424101980 2023-10-17T01:13:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.046+0000 7f7421ffb700 1 -- 172.21.15.45:0/3387008595 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259617 (secure 0 0 0) 0x7f74100992a0 con 0x7f7424101980 2023-10-17T01:13:17.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.049+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f74140a0c30 msgr2=0x7f74140a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.049+0000 7f742aff1700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f74140a0c30 0x7f74140a30f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7418009c60 tx=0x7f7418009260 comp rx=0 tx=0).stop 2023-10-17T01:13:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.049+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 msgr2=0x7f74241974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.049+0000 7f742aff1700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 0x7f74241974d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f741000e970 tx=0x7f741000ec80 comp rx=0 tx=0).stop 2023-10-17T01:13:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 shutdown_connections 2023-10-17T01:13:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f74140a0c30 0x7f74140a30f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74241060b0 0x7f742419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7424100fd0 0x7f7424196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 --2- 172.21.15.45:0/3387008595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7424101980 0x7f74241974d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 >> 172.21.15.45:0/3387008595 conn(0x7f74240fc840 msgr2=0x7f74240fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 shutdown_connections 2023-10-17T01:13:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:17.050+0000 7f742aff1700 1 -- 172.21.15.45:0/3387008595 wait complete. 2023-10-17T01:13:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":869456926},{"type":"v1","addr":"172.21.15.45:6801","nonce":869456926}]},"active_addr":"172.21.15.45:6801/869456926","active_change":"2023-10-17T01:06:32.349062+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":4118900047}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":4229440561}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1235220098}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":55059091}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1176656230}]}]} 2023-10-17T01:13:17.495 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-17T01:13:17.495 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-17T01:13:17.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-17T01:13:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:17 smithi045 ceph-mon[98474]: pgmap v275: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2023-10-17T01:13:17.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:17 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3387008595' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-17T01:13:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:17 smithi060 ceph-mon[134136]: pgmap v275: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2023-10-17T01:13:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:17 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3387008595' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-17T01:13:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:17 smithi181 ceph-mon[134520]: pgmap v275: 97 pgs: 97 active+clean; 624 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2023-10-17T01:13:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:17 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3387008595' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-17T01:13:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:18.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:18 smithi045 ceph-mon[98474]: pgmap v276: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2023-10-17T01:13:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:18 smithi181 ceph-mon[134520]: pgmap v276: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2023-10-17T01:13:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:18 smithi060 ceph-mon[134136]: pgmap v276: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2023-10-17T01:13:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.908+0000 7fe7ecd87700 1 -- 172.21.15.45:0/884495171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 msgr2=0x7fe7e810dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.908+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/884495171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e810dac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000b0a0 tx=0x7fe7e000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 -- 172.21.15.45:0/884495171 shutdown_connections 2023-10-17T01:13:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/884495171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e810dac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/884495171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 0x7fe7e81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/884495171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7e81017c0 0x7fe7e8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 -- 172.21.15.45:0/884495171 >> 172.21.15.45:0/884495171 conn(0x7fe7e80fc700 msgr2=0x7fe7e80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 -- 172.21.15.45:0/884495171 shutdown_connections 2023-10-17T01:13:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.909+0000 7fe7ecd87700 1 -- 172.21.15.45:0/884495171 wait complete. 2023-10-17T01:13:19.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.910+0000 7fe7ecd87700 1 Processor -- start 2023-10-17T01:13:19.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 -- start start 2023-10-17T01:13:19.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 0x7fe7e8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:19.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7e81017c0 0x7fe7e8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:19.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:19.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e819e000 con 0x7fe7e8105ef0 2023-10-17T01:13:19.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e819e170 con 0x7fe7e8100e10 2023-10-17T01:13:19.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.911+0000 7fe7ecd87700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e819e330 con 0x7fe7e81017c0 2023-10-17T01:13:19.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:19.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 0x7fe7e8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:19.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e819d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42820/0 (socket says 172.21.15.45:42820) 2023-10-17T01:13:19.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7e81017c0 0x7fe7e8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:19.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 -- 172.21.15.45:0/1853188436 learned_addr learned my addr 172.21.15.45:0/1853188436 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:19.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 -- 172.21.15.45:0/1853188436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7e81017c0 msgr2=0x7fe7e8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:19.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7e81017c0 0x7fe7e8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:19.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 -- 172.21.15.45:0/1853188436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 msgr2=0x7fe7e8199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:19.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 0x7fe7e8199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:19.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 -- 172.21.15.45:0/1853188436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0009d00 con 0x7fe7e8105ef0 2023-10-17T01:13:19.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e77fe700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 0x7fe7e8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:19.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.912+0000 7fe7e7fff700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e819d9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009fd0 tx=0x7fe7e000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:19.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.913+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e000e040 con 0x7fe7e8105ef0 2023-10-17T01:13:19.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.913+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e819e5b0 con 0x7fe7e8105ef0 2023-10-17T01:13:19.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.913+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7e00094c0 con 0x7fe7e8105ef0 2023-10-17T01:13:19.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.913+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00052a0 con 0x7fe7e8105ef0 2023-10-17T01:13:19.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.913+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e81a5e70 con 0x7fe7e8105ef0 2023-10-17T01:13:19.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.914+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e804f2a0 con 0x7fe7e8105ef0 2023-10-17T01:13:19.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.916+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fe7e0003a00 con 0x7fe7e8105ef0 2023-10-17T01:13:19.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.917+0000 7fe7e4ff9700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7d00a0bb0 0x7fe7d00a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:19.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.918+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7e00cd860 con 0x7fe7e8105ef0 2023-10-17T01:13:19.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.918+0000 7fe7e77fe700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7d00a0bb0 0x7fe7d00a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:19.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.918+0000 7fe7e77fe700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7d00a0bb0 0x7fe7d00a3070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8011490 tx=0x7fe7d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:19.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:19.920+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe7e00971d0 con 0x7fe7e8105ef0 2023-10-17T01:13:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.131+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e819e7a0 con 0x7fe7e8105ef0 2023-10-17T01:13:20.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.132+0000 7fe7e4ff9700 1 -- 172.21.15.45:0/1853188436 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7fe7e0003cb0 con 0x7fe7e8105ef0 2023-10-17T01:13:20.134 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:20.135 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:02.040707+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:13:20.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7d00a0bb0 msgr2=0x7fe7d00a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7d00a0bb0 0x7fe7d00a3070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8011490 tx=0x7fe7d8009260 comp rx=0 tx=0).stop 2023-10-17T01:13:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 msgr2=0x7fe7e819d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e819d9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009fd0 tx=0x7fe7e000bce0 comp rx=0 tx=0).stop 2023-10-17T01:13:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 shutdown_connections 2023-10-17T01:13:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7d00a0bb0 0x7fe7d00a3070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:20.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7e8105ef0 0x7fe7e819d9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:20.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7e8100e10 0x7fe7e8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:20.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 --2- 172.21.15.45:0/1853188436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7e81017c0 0x7fe7e8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:20.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 >> 172.21.15.45:0/1853188436 conn(0x7fe7e80fc700 msgr2=0x7fe7e80fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:20.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 shutdown_connections 2023-10-17T01:13:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:20.135+0000 7fe7ecd87700 1 -- 172.21.15.45:0/1853188436 wait complete. 2023-10-17T01:13:20.622 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-17T01:13:20.622 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-17T01:13:20.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:20 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1853188436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:20.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:20 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1853188436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:20.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:20 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1853188436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:21.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:21 smithi045 ceph-mon[98474]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 19 op/s 2023-10-17T01:13:21.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:21 smithi060 ceph-mon[134136]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 19 op/s 2023-10-17T01:13:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:21 smithi181 ceph-mon[134520]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 19 op/s 2023-10-17T01:13:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 -- 172.21.15.45:0/3825516394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce0101980 msgr2=0x7efce01059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:22.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 --2- 172.21.15.45:0/3825516394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce0101980 0x7efce01059d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009990 tx=0x7efcd4009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:22.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 -- 172.21.15.45:0/3825516394 shutdown_connections 2023-10-17T01:13:22.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 --2- 172.21.15.45:0/3825516394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce01060b0 0x7efce010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:22.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 --2- 172.21.15.45:0/3825516394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0100fd0 0x7efce01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 --2- 172.21.15.45:0/3825516394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce0101980 0x7efce01059d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 -- 172.21.15.45:0/3825516394 >> 172.21.15.45:0/3825516394 conn(0x7efce00fc840 msgr2=0x7efce00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 -- 172.21.15.45:0/3825516394 shutdown_connections 2023-10-17T01:13:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.883+0000 7efce68f2700 1 -- 172.21.15.45:0/3825516394 wait complete. 2023-10-17T01:13:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.884+0000 7efce68f2700 1 Processor -- start 2023-10-17T01:13:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.884+0000 7efce68f2700 1 -- start start 2023-10-17T01:13:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce68f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 0x7efce0196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce68f2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0101980 0x7efce0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce68f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 0x7efce019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce68f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019beb0 con 0x7efce01060b0 2023-10-17T01:13:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce68f2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019c020 con 0x7efce0101980 2023-10-17T01:13:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce68f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019c1e0 con 0x7efce0100fd0 2023-10-17T01:13:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 0x7efce019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efcdffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 0x7efce0196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 0x7efce019b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42828/0 (socket says 172.21.15.45:42828) 2023-10-17T01:13:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 -- 172.21.15.45:0/250550881 learned_addr learned my addr 172.21.15.45:0/250550881 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efcdffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 0x7efce0196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37688/0 (socket says 172.21.15.45:37688) 2023-10-17T01:13:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efcdf7fe700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0101980 0x7efce0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 -- 172.21.15.45:0/250550881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 msgr2=0x7efce0196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 0x7efce0196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 -- 172.21.15.45:0/250550881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0101980 msgr2=0x7efce0197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0101980 0x7efce0197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efce4e8f700 1 -- 172.21.15.45:0/250550881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40095f0 con 0x7efce01060b0 2023-10-17T01:13:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.885+0000 7efcdffff700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 0x7efce0196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.886+0000 7efce4e8f700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 0x7efce019b8a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efcd800ef30 tx=0x7efcd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.886+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8018070 con 0x7efce01060b0 2023-10-17T01:13:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.886+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce019c460 con 0x7efce01060b0 2023-10-17T01:13:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.886+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd800f040 con 0x7efce01060b0 2023-10-17T01:13:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.886+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800cd00 con 0x7efce01060b0 2023-10-17T01:13:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.886+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01a3d20 con 0x7efce01060b0 2023-10-17T01:13:22.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.887+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce004ea50 con 0x7efce01060b0 2023-10-17T01:13:22.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.889+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7efcd8003a00 con 0x7efce01060b0 2023-10-17T01:13:22.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.890+0000 7efcdd7fa700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efcc80a0bb0 0x7efcc80a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:22.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.891+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efcd80cbd90 con 0x7efce01060b0 2023-10-17T01:13:22.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.891+0000 7efcdffff700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efcc80a0bb0 0x7efcc80a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:22.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.891+0000 7efcdffff700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efcc80a0bb0 0x7efcc80a3070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efcd0007fd0 tx=0x7efcd00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:22.895+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7efcd8095700 con 0x7efce01060b0 2023-10-17T01:13:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.105+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efce01a3f50 con 0x7efce01060b0 2023-10-17T01:13:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.106+0000 7efcdd7fa700 1 -- 172.21.15.45:0/250550881 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22059 (secure 0 0 0) 0x7efcd8098d20 con 0x7efce01060b0 2023-10-17T01:13:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:23.108 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:02.040707+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:13:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.108+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efcc80a0bb0 msgr2=0x7efcc80a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.108+0000 7efce68f2700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efcc80a0bb0 0x7efcc80a3070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efcd0007fd0 tx=0x7efcd00073d0 comp rx=0 tx=0).stop 2023-10-17T01:13:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.108+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 msgr2=0x7efce019b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.108+0000 7efce68f2700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 0x7efce019b8a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efcd800ef30 tx=0x7efcd800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:13:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 shutdown_connections 2023-10-17T01:13:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efcc80a0bb0 0x7efcc80a3070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce01060b0 0x7efce019b8a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efce0100fd0 0x7efce0196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 --2- 172.21.15.45:0/250550881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0101980 0x7efce0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 >> 172.21.15.45:0/250550881 conn(0x7efce00fc840 msgr2=0x7efce00fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 shutdown_connections 2023-10-17T01:13:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:23.109+0000 7efce68f2700 1 -- 172.21.15.45:0/250550881 wait complete. 2023-10-17T01:13:23.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-17T01:13:23.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-17T01:13:23.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-17T01:13:23.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-17T01:13:23.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-17T01:13:23.657 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-17T01:13:23.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-17T01:13:23.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-17T01:13:23.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-17T01:13:23.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-17T01:13:23.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-17T01:13:23.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-17T01:13:23.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:23 smithi045 ceph-mon[98474]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2023-10-17T01:13:23.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:23 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/250550881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:23 smithi060 ceph-mon[134136]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2023-10-17T01:13:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:23 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/250550881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:23 smithi181 ceph-mon[134520]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2023-10-17T01:13:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:23 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/250550881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:13:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:25.694 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:25.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:25 smithi045 ceph-mon[98474]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-17T01:13:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:25 smithi060 ceph-mon[134136]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-17T01:13:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:25 smithi181 ceph-mon[134520]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-17T01:13:25.985 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.082 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.085 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.400 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.403 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:27 smithi045 ceph-mon[98474]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-17T01:13:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:27 smithi060 ceph-mon[134136]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-17T01:13:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:27 smithi181 ceph-mon[134520]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-17T01:13:29.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:29 smithi045 ceph-mon[98474]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-10-17T01:13:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:29 smithi060 ceph-mon[134136]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-10-17T01:13:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:29 smithi181 ceph-mon[134520]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-10-17T01:13:31.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.563+0000 7f4a21817700 1 -- 172.21.15.45:0/2696891082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a1c1017c0 msgr2=0x7f4a1c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:31.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.563+0000 7f4a21817700 1 --2- 172.21.15.45:0/2696891082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a1c1017c0 0x7f4a1c105810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009990 tx=0x7f4a08009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 -- 172.21.15.45:0/2696891082 shutdown_connections 2023-10-17T01:13:31.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 --2- 172.21.15.45:0/2696891082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c105ef0 0x7f4a1c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 --2- 172.21.15.45:0/2696891082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a1c100e10 0x7f4a1c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 --2- 172.21.15.45:0/2696891082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a1c1017c0 0x7f4a1c105810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 -- 172.21.15.45:0/2696891082 >> 172.21.15.45:0/2696891082 conn(0x7f4a1c0fc700 msgr2=0x7f4a1c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:31.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 -- 172.21.15.45:0/2696891082 shutdown_connections 2023-10-17T01:13:31.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.564+0000 7f4a21817700 1 -- 172.21.15.45:0/2696891082 wait complete. 2023-10-17T01:13:31.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.565+0000 7f4a21817700 1 Processor -- start 2023-10-17T01:13:31.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.565+0000 7f4a21817700 1 -- start start 2023-10-17T01:13:31.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.565+0000 7f4a21817700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c100e10 0x7f4a1c107b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:31.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.565+0000 7f4a21817700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a1c1017c0 0x7f4a1c108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21817700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a1c105ef0 0x7f4a1c1085c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21817700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c1a96b0 con 0x7f4a1c1017c0 2023-10-17T01:13:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21817700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c198db0 con 0x7f4a1c105ef0 2023-10-17T01:13:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21817700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c198f70 con 0x7f4a1c100e10 2023-10-17T01:13:31.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c100e10 0x7f4a1c107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:31.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21016700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a1c105ef0 0x7f4a1c1085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:31.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21016700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a1c105ef0 0x7f4a1c1085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59478/0 (socket says 172.21.15.45:59478) 2023-10-17T01:13:31.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c100e10 0x7f4a1c107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36084/0 (socket says 172.21.15.45:36084) 2023-10-17T01:13:31.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a21016700 1 -- 172.21.15.45:0/1348326031 learned_addr learned my addr 172.21.15.45:0/1348326031 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:31.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 -- 172.21.15.45:0/1348326031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a1c105ef0 msgr2=0x7f4a1c1085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:31.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a1c105ef0 0x7f4a1c1085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 -- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a1c1017c0 msgr2=0x7f4a1c108080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:31.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a1c1017c0 0x7f4a1c108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.566+0000 7f4a20815700 1 -- 172.21.15.45:0/1348326031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080095f0 con 0x7f4a1c100e10 2023-10-17T01:13:31.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.567+0000 7f4a20815700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c100e10 0x7f4a1c107b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e9f0 tx=0x7f4a1000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:31.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.567+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000cbd0 con 0x7f4a1c100e10 2023-10-17T01:13:31.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.567+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c199250 con 0x7f4a1c100e10 2023-10-17T01:13:31.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.568+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c1997a0 con 0x7f4a1c100e10 2023-10-17T01:13:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.568+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a1c110df0 con 0x7f4a1c100e10 2023-10-17T01:13:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.569+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a1000cd30 con 0x7f4a1c100e10 2023-10-17T01:13:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.569+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10018710 con 0x7f4a1c100e10 2023-10-17T01:13:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.572+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f4a10018870 con 0x7f4a1c100e10 2023-10-17T01:13:31.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.574+0000 7f4a19ffb700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4a0c0a0c30 0x7f4a0c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:31.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.574+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4a10014070 con 0x7f4a1c100e10 2023-10-17T01:13:31.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.574+0000 7f4a1bfff700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4a0c0a0c30 0x7f4a0c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:31.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.574+0000 7f4a19ffb700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] conn(0x7f4a0c0aa410 0x7f4a0c0ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:31.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.574+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 --> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a0c0acee0 con 0x7f4a0c0aa410 2023-10-17T01:13:31.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.574+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a100cd210 con 0x7f4a1c100e10 2023-10-17T01:13:31.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.575+0000 7f4a21016700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] conn(0x7f4a0c0aa410 0x7f4a0c0ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:31.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.575+0000 7f4a1bfff700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4a0c0a0c30 0x7f4a0c0a30f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a08006010 tx=0x7f4a08005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:31.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.575+0000 7f4a21016700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] conn(0x7f4a0c0aa410 0x7f4a0c0ac830 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:31.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.577+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== osd.6 v2:172.21.15.45:6818/2740142104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4a0c0acee0 con 0x7f4a0c0aa410 2023-10-17T01:13:31.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.599+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 --> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a1c04f2a0 con 0x7f4a0c0aa410 2023-10-17T01:13:31.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.599+0000 7f4a19ffb700 1 -- 172.21.15.45:0/1348326031 <== osd.6 v2:172.21.15.45:6818/2740142104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4a1c04f2a0 con 0x7f4a0c0aa410 2023-10-17T01:13:31.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] conn(0x7f4a0c0aa410 msgr2=0x7f4a0c0ac830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:31.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] conn(0x7f4a0c0aa410 0x7f4a0c0ac830 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4a0c0a0c30 msgr2=0x7f4a0c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4a0c0a0c30 0x7f4a0c0a30f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a08006010 tx=0x7f4a08005e80 comp rx=0 tx=0).stop 2023-10-17T01:13:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c100e10 msgr2=0x7f4a1c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 --2- 172.21.15.45:0/1348326031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c100e10 0x7f4a1c107b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e9f0 tx=0x7f4a1000edb0 comp rx=0 tx=0).stop 2023-10-17T01:13:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21016700 1 -- 172.21.15.45:0/1348326031 reap_dead start 2023-10-17T01:13:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 shutdown_connections 2023-10-17T01:13:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 >> 172.21.15.45:0/1348326031 conn(0x7f4a1c0fc700 msgr2=0x7f4a1c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:31.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.602+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 shutdown_connections 2023-10-17T01:13:31.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.602+0000 7f4a21817700 1 -- 172.21.15.45:0/1348326031 wait complete. 2023-10-17T01:13:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:31 smithi045 ceph-mon[98474]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-17T01:13:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:31 smithi060 ceph-mon[134136]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-17T01:13:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:31 smithi181 ceph-mon[134520]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-17T01:13:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.599+0000 7fa607b14700 1 -- 172.21.15.45:0/686066850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600068870 msgr2=0x7fa600068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.599+0000 7fa607b14700 1 --2- 172.21.15.45:0/686066850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600068870 0x7fa600068cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b0a0 tx=0x7fa5f800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 -- 172.21.15.45:0/686066850 shutdown_connections 2023-10-17T01:13:32.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 --2- 172.21.15.45:0/686066850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600068870 0x7fa600068cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 --2- 172.21.15.45:0/686066850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa600105d10 0x7fa6001037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 --2- 172.21.15.45:0/686066850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa600103ce0 0x7fa600104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 -- 172.21.15.45:0/686066850 >> 172.21.15.45:0/686066850 conn(0x7fa600076d50 msgr2=0x7fa600077160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:32.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 -- 172.21.15.45:0/686066850 shutdown_connections 2023-10-17T01:13:32.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.600+0000 7fa607b14700 1 -- 172.21.15.45:0/686066850 wait complete. 2023-10-17T01:13:32.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.604+0000 7fa607b14700 1 Processor -- start 2023-10-17T01:13:32.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.605+0000 7fa607b14700 1 -- start start 2023-10-17T01:13:32.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.606+0000 7fa607b14700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa600068870 0x7fa60019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.606+0000 7fa607b14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600103ce0 0x7fa60019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.606+0000 7fa607b14700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa600105d10 0x7fa6001a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.606+0000 7fa6050af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600103ce0 0x7fa60019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.606+0000 7fa6060b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa600105d10 0x7fa6001a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600103ce0 0x7fa60019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36106/0 (socket says 172.21.15.45:36106) 2023-10-17T01:13:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 -- 172.21.15.45:0/3191109617 learned_addr learned my addr 172.21.15.45:0/3191109617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:32.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001a2610 con 0x7fa600105d10 2023-10-17T01:13:32.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001a2780 con 0x7fa600068870 2023-10-17T01:13:32.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001a2940 con 0x7fa600103ce0 2023-10-17T01:13:32.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 -- 172.21.15.45:0/3191109617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa600068870 msgr2=0x7fa60019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa600068870 0x7fa60019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 -- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa600105d10 msgr2=0x7fa6001a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa600105d10 0x7fa6001a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009d00 con 0x7fa600103ce0 2023-10-17T01:13:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.607+0000 7fa6050af700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600103ce0 0x7fa60019db90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b500 tx=0x7fa5f000b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.608+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0011830 con 0x7fa600103ce0 2023-10-17T01:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.608+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f0011e50 con 0x7fa600103ce0 2023-10-17T01:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.608+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001a2c20 con 0x7fa600103ce0 2023-10-17T01:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.608+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001aa480 con 0x7fa600103ce0 2023-10-17T01:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.608+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000f2e0 con 0x7fa600103ce0 2023-10-17T01:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.610+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5e4000ff0 con 0x7fa600103ce0 2023-10-17T01:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.612+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fa5f0011990 con 0x7fa600103ce0 2023-10-17T01:13:32.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.615+0000 7fa5f6ffd700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa5ec0a0c30 0x7fa5ec0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.615+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa5f00cb9d0 con 0x7fa600103ce0 2023-10-17T01:13:32.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.616+0000 7fa5f6ffd700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] conn(0x7fa5ec0aa410 0x7fa5ec0ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.616+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5ec0acee0 con 0x7fa5ec0aa410 2023-10-17T01:13:32.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.616+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5f00d02c0 con 0x7fa600103ce0 2023-10-17T01:13:32.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.616+0000 7fa6058b0700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa5ec0a0c30 0x7fa5ec0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.616+0000 7fa6060b1700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] conn(0x7fa5ec0aa410 0x7fa5ec0ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.617+0000 7fa6060b1700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] conn(0x7fa5ec0aa410 0x7fa5ec0ac830 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:32.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.617+0000 7fa6058b0700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa5ec0a0c30 0x7fa5ec0a30f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc005950 tx=0x7fa5fc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:32.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.617+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== osd.8 v2:172.21.15.181:6816/3559316201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa5ec0acee0 con 0x7fa5ec0aa410 2023-10-17T01:13:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.638+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 --> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa5e4002d70 con 0x7fa5ec0aa410 2023-10-17T01:13:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa5f6ffd700 1 -- 172.21.15.45:0/3191109617 <== osd.8 v2:172.21.15.181:6816/3559316201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa5e4002d70 con 0x7fa5ec0aa410 2023-10-17T01:13:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] conn(0x7fa5ec0aa410 msgr2=0x7fa5ec0ac830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa607b14700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] conn(0x7fa5ec0aa410 0x7fa5ec0ac830 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa5ec0a0c30 msgr2=0x7fa5ec0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa607b14700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa5ec0a0c30 0x7fa5ec0a30f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc005950 tx=0x7fa5fc0058e0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600103ce0 msgr2=0x7fa60019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa607b14700 1 --2- 172.21.15.45:0/3191109617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa600103ce0 0x7fa60019db90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b500 tx=0x7fa5f000b810 comp rx=0 tx=0).stop 2023-10-17T01:13:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.639+0000 7fa6060b1700 1 -- 172.21.15.45:0/3191109617 reap_dead start 2023-10-17T01:13:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.640+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 shutdown_connections 2023-10-17T01:13:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.640+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 >> 172.21.15.45:0/3191109617 conn(0x7fa600076d50 msgr2=0x7fa6000fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.640+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 shutdown_connections 2023-10-17T01:13:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.640+0000 7fa607b14700 1 -- 172.21.15.45:0/3191109617 wait complete. 2023-10-17T01:13:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 -- 172.21.15.45:0/3513374162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8106ee0 msgr2=0x7f94f81072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 --2- 172.21.15.45:0/3513374162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8106ee0 0x7f94f81072c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f94f40098f0 tx=0x7f94f4009c00 comp rx=0 tx=0).stop 2023-10-17T01:13:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 -- 172.21.15.45:0/3513374162 shutdown_connections 2023-10-17T01:13:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 --2- 172.21.15.45:0/3513374162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8069230 0x7f94f810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 --2- 172.21.15.45:0/3513374162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8106ee0 0x7f94f81072c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 --2- 172.21.15.45:0/3513374162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f8068870 0x7f94f8068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 -- 172.21.15.45:0/3513374162 >> 172.21.15.45:0/3513374162 conn(0x7f94f8076d50 msgr2=0x7f94f8077160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 -- 172.21.15.45:0/3513374162 shutdown_connections 2023-10-17T01:13:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.351+0000 7f9500711700 1 -- 172.21.15.45:0/3513374162 wait complete. 2023-10-17T01:13:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.352+0000 7f9500711700 1 Processor -- start 2023-10-17T01:13:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.352+0000 7f9500711700 1 -- start start 2023-10-17T01:13:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f9500711700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8068870 0x7f94f80ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f9500711700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8069230 0x7f94f8100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f9500711700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f8106ee0 0x7f94f81007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f9500711700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81a5280 con 0x7f94f8068870 2023-10-17T01:13:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f9500711700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81a53f0 con 0x7f94f8106ee0 2023-10-17T01:13:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f9500711700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81a55b0 con 0x7f94f8069230 2023-10-17T01:13:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f94fe4ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8068870 0x7f94f80ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f94fe4ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8068870 0x7f94f80ffd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40308/0 (socket says 172.21.15.45:40308) 2023-10-17T01:13:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f94fe4ad700 1 -- 172.21.15.45:0/817069454 learned_addr learned my addr 172.21.15.45:0/817069454 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f94fdcac700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8069230 0x7f94f8100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.353+0000 7f94fecae700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f8106ee0 0x7f94f81007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fe4ad700 1 -- 172.21.15.45:0/817069454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8069230 msgr2=0x7f94f8100240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fe4ad700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f8069230 0x7f94f8100240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fe4ad700 1 -- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f8106ee0 msgr2=0x7f94f81007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fe4ad700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f8106ee0 0x7f94f81007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fe4ad700 1 -- 172.21.15.45:0/817069454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f4009520 con 0x7f94f8068870 2023-10-17T01:13:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fecae700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f8106ee0 0x7f94f81007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94fe4ad700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8068870 0x7f94f80ffd00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009810 tx=0x7f94f400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f401c070 con 0x7f94f8068870 2023-10-17T01:13:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f400bbd0 con 0x7f94f8068870 2023-10-17T01:13:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f81a5830 con 0x7f94f8068870 2023-10-17T01:13:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40049c0 con 0x7f94f8068870 2023-10-17T01:13:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.354+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f81a5c40 con 0x7f94f8068870 2023-10-17T01:13:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.356+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94f804ea50 con 0x7f94f8068870 2023-10-17T01:13:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.357+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f94f4020070 con 0x7f94f8068870 2023-10-17T01:13:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.360+0000 7f94ef7fe700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f94e40a0bb0 0x7f94e40a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.360+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f94f40ccf70 con 0x7f94f8068870 2023-10-17T01:13:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.360+0000 7f94fdcac700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f94e40a0bb0 0x7f94e40a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.360+0000 7f94ef7fe700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] conn(0x7f94e40aa390 0x7f94e40ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.360+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 --> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94e40ace60 con 0x7f94e40aa390 2023-10-17T01:13:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.360+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94f40cf550 con 0x7f94f8068870 2023-10-17T01:13:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.361+0000 7f94fdcac700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f94e40a0bb0 0x7f94e40a3070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f94f81014f0 tx=0x7f94e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.361+0000 7f94fecae700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] conn(0x7f94e40aa390 0x7f94e40ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.361+0000 7f94fecae700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] conn(0x7f94e40aa390 0x7f94e40ac7b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.362+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== osd.1 v2:172.21.15.60:6800/1446433682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f94e40ace60 con 0x7f94e40aa390 2023-10-17T01:13:32.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.382+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 --> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94f8110fc0 con 0x7f94e40aa390 2023-10-17T01:13:32.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.383+0000 7f94ef7fe700 1 -- 172.21.15.45:0/817069454 <== osd.1 v2:172.21.15.60:6800/1446433682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f94f000f040 con 0x7f94e40aa390 2023-10-17T01:13:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.383+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] conn(0x7f94e40aa390 msgr2=0x7f94e40ac7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.383+0000 7f9500711700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] conn(0x7f94e40aa390 0x7f94e40ac7b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f94e40a0bb0 msgr2=0x7f94e40a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f94e40a0bb0 0x7f94e40a3070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f94f81014f0 tx=0x7f94e8009260 comp rx=0 tx=0).stop 2023-10-17T01:13:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8068870 msgr2=0x7f94f80ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 --2- 172.21.15.45:0/817069454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94f8068870 0x7f94f80ffd00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009810 tx=0x7f94f400fa00 comp rx=0 tx=0).stop 2023-10-17T01:13:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f94fecae700 1 -- 172.21.15.45:0/817069454 reap_dead start 2023-10-17T01:13:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 shutdown_connections 2023-10-17T01:13:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 >> 172.21.15.45:0/817069454 conn(0x7f94f8076d50 msgr2=0x7f94f80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 shutdown_connections 2023-10-17T01:13:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:32.384+0000 7f9500711700 1 -- 172.21.15.45:0/817069454 wait complete. 2023-10-17T01:13:32.762 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-17T01:13:32.762 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-17T01:13:32.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-17T01:13:32.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-10-17T01:13:32.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-10-17T01:13:32.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-10-17T01:13:32.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:32 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-10-17T01:13:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-17T01:13:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-17T01:13:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-17T01:13:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-10-17T01:13:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-10-17T01:13:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-10-17T01:13:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:32 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-10-17T01:13:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-17T01:13:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-17T01:13:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-17T01:13:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-10-17T01:13:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-10-17T01:13:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-10-17T01:13:32.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:32 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-10-17T01:13:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:197568495657 2023-10-17T01:13:33.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-17T01:13:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 -- 172.21.15.45:0/1965435354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 msgr2=0x7f3f241059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1965435354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f241059d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009990 tx=0x7f3f14009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 -- 172.21.15.45:0/1965435354 shutdown_connections 2023-10-17T01:13:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1965435354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f241060b0 0x7f3f2410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1965435354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100fd0 0x7f3f241013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1965435354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f241059d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 -- 172.21.15.45:0/1965435354 >> 172.21.15.45:0/1965435354 conn(0x7f3f240fc840 msgr2=0x7f3f240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 -- 172.21.15.45:0/1965435354 shutdown_connections 2023-10-17T01:13:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.595+0000 7f3f2b605700 1 -- 172.21.15.45:0/1965435354 wait complete. 2023-10-17T01:13:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 Processor -- start 2023-10-17T01:13:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 -- start start 2023-10-17T01:13:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100fd0 0x7f3f24196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f24197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f241060b0 0x7f3f2419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2419bf10 con 0x7f3f24101980 2023-10-17T01:13:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2419c080 con 0x7f3f241060b0 2023-10-17T01:13:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.596+0000 7f3f2b605700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2419c240 con 0x7f3f24100fd0 2023-10-17T01:13:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f24197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f24197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40320/0 (socket says 172.21.15.45:40320) 2023-10-17T01:13:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 -- 172.21.15.45:0/1780743798 learned_addr learned my addr 172.21.15.45:0/1780743798 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f29ba2700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f241060b0 0x7f3f2419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f293a1700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100fd0 0x7f3f24196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 -- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100fd0 msgr2=0x7f3f24196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f24100fd0 0x7f3f24196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 -- 172.21.15.45:0/1780743798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f241060b0 msgr2=0x7f3f2419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f241060b0 0x7f3f2419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 -- 172.21.15.45:0/1780743798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c009c40 con 0x7f3f24101980 2023-10-17T01:13:33.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f29ba2700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f241060b0 0x7f3f2419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:33.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.597+0000 7f3f28ba0700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f24197490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f14004730 tx=0x7f3f14004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:33.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.598+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1401d070 con 0x7f3f24101980 2023-10-17T01:13:33.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.598+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f140095f0 con 0x7f3f24101980 2023-10-17T01:13:33.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.598+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f140047d0 con 0x7f3f24101980 2023-10-17T01:13:33.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.598+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f140053a0 con 0x7f3f24101980 2023-10-17T01:13:33.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.598+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2419c740 con 0x7f3f24101980 2023-10-17T01:13:33.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.599+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f24110fb0 con 0x7f3f24101980 2023-10-17T01:13:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.601+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f3f14004940 con 0x7f3f24101980 2023-10-17T01:13:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.602+0000 7f3f1a7fc700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3f100a0bb0 0x7f3f100a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.602+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3f140cda20 con 0x7f3f24101980 2023-10-17T01:13:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.602+0000 7f3f293a1700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3f100a0bb0 0x7f3f100a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.602+0000 7f3f1a7fc700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] conn(0x7f3f100aa670 0x7f3f100acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.602+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 --> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f100ad1e0 con 0x7f3f100aa670 2023-10-17T01:13:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.603+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f140cdd90 con 0x7f3f24101980 2023-10-17T01:13:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.603+0000 7f3f293a1700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3f100a0bb0 0x7f3f100a3070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005950 tx=0x7f3f2000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.603+0000 7f3f29ba2700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] conn(0x7f3f100aa670 0x7f3f100acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.604+0000 7f3f29ba2700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] conn(0x7f3f100aa670 0x7f3f100acb10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.605+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== osd.5 v2:172.21.15.181:6808/547299005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3f100ad1e0 con 0x7f3f100aa670 2023-10-17T01:13:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.626+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 --> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f2404ea50 con 0x7f3f100aa670 2023-10-17T01:13:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.627+0000 7f3f1a7fc700 1 -- 172.21.15.45:0/1780743798 <== osd.5 v2:172.21.15.181:6808/547299005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3f2404ea50 con 0x7f3f100aa670 2023-10-17T01:13:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] conn(0x7f3f100aa670 msgr2=0x7f3f100acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] conn(0x7f3f100aa670 0x7f3f100acb10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3f100a0bb0 msgr2=0x7f3f100a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3f100a0bb0 0x7f3f100a3070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005950 tx=0x7f3f2000b410 comp rx=0 tx=0).stop 2023-10-17T01:13:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 msgr2=0x7f3f24197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 --2- 172.21.15.45:0/1780743798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f24101980 0x7f3f24197490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f14004730 tx=0x7f3f14004760 comp rx=0 tx=0).stop 2023-10-17T01:13:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f29ba2700 1 -- 172.21.15.45:0/1780743798 reap_dead start 2023-10-17T01:13:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 shutdown_connections 2023-10-17T01:13:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 >> 172.21.15.45:0/1780743798 conn(0x7f3f240fc840 msgr2=0x7f3f240fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 shutdown_connections 2023-10-17T01:13:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:33.628+0000 7f3f2b605700 1 -- 172.21.15.45:0/1780743798 wait complete. 2023-10-17T01:13:33.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3370745024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a80100630 msgr2=0x7f7a80104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3370745024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a80100630 0x7f7a80104b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009990 tx=0x7f7a70009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3370745024 shutdown_connections 2023-10-17T01:13:33.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3370745024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a80105120 0x7f7a801055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3370745024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a800ffd10 0x7f7a801000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3370745024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a80100630 0x7f7a80104b00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3370745024 >> 172.21.15.45:0/3370745024 conn(0x7f7a800fb710 msgr2=0x7f7a800fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:33.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3370745024 shutdown_connections 2023-10-17T01:13:33.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.760+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3370745024 wait complete. 2023-10-17T01:13:33.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.761+0000 7f7a87e5b700 1 Processor -- start 2023-10-17T01:13:33.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 -- start start 2023-10-17T01:13:33.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a800ffd10 0x7f7a8019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80100630 0x7f7a8019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a80105120 0x7f7a801a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801a2570 con 0x7f7a80105120 2023-10-17T01:13:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801a26e0 con 0x7f7a800ffd10 2023-10-17T01:13:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a87e5b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801a28a0 con 0x7f7a80100630 2023-10-17T01:13:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a85bf7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a800ffd10 0x7f7a8019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a85bf7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a800ffd10 0x7f7a8019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59532/0 (socket says 172.21.15.45:59532) 2023-10-17T01:13:33.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a85bf7700 1 -- 172.21.15.45:0/3355969723 learned_addr learned my addr 172.21.15.45:0/3355969723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:33.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a863f8700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a80105120 0x7f7a801a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.762+0000 7f7a853f6700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80100630 0x7f7a8019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a85bf7700 1 -- 172.21.15.45:0/3355969723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80100630 msgr2=0x7f7a8019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a85bf7700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80100630 0x7f7a8019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a85bf7700 1 -- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a80105120 msgr2=0x7f7a801a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a85bf7700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a80105120 0x7f7a801a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a85bf7700 1 -- 172.21.15.45:0/3355969723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700095f0 con 0x7f7a800ffd10 2023-10-17T01:13:33.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a85bf7700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a800ffd10 0x7f7a8019d5b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00e9a0 tx=0x7f7a7c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:33.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c009790 con 0x7f7a800ffd10 2023-10-17T01:13:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a7c010430 con 0x7f7a800ffd10 2023-10-17T01:13:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a801a2b80 con 0x7f7a800ffd10 2023-10-17T01:13:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.763+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c018680 con 0x7f7a800ffd10 2023-10-17T01:13:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.764+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a801aa3e0 con 0x7f7a800ffd10 2023-10-17T01:13:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.764+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a801175c0 con 0x7f7a800ffd10 2023-10-17T01:13:33.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.767+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f7a7c0187e0 con 0x7f7a800ffd10 2023-10-17T01:13:33.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.768+0000 7f7a76ffd700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a6c0a0c30 0x7f7a6c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.769+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7a7c014070 con 0x7f7a800ffd10 2023-10-17T01:13:33.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.769+0000 7f7a853f6700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a6c0a0c30 0x7f7a6c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.769+0000 7f7a76ffd700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] conn(0x7f7a6c0aa410 0x7f7a6c0ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:33.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.769+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 --> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a6c0acee0 con 0x7f7a6c0aa410 2023-10-17T01:13:33.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.769+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a7c0cdab0 con 0x7f7a800ffd10 2023-10-17T01:13:33.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.770+0000 7f7a863f8700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] conn(0x7f7a6c0aa410 0x7f7a6c0ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:33.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.770+0000 7f7a853f6700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a6c0a0c30 0x7f7a6c0a30f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a8019ebd0 tx=0x7f7a70005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:33.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.770+0000 7f7a863f8700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] conn(0x7f7a6c0aa410 0x7f7a6c0ac830 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:33.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.771+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== osd.7 v2:172.21.15.60:6816/4084805853 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7a6c0acee0 con 0x7f7a6c0aa410 2023-10-17T01:13:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.789+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 --> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a8004ea50 con 0x7f7a6c0aa410 2023-10-17T01:13:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.789+0000 7f7a76ffd700 1 -- 172.21.15.45:0/3355969723 <== osd.7 v2:172.21.15.60:6816/4084805853 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7a8004ea50 con 0x7f7a6c0aa410 2023-10-17T01:13:33.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] conn(0x7f7a6c0aa410 msgr2=0x7f7a6c0ac830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] conn(0x7f7a6c0aa410 0x7f7a6c0ac830 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a6c0a0c30 msgr2=0x7f7a6c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7a6c0a0c30 0x7f7a6c0a30f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a8019ebd0 tx=0x7f7a70005fb0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a800ffd10 msgr2=0x7f7a8019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:33.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 --2- 172.21.15.45:0/3355969723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a800ffd10 0x7f7a8019d5b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00e9a0 tx=0x7f7a7c00ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:33.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a863f8700 1 -- 172.21.15.45:0/3355969723 reap_dead start 2023-10-17T01:13:33.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 shutdown_connections 2023-10-17T01:13:33.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 >> 172.21.15.45:0/3355969723 conn(0x7f7a800fb710 msgr2=0x7f7a80102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:33.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 shutdown_connections 2023-10-17T01:13:33.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:31.790+0000 7f7a87e5b700 1 -- 172.21.15.45:0/3355969723 wait complete. 2023-10-17T01:13:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2023-10-17T01:13:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-17T01:13:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-17T01:13:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-17T01:13:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-10-17T01:13:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-10-17T01:13:33.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-10-17T01:13:33.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-10-17T01:13:33.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:33 smithi045 ceph-mon[98474]: osdmap e84: 12 total, 12 up, 12 in 2023-10-17T01:13:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2023-10-17T01:13:33.915 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-17T01:13:33.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-17T01:13:33.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-17T01:13:33.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-10-17T01:13:33.916 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-10-17T01:13:33.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-10-17T01:13:33.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-10-17T01:13:33.917 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:33 smithi060 ceph-mon[134136]: osdmap e84: 12 total, 12 up, 12 in 2023-10-17T01:13:33.918 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2023-10-17T01:13:33.918 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-17T01:13:33.918 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-17T01:13:33.918 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-17T01:13:33.918 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-10-17T01:13:33.919 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-10-17T01:13:33.919 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-10-17T01:13:33.919 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-10-17T01:13:33.919 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:33 smithi181 ceph-mon[134520]: osdmap e84: 12 total, 12 up, 12 in 2023-10-17T01:13:34.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 -- 172.21.15.45:0/765023311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6898100fd0 msgr2=0x7f68981013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:34.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 --2- 172.21.15.45:0/765023311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6898100fd0 0x7f68981013b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6894009990 tx=0x7f6894009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 -- 172.21.15.45:0/765023311 shutdown_connections 2023-10-17T01:13:34.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 --2- 172.21.15.45:0/765023311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68981060b0 0x7f689810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 --2- 172.21.15.45:0/765023311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6898100fd0 0x7f68981013b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 --2- 172.21.15.45:0/765023311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898101980 0x7f68981059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 -- 172.21.15.45:0/765023311 >> 172.21.15.45:0/765023311 conn(0x7f68980fc840 msgr2=0x7f68980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:34.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 -- 172.21.15.45:0/765023311 shutdown_connections 2023-10-17T01:13:34.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.036+0000 7f689fb09700 1 -- 172.21.15.45:0/765023311 wait complete. 2023-10-17T01:13:34.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 Processor -- start 2023-10-17T01:13:34.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 -- start start 2023-10-17T01:13:34.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898100fd0 0x7f68981990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:34.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6898101980 0x7f6898199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:34.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68981060b0 0x7f689819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:34.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689819e090 con 0x7f68981060b0 2023-10-17T01:13:34.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.037+0000 7f689fb09700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689819e200 con 0x7f6898101980 2023-10-17T01:13:34.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689fb09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689819e3c0 con 0x7f6898100fd0 2023-10-17T01:13:34.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68981060b0 0x7f689819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:34.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68981060b0 0x7f689819da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40346/0 (socket says 172.21.15.45:40346) 2023-10-17T01:13:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 -- 172.21.15.45:0/3996969963 learned_addr learned my addr 172.21.15.45:0/3996969963 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689d0a4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6898101980 0x7f6898199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689d8a5700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898100fd0 0x7f68981990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 -- 172.21.15.45:0/3996969963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898100fd0 msgr2=0x7f68981990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898100fd0 0x7f68981990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 -- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6898101980 msgr2=0x7f6898199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:34.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6898101980 0x7f6898199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 -- 172.21.15.45:0/3996969963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68940095f0 con 0x7f68981060b0 2023-10-17T01:13:34.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.038+0000 7f689e0a6700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68981060b0 0x7f689819da80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6890007ee0 tx=0x7f689000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:34.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.039+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f68981060b0 2023-10-17T01:13:34.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.039+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689000f040 con 0x7f68981060b0 2023-10-17T01:13:34.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.039+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689819e6a0 con 0x7f68981060b0 2023-10-17T01:13:34.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.039+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890005580 con 0x7f68981060b0 2023-10-17T01:13:34.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.039+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68981a5f00 con 0x7f68981060b0 2023-10-17T01:13:34.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.040+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6898111030 con 0x7f68981060b0 2023-10-17T01:13:34.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.042+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f689000db20 con 0x7f68981060b0 2023-10-17T01:13:34.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.043+0000 7f688effd700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68840a0bb0 0x7f68840a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:34.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.043+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68900cc120 con 0x7f68981060b0 2023-10-17T01:13:34.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.044+0000 7f689d8a5700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68840a0bb0 0x7f68840a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:34.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.044+0000 7f688effd700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] conn(0x7f68840aa670 0x7f68840acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:34.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.044+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 --> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68840ad1e0 con 0x7f68840aa670 2023-10-17T01:13:34.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.044+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68900cc490 con 0x7f68981060b0 2023-10-17T01:13:34.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.044+0000 7f689d0a4700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] conn(0x7f68840aa670 0x7f68840acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:34.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.044+0000 7f689d8a5700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68840a0bb0 0x7f68840a3070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f689400b2e0 tx=0x7f6894005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:34.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.045+0000 7f689d0a4700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] conn(0x7f68840aa670 0x7f68840acb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:34.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.045+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== osd.4 v2:172.21.15.60:6808/2865514976 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f68840ad1e0 con 0x7f68840aa670 2023-10-17T01:13:34.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.062+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 --> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f689804ea50 con 0x7f68840aa670 2023-10-17T01:13:34.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f688effd700 1 -- 172.21.15.45:0/3996969963 <== osd.4 v2:172.21.15.60:6808/2865514976 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f689804ea50 con 0x7f68840aa670 2023-10-17T01:13:34.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] conn(0x7f68840aa670 msgr2=0x7f68840acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689fb09700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] conn(0x7f68840aa670 0x7f68840acb10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68840a0bb0 msgr2=0x7f68840a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689fb09700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68840a0bb0 0x7f68840a3070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f689400b2e0 tx=0x7f6894005fd0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68981060b0 msgr2=0x7f689819da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689fb09700 1 --2- 172.21.15.45:0/3996969963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68981060b0 0x7f689819da80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6890007ee0 tx=0x7f689000d3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.063+0000 7f689e0a6700 1 -- 172.21.15.45:0/3996969963 reap_dead start 2023-10-17T01:13:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.064+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 shutdown_connections 2023-10-17T01:13:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.064+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 >> 172.21.15.45:0/3996969963 conn(0x7f68980fc840 msgr2=0x7f68980fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.064+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 shutdown_connections 2023-10-17T01:13:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:34.064+0000 7f689fb09700 1 -- 172.21.15.45:0/3996969963 wait complete. 2023-10-17T01:13:34.660 INFO:teuthology.orchestra.run.smithi045.stdout:55834574910 2023-10-17T01:13:34.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-17T01:13:34.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:34 smithi045 ceph-mon[98474]: osdmap e85: 12 total, 12 up, 12 in 2023-10-17T01:13:34.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:34 smithi060 ceph-mon[134136]: osdmap e85: 12 total, 12 up, 12 in 2023-10-17T01:13:34.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:34 smithi181 ceph-mon[134520]: osdmap e85: 12 total, 12 up, 12 in 2023-10-17T01:13:35.106 INFO:teuthology.orchestra.run.smithi045.stdout:163208757293 2023-10-17T01:13:35.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-17T01:13:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 -- 172.21.15.45:0/3075768850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 msgr2=0x7efe141013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:35.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3075768850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe141013b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe040098f0 tx=0x7efe04009c00 comp rx=0 tx=0).stop 2023-10-17T01:13:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 -- 172.21.15.45:0/3075768850 shutdown_connections 2023-10-17T01:13:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3075768850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe141060b0 0x7efe1410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3075768850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe141013b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3075768850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe14101980 0x7efe141059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 -- 172.21.15.45:0/3075768850 >> 172.21.15.45:0/3075768850 conn(0x7efe140fc840 msgr2=0x7efe140fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:35.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 -- 172.21.15.45:0/3075768850 shutdown_connections 2023-10-17T01:13:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.240+0000 7efe1a22e700 1 -- 172.21.15.45:0/3075768850 wait complete. 2023-10-17T01:13:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.242+0000 7efe1a22e700 1 Processor -- start 2023-10-17T01:13:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.242+0000 7efe1a22e700 1 -- start start 2023-10-17T01:13:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe1a22e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe14199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe1a22e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe14101980 0x7efe14199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe1a22e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe141060b0 0x7efe1419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe1a22e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419e0d0 con 0x7efe14100fd0 2023-10-17T01:13:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe1a22e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419e240 con 0x7efe14101980 2023-10-17T01:13:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe1a22e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419e400 con 0x7efe141060b0 2023-10-17T01:13:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe14199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe14199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40376/0 (socket says 172.21.15.45:40376) 2023-10-17T01:13:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 -- 172.21.15.45:0/3414581213 learned_addr learned my addr 172.21.15.45:0/3414581213 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe12ffd700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe14101980 0x7efe14199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe13fff700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe141060b0 0x7efe1419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 -- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe141060b0 msgr2=0x7efe1419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe141060b0 0x7efe1419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 -- 172.21.15.45:0/3414581213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe14101980 msgr2=0x7efe14199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.243+0000 7efe137fe700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe14101980 0x7efe14199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe137fe700 1 -- 172.21.15.45:0/3414581213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04009520 con 0x7efe14100fd0 2023-10-17T01:13:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe137fe700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe14199110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe04009810 tx=0x7efe0400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0401c070 con 0x7efe14100fd0 2023-10-17T01:13:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe04005040 con 0x7efe14100fd0 2023-10-17T01:13:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1419e680 con 0x7efe14100fd0 2023-10-17T01:13:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040049a0 con 0x7efe14100fd0 2023-10-17T01:13:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.244+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe141a5f40 con 0x7efe14100fd0 2023-10-17T01:13:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.245+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe14110fd0 con 0x7efe14100fd0 2023-10-17T01:13:35.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.247+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7efe04020070 con 0x7efe14100fd0 2023-10-17T01:13:35.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.250+0000 7efe10ff9700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efdfc0a0bb0 0x7efdfc0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:35.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.250+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efe040cd760 con 0x7efe14100fd0 2023-10-17T01:13:35.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.250+0000 7efe12ffd700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efdfc0a0bb0 0x7efdfc0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:35.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.250+0000 7efe10ff9700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] conn(0x7efdfc0aa670 0x7efdfc0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:35.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.251+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 --> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdfc0ad1e0 con 0x7efdfc0aa670 2023-10-17T01:13:35.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.251+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe040cdad0 con 0x7efe14100fd0 2023-10-17T01:13:35.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.251+0000 7efe12ffd700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efdfc0a0bb0 0x7efdfc0a3070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe1419a730 tx=0x7efe08009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:35.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.251+0000 7efe13fff700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] conn(0x7efdfc0aa670 0x7efdfc0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:35.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.251+0000 7efe13fff700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] conn(0x7efdfc0aa670 0x7efdfc0acb10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.252+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== osd.11 v2:172.21.15.181:6824/429412155 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efdfc0ad1e0 con 0x7efdfc0aa670 2023-10-17T01:13:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.269+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 --> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe1419e950 con 0x7efdfc0aa670 2023-10-17T01:13:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe10ff9700 1 -- 172.21.15.45:0/3414581213 <== osd.11 v2:172.21.15.181:6824/429412155 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efe0c00f040 con 0x7efdfc0aa670 2023-10-17T01:13:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] conn(0x7efdfc0aa670 msgr2=0x7efdfc0acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] conn(0x7efdfc0aa670 0x7efdfc0acb10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efdfc0a0bb0 msgr2=0x7efdfc0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efdfc0a0bb0 0x7efdfc0a3070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe1419a730 tx=0x7efe08009310 comp rx=0 tx=0).stop 2023-10-17T01:13:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 msgr2=0x7efe14199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 --2- 172.21.15.45:0/3414581213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14100fd0 0x7efe14199110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe04009810 tx=0x7efe0400fa70 comp rx=0 tx=0).stop 2023-10-17T01:13:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe13fff700 1 -- 172.21.15.45:0/3414581213 reap_dead start 2023-10-17T01:13:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 shutdown_connections 2023-10-17T01:13:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.270+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 >> 172.21.15.45:0/3414581213 conn(0x7efe140fc840 msgr2=0x7efe140fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.271+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 shutdown_connections 2023-10-17T01:13:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.271+0000 7efe1a22e700 1 -- 172.21.15.45:0/3414581213 wait complete. 2023-10-17T01:13:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:35 smithi045 ceph-mon[98474]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 639 B/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2023-10-17T01:13:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:35 smithi060 ceph-mon[134136]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 639 B/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2023-10-17T01:13:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:35 smithi181 ceph-mon[134520]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 639 B/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2023-10-17T01:13:35.961 INFO:teuthology.orchestra.run.smithi045.stdout:249108103199 2023-10-17T01:13:35.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-17T01:13:35.962 INFO:teuthology.orchestra.run.smithi045.stdout:141733920818 2023-10-17T01:13:35.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-17T01:13:36.118 INFO:teuthology.orchestra.run.smithi045.stdout:219043332132 2023-10-17T01:13:36.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-17T01:13:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 -- 172.21.15.45:0/3838134355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073a40 msgr2=0x7fa95010d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3838134355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073a40 0x7fa95010d0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa944009990 tx=0x7fa944009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 -- 172.21.15.45:0/3838134355 shutdown_connections 2023-10-17T01:13:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3838134355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa95010d610 0x7fa95010fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3838134355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa950073120 0x7fa950073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3838134355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073a40 0x7fa95010d0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.124+0000 7fa955c9a700 1 -- 172.21.15.45:0/3838134355 >> 172.21.15.45:0/3838134355 conn(0x7fa9500fc840 msgr2=0x7fa9500fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.126+0000 7fa955c9a700 1 -- 172.21.15.45:0/3838134355 shutdown_connections 2023-10-17T01:13:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.126+0000 7fa955c9a700 1 -- 172.21.15.45:0/3838134355 wait complete. 2023-10-17T01:13:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.127+0000 7fa955c9a700 1 Processor -- start 2023-10-17T01:13:36.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.127+0000 7fa955c9a700 1 -- start start 2023-10-17T01:13:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.128+0000 7fa955c9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073120 0x7fa950199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.128+0000 7fa955c9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa950073a40 0x7fa950199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.128+0000 7fa955c9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95010d610 0x7fa95019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.128+0000 7fa955c9a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95019e210 con 0x7fa95010d610 2023-10-17T01:13:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.128+0000 7fa955c9a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95019e380 con 0x7fa950073120 2023-10-17T01:13:36.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.128+0000 7fa955c9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95019e540 con 0x7fa950073a40 2023-10-17T01:13:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.129+0000 7fa94f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073120 0x7fa950199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.129+0000 7fa94f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073120 0x7fa950199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59644/0 (socket says 172.21.15.45:59644) 2023-10-17T01:13:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.129+0000 7fa94f7fe700 1 -- 172.21.15.45:0/3968369150 learned_addr learned my addr 172.21.15.45:0/3968369150 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.129+0000 7fa94ffff700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95010d610 0x7fa95019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.129+0000 7fa94effd700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa950073a40 0x7fa950199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.130+0000 7fa94ffff700 1 -- 172.21.15.45:0/3968369150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa950073a40 msgr2=0x7fa950199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.130+0000 7fa94ffff700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa950073a40 0x7fa950199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.130+0000 7fa94ffff700 1 -- 172.21.15.45:0/3968369150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073120 msgr2=0x7fa950199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.130+0000 7fa94ffff700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa950073120 0x7fa950199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.130+0000 7fa94ffff700 1 -- 172.21.15.45:0/3968369150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9440095f0 con 0x7fa95010d610 2023-10-17T01:13:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.130+0000 7fa94ffff700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95010d610 0x7fa95019dc00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa94800bad0 tx=0x7fa94800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.131+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800c930 con 0x7fa95010d610 2023-10-17T01:13:36.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.131+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95019e820 con 0x7fa95010d610 2023-10-17T01:13:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.131+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa948014440 con 0x7fa95010d610 2023-10-17T01:13:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.131+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480125a0 con 0x7fa95010d610 2023-10-17T01:13:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.132+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9501a6080 con 0x7fa95010d610 2023-10-17T01:13:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.133+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa950113340 con 0x7fa95010d610 2023-10-17T01:13:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.138+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fa9480148b0 con 0x7fa95010d610 2023-10-17T01:13:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.140+0000 7fa94cff9700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9380a0cf0 0x7fa9380a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.141+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9480cc430 con 0x7fa95010d610 2023-10-17T01:13:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.141+0000 7fa94cff9700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] conn(0x7fa9380aa7b0 0x7fa9380acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.141+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 --> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9380ad320 con 0x7fa9380aa7b0 2023-10-17T01:13:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.141+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9480cc7e0 con 0x7fa95010d610 2023-10-17T01:13:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.142+0000 7fa94f7fe700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9380a0cf0 0x7fa9380a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.142+0000 7fa94f7fe700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9380a0cf0 0x7fa9380a31b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa940009ad0 tx=0x7fa940009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.143+0000 7fa94effd700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] conn(0x7fa9380aa7b0 0x7fa9380acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.144+0000 7fa94effd700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] conn(0x7fa9380aa7b0 0x7fa9380acc50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.146+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== osd.9 v2:172.21.15.45:6826/1619950386 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa9380ad320 con 0x7fa9380aa7b0 2023-10-17T01:13:36.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 -- 172.21.15.45:0/3461519669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464068870 msgr2=0x7f1464068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 --2- 172.21.15.45:0/3461519669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464068870 0x7f1464068cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1458009990 tx=0x7f1458009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 -- 172.21.15.45:0/3461519669 shutdown_connections 2023-10-17T01:13:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 --2- 172.21.15.45:0/3461519669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464069230 0x7f146410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 --2- 172.21.15.45:0/3461519669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464106ee0 0x7f14641072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 --2- 172.21.15.45:0/3461519669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464068870 0x7f1464068cf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 -- 172.21.15.45:0/3461519669 >> 172.21.15.45:0/3461519669 conn(0x7f1464076d50 msgr2=0x7f1464077160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 -- 172.21.15.45:0/3461519669 shutdown_connections 2023-10-17T01:13:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.169+0000 7f146a14c700 1 -- 172.21.15.45:0/3461519669 wait complete. 2023-10-17T01:13:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.171+0000 7f146a14c700 1 Processor -- start 2023-10-17T01:13:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.171+0000 7f146a14c700 1 -- start start 2023-10-17T01:13:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.171+0000 7f146a14c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464068870 0x7f14640ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.171+0000 7f146a14c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464069230 0x7f1464100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f146a14c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464106ee0 0x7f14641007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f146a14c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641a5260 con 0x7f1464106ee0 2023-10-17T01:13:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f146a14c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641a53d0 con 0x7f1464068870 2023-10-17T01:13:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f146a14c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641a5590 con 0x7f1464069230 2023-10-17T01:13:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464068870 0x7f14640ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464068870 0x7f14640ffd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59608/0 (socket says 172.21.15.45:59608) 2023-10-17T01:13:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 -- 172.21.15.45:0/3724482241 learned_addr learned my addr 172.21.15.45:0/3724482241 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f1462ffd700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464069230 0x7f1464100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f1463fff700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464106ee0 0x7f14641007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 -- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464069230 msgr2=0x7f1464100240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464069230 0x7f1464100240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 -- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464106ee0 msgr2=0x7f14641007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464106ee0 0x7f14641007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f14637fe700 1 -- 172.21.15.45:0/3724482241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580095f0 con 0x7f1464068870 2023-10-17T01:13:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f1463fff700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1464106ee0 0x7f14641007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.172+0000 7f1462ffd700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464069230 0x7f1464100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.173+0000 7f14637fe700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464068870 0x7f14640ffd00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f145400e9a0 tx=0x7f145400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.173+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454009790 con 0x7f1464068870 2023-10-17T01:13:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.173+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145400cbd0 con 0x7f1464068870 2023-10-17T01:13:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.173+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14641a5870 con 0x7f1464068870 2023-10-17T01:13:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.173+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454018680 con 0x7f1464068870 2023-10-17T01:13:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.173+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641a5d70 con 0x7f1464068870 2023-10-17T01:13:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.174+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f146404ea50 con 0x7f1464068870 2023-10-17T01:13:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.178+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f145400cd40 con 0x7f1464068870 2023-10-17T01:13:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.179+0000 7f1460ff9700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f144c0a0bb0 0x7f144c0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.179+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1454014070 con 0x7f1464068870 2023-10-17T01:13:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.180+0000 7f1460ff9700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] conn(0x7f144c0aa670 0x7f144c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.180+0000 7f1462ffd700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f144c0a0bb0 0x7f144c0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.180+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 --> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f144c0ad1e0 con 0x7f144c0aa670 2023-10-17T01:13:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.180+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14540ccd10 con 0x7f1464068870 2023-10-17T01:13:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.180+0000 7f1463fff700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] conn(0x7f144c0aa670 0x7f144c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.181+0000 7f1462ffd700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f144c0a0bb0 0x7f144c0a3070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f14641014f0 tx=0x7f1458005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.181+0000 7f1463fff700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] conn(0x7f144c0aa670 0x7f144c0acb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.182+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== osd.2 v2:172.21.15.181:6800/2332429544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f144c0ad1e0 con 0x7f144c0aa670 2023-10-17T01:13:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.199+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 --> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1464111000 con 0x7f144c0aa670 2023-10-17T01:13:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.200+0000 7f1460ff9700 1 -- 172.21.15.45:0/3724482241 <== osd.2 v2:172.21.15.181:6800/2332429544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f145c00f040 con 0x7f144c0aa670 2023-10-17T01:13:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] conn(0x7f144c0aa670 msgr2=0x7f144c0acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] conn(0x7f144c0aa670 0x7f144c0acb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f144c0a0bb0 msgr2=0x7f144c0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f144c0a0bb0 0x7f144c0a3070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f14641014f0 tx=0x7f1458005fb0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464068870 msgr2=0x7f14640ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 --2- 172.21.15.45:0/3724482241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1464068870 0x7f14640ffd00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f145400e9a0 tx=0x7f145400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f1463fff700 1 -- 172.21.15.45:0/3724482241 reap_dead start 2023-10-17T01:13:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 shutdown_connections 2023-10-17T01:13:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 >> 172.21.15.45:0/3724482241 conn(0x7f1464076d50 msgr2=0x7f14640fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 shutdown_connections 2023-10-17T01:13:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:35.201+0000 7f146a14c700 1 -- 172.21.15.45:0/3724482241 wait complete. 2023-10-17T01:13:36.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.167+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 --> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa95004ea50 con 0x7fa9380aa7b0 2023-10-17T01:13:36.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.168+0000 7fa94cff9700 1 -- 172.21.15.45:0/3968369150 <== osd.9 v2:172.21.15.45:6826/1619950386 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa95004ea50 con 0x7fa9380aa7b0 2023-10-17T01:13:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] conn(0x7fa9380aa7b0 msgr2=0x7fa9380acc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] conn(0x7fa9380aa7b0 0x7fa9380acc50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9380a0cf0 msgr2=0x7fa9380a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9380a0cf0 0x7fa9380a31b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa940009ad0 tx=0x7fa940009290 comp rx=0 tx=0).stop 2023-10-17T01:13:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95010d610 msgr2=0x7fa95019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa955c9a700 1 --2- 172.21.15.45:0/3968369150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95010d610 0x7fa95019dc00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa94800bad0 tx=0x7fa94800bde0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.169+0000 7fa94ffff700 1 -- 172.21.15.45:0/3968369150 reap_dead start 2023-10-17T01:13:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.170+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 shutdown_connections 2023-10-17T01:13:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.170+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 >> 172.21.15.45:0/3968369150 conn(0x7fa9500fc840 msgr2=0x7fa95010f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.170+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 shutdown_connections 2023-10-17T01:13:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.170+0000 7fa955c9a700 1 -- 172.21.15.45:0/3968369150 wait complete. 2023-10-17T01:13:36.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 -- 172.21.15.45:0/2487436628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073180 msgr2=0x7fe088073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 --2- 172.21.15.45:0/2487436628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073180 0x7fe088073560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe078009990 tx=0x7fe078009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 -- 172.21.15.45:0/2487436628 shutdown_connections 2023-10-17T01:13:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 --2- 172.21.15.45:0/2487436628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08810d5b0 0x7fe08810fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 --2- 172.21.15.45:0/2487436628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073180 0x7fe088073560 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 --2- 172.21.15.45:0/2487436628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe088073aa0 0x7fe08810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 -- 172.21.15.45:0/2487436628 >> 172.21.15.45:0/2487436628 conn(0x7fe0880fc840 msgr2=0x7fe0880fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 -- 172.21.15.45:0/2487436628 shutdown_connections 2023-10-17T01:13:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.553+0000 7fe08e553700 1 -- 172.21.15.45:0/2487436628 wait complete. 2023-10-17T01:13:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 Processor -- start 2023-10-17T01:13:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 -- start start 2023-10-17T01:13:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe088073180 0x7fe0881990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073aa0 0x7fe0881995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08810d5b0 0x7fe08819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08819e070 con 0x7fe088073aa0 2023-10-17T01:13:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08819e1e0 con 0x7fe088073180 2023-10-17T01:13:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.554+0000 7fe08e553700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08819e3a0 con 0x7fe08810d5b0 2023-10-17T01:13:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073aa0 0x7fe0881995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073aa0 0x7fe0881995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40398/0 (socket says 172.21.15.45:40398) 2023-10-17T01:13:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 -- 172.21.15.45:0/759732583 learned_addr learned my addr 172.21.15.45:0/759732583 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe08caf0700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08810d5b0 0x7fe08819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe087fff700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe088073180 0x7fe0881990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 -- 172.21.15.45:0/759732583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08810d5b0 msgr2=0x7fe08819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08810d5b0 0x7fe08819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 -- 172.21.15.45:0/759732583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe088073180 msgr2=0x7fe0881990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe088073180 0x7fe0881990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 -- 172.21.15.45:0/759732583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe080009c40 con 0x7fe088073aa0 2023-10-17T01:13:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0877fe700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073aa0 0x7fe0881995f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00d950 tx=0x7fe07c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c009930 con 0x7fe088073aa0 2023-10-17T01:13:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0780095f0 con 0x7fe088073aa0 2023-10-17T01:13:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07c010460 con 0x7fe088073aa0 2023-10-17T01:13:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.555+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00f620 con 0x7fe088073aa0 2023-10-17T01:13:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.556+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0881a5d10 con 0x7fe088073aa0 2023-10-17T01:13:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.556+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe08804ea50 con 0x7fe088073aa0 2023-10-17T01:13:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.558+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fe07c0108d0 con 0x7fe088073aa0 2023-10-17T01:13:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe0857fa700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe0700a0bb0 0x7fe0700a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe07c0cc600 con 0x7fe088073aa0 2023-10-17T01:13:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe087fff700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe0700a0bb0 0x7fe0700a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe0857fa700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] conn(0x7fe0700aa670 0x7fe0700acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 --> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0700ad1e0 con 0x7fe0700aa670 2023-10-17T01:13:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe07c0cc970 con 0x7fe088073aa0 2023-10-17T01:13:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.560+0000 7fe08caf0700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] conn(0x7fe0700aa670 0x7fe0700acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.561+0000 7fe087fff700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe0700a0bb0 0x7fe0700a3070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe07800b2e0 tx=0x7fe078005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.561+0000 7fe08caf0700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] conn(0x7fe0700aa670 0x7fe0700acb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.562+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== osd.3 v2:172.21.15.45:6810/570572613 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe0700ad1e0 con 0x7fe0700aa670 2023-10-17T01:13:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.579+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 --> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe088113210 con 0x7fe0700aa670 2023-10-17T01:13:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.579+0000 7fe0857fa700 1 -- 172.21.15.45:0/759732583 <== osd.3 v2:172.21.15.45:6810/570572613 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe08000f040 con 0x7fe0700aa670 2023-10-17T01:13:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.579+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] conn(0x7fe0700aa670 msgr2=0x7fe0700acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.579+0000 7fe08e553700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] conn(0x7fe0700aa670 0x7fe0700acb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe0700a0bb0 msgr2=0x7fe0700a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe0700a0bb0 0x7fe0700a3070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe07800b2e0 tx=0x7fe078005fd0 comp rx=0 tx=0).stop 2023-10-17T01:13:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073aa0 msgr2=0x7fe0881995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 --2- 172.21.15.45:0/759732583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe088073aa0 0x7fe0881995f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00d950 tx=0x7fe07c00dc60 comp rx=0 tx=0).stop 2023-10-17T01:13:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08caf0700 1 -- 172.21.15.45:0/759732583 reap_dead start 2023-10-17T01:13:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 shutdown_connections 2023-10-17T01:13:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 >> 172.21.15.45:0/759732583 conn(0x7fe0880fc840 msgr2=0x7fe08810f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 shutdown_connections 2023-10-17T01:13:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:36.580+0000 7fe08e553700 1 -- 172.21.15.45:0/759732583 wait complete. 2023-10-17T01:13:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:335007449106 2023-10-17T01:13:36.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-17T01:13:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 -- 172.21.15.45:0/23701022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150101980 msgr2=0x7f51501059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 --2- 172.21.15.45:0/23701022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150101980 0x7f51501059d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5144009990 tx=0x7f5144009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 -- 172.21.15.45:0/23701022 shutdown_connections 2023-10-17T01:13:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 --2- 172.21.15.45:0/23701022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51501060b0 0x7f515010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 --2- 172.21.15.45:0/23701022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5150100fd0 0x7f51501013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 --2- 172.21.15.45:0/23701022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150101980 0x7f51501059d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.057+0000 7f51549f2700 1 -- 172.21.15.45:0/23701022 >> 172.21.15.45:0/23701022 conn(0x7f51500fc840 msgr2=0x7f51500fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.058+0000 7f51549f2700 1 -- 172.21.15.45:0/23701022 shutdown_connections 2023-10-17T01:13:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.058+0000 7f51549f2700 1 -- 172.21.15.45:0/23701022 wait complete. 2023-10-17T01:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 Processor -- start 2023-10-17T01:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 -- start start 2023-10-17T01:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150100fd0 0x7f51501990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5150101980 0x7f5150199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51501060b0 0x7f515019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019e0a0 con 0x7f5150100fd0 2023-10-17T01:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019e210 con 0x7f5150101980 2023-10-17T01:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.059+0000 7f51549f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019e3d0 con 0x7f51501060b0 2023-10-17T01:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150100fd0 0x7f51501990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150100fd0 0x7f51501990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40412/0 (socket says 172.21.15.45:40412) 2023-10-17T01:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514e59c700 1 -- 172.21.15.45:0/1832771914 learned_addr learned my addr 172.21.15.45:0/1832771914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514dd9b700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5150101980 0x7f5150199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514ed9d700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51501060b0 0x7f515019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514dd9b700 1 -- 172.21.15.45:0/1832771914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51501060b0 msgr2=0x7f515019da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514dd9b700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51501060b0 0x7f515019da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514dd9b700 1 -- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150100fd0 msgr2=0x7f51501990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514dd9b700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5150100fd0 0x7f51501990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.060+0000 7f514dd9b700 1 -- 172.21.15.45:0/1832771914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440095f0 con 0x7f5150101980 2023-10-17T01:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.061+0000 7f514dd9b700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5150101980 0x7f5150199620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5144000c00 tx=0x7f5144004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.061+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514401d070 con 0x7f5150101980 2023-10-17T01:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.061+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515019e650 con 0x7f5150101980 2023-10-17T01:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.061+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5144005030 con 0x7f5150101980 2023-10-17T01:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.061+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144004a90 con 0x7f5150101980 2023-10-17T01:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.061+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51501a5f10 con 0x7f5150101980 2023-10-17T01:13:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.062+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5150110f20 con 0x7f5150101980 2023-10-17T01:13:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.066+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f5144005430 con 0x7f5150101980 2023-10-17T01:13:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.067+0000 7f513f7fe700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f51380a0bb0 0x7f51380a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.067+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f51440cdac0 con 0x7f5150101980 2023-10-17T01:13:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.068+0000 7f514e59c700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f51380a0bb0 0x7f51380a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.068+0000 7f513f7fe700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] conn(0x7f51380aa670 0x7f51380acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.068+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 --> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51380ad1e0 con 0x7f51380aa670 2023-10-17T01:13:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.068+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51440cde30 con 0x7f5150101980 2023-10-17T01:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.068+0000 7f514ed9d700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] conn(0x7f51380aa670 0x7f51380acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.069+0000 7f514e59c700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f51380a0bb0 0x7f51380a3070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5140009d20 tx=0x7f5140009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.069+0000 7f514ed9d700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] conn(0x7f51380aa670 0x7f51380acb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.070+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== osd.10 v2:172.21.15.60:6824/1158474064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f51380ad1e0 con 0x7f51380aa670 2023-10-17T01:13:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.087+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 --> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f515004ea50 con 0x7f51380aa670 2023-10-17T01:13:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.087+0000 7f513f7fe700 1 -- 172.21.15.45:0/1832771914 <== osd.10 v2:172.21.15.60:6824/1158474064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f515004ea50 con 0x7f51380aa670 2023-10-17T01:13:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] conn(0x7f51380aa670 msgr2=0x7f51380acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] conn(0x7f51380aa670 0x7f51380acb10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f51380a0bb0 msgr2=0x7f51380a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f51380a0bb0 0x7f51380a3070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5140009d20 tx=0x7f5140009260 comp rx=0 tx=0).stop 2023-10-17T01:13:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5150101980 msgr2=0x7f5150199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 --2- 172.21.15.45:0/1832771914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5150101980 0x7f5150199620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5144000c00 tx=0x7f5144004730 comp rx=0 tx=0).stop 2023-10-17T01:13:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f514ed9d700 1 -- 172.21.15.45:0/1832771914 reap_dead start 2023-10-17T01:13:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 shutdown_connections 2023-10-17T01:13:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 >> 172.21.15.45:0/1832771914 conn(0x7f51500fc840 msgr2=0x7f51500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.088+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 shutdown_connections 2023-10-17T01:13:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.089+0000 7f51549f2700 1 -- 172.21.15.45:0/1832771914 wait complete. 2023-10-17T01:13:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.119+0000 7f2682eee700 1 -- 172.21.15.45:0/3296707606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 msgr2=0x7f267c111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.119+0000 7f2682eee700 1 --2- 172.21.15.45:0/3296707606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c111840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2668009990 tx=0x7f2668009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 -- 172.21.15.45:0/3296707606 shutdown_connections 2023-10-17T01:13:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 --2- 172.21.15.45:0/3296707606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f267c111d80 0x7f267c114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 --2- 172.21.15.45:0/3296707606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267c0730f0 0x7f267c0734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 --2- 172.21.15.45:0/3296707606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c111840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 -- 172.21.15.45:0/3296707606 >> 172.21.15.45:0/3296707606 conn(0x7f267c0fc840 msgr2=0x7f267c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 -- 172.21.15.45:0/3296707606 shutdown_connections 2023-10-17T01:13:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.120+0000 7f2682eee700 1 -- 172.21.15.45:0/3296707606 wait complete. 2023-10-17T01:13:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 Processor -- start 2023-10-17T01:13:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 -- start start 2023-10-17T01:13:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267c0730f0 0x7f267c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f267c111d80 0x7f267c1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1a2400 con 0x7f267c073a10 2023-10-17T01:13:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1a2570 con 0x7f267c111d80 2023-10-17T01:13:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.121+0000 7f2682eee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1a2730 con 0x7f267c0730f0 2023-10-17T01:13:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c19d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40432/0 (socket says 172.21.15.45:40432) 2023-10-17T01:13:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 -- 172.21.15.45:0/2026937701 learned_addr learned my addr 172.21.15.45:0/2026937701 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:37.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f2680c8a700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267c0730f0 0x7f267c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f268148b700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f267c111d80 0x7f267c1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 -- 172.21.15.45:0/2026937701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267c0730f0 msgr2=0x7f267c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267c0730f0 0x7f267c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 -- 172.21.15.45:0/2026937701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f267c111d80 msgr2=0x7f267c1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f267c111d80 0x7f267c1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 -- 172.21.15.45:0/2026937701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26680095f0 con 0x7f267c073a10 2023-10-17T01:13:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f268148b700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f267c111d80 0x7f267c1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.122+0000 7f267bfff700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c19d980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26680092e0 tx=0x7f2668005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.123+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266801d070 con 0x7f267c073a10 2023-10-17T01:13:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.123+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2668003dd0 con 0x7f267c073a10 2023-10-17T01:13:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.123+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800f460 con 0x7f267c073a10 2023-10-17T01:13:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.123+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c1a29b0 con 0x7f267c073a10 2023-10-17T01:13:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.123+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c1aa0a0 con 0x7f267c073a10 2023-10-17T01:13:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.124+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f267c04ea50 con 0x7f267c073a10 2023-10-17T01:13:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.126+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f2668003910 con 0x7f267c073a10 2023-10-17T01:13:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.127+0000 7f2679ffb700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f266c0a0bb0 0x7f266c0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.128+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26680ce200 con 0x7f267c073a10 2023-10-17T01:13:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.128+0000 7f2680c8a700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f266c0a0bb0 0x7f266c0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.128+0000 7f2679ffb700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] conn(0x7f266c0aa670 0x7f266c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.128+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 --> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f266c0ad1e0 con 0x7f266c0aa670 2023-10-17T01:13:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.128+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26680ce570 con 0x7f267c073a10 2023-10-17T01:13:37.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.128+0000 7f268148b700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] conn(0x7f266c0aa670 0x7f266c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.129+0000 7f2680c8a700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f266c0a0bb0 0x7f266c0a3070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2670009c60 tx=0x7f2670009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.129+0000 7f268148b700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] conn(0x7f266c0aa670 0x7f266c0acb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:37.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.130+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== osd.0 v2:172.21.15.45:6802/563780419 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f266c0ad1e0 con 0x7f266c0aa670 2023-10-17T01:13:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.147+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 --> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f267c117680 con 0x7f266c0aa670 2023-10-17T01:13:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.147+0000 7f2679ffb700 1 -- 172.21.15.45:0/2026937701 <== osd.0 v2:172.21.15.45:6802/563780419 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2674007ce0 con 0x7f266c0aa670 2023-10-17T01:13:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.147+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] conn(0x7f266c0aa670 msgr2=0x7f266c0acb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.147+0000 7f2682eee700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] conn(0x7f266c0aa670 0x7f266c0acb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f266c0a0bb0 msgr2=0x7f266c0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f266c0a0bb0 0x7f266c0a3070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2670009c60 tx=0x7f2670009260 comp rx=0 tx=0).stop 2023-10-17T01:13:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 msgr2=0x7f267c19d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 --2- 172.21.15.45:0/2026937701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c073a10 0x7f267c19d980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26680092e0 tx=0x7f2668005740 comp rx=0 tx=0).stop 2023-10-17T01:13:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f268148b700 1 -- 172.21.15.45:0/2026937701 reap_dead start 2023-10-17T01:13:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 shutdown_connections 2023-10-17T01:13:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 >> 172.21.15.45:0/2026937701 conn(0x7f267c0fc840 msgr2=0x7f267c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:37.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 shutdown_connections 2023-10-17T01:13:37.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:37.148+0000 7f2682eee700 1 -- 172.21.15.45:0/2026937701 wait complete. 2023-10-17T01:13:37.629 INFO:teuthology.orchestra.run.smithi045.stdout:283467841564 2023-10-17T01:13:37.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-17T01:13:37.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:37 smithi045 ceph-mon[98474]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:37 smithi060 ceph-mon[134136]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:37.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:37 smithi181 ceph-mon[134520]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:107374182454 2023-10-17T01:13:38.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-17T01:13:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:77309411387 2023-10-17T01:13:38.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-17T01:13:38.262 INFO:teuthology.orchestra.run.smithi045.stdout:304942678040 2023-10-17T01:13:38.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-17T01:13:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:38.643 INFO:teuthology.orchestra.run.smithi045.stdout:34359738437 2023-10-17T01:13:38.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-17T01:13:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.032 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.033 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.131 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.360 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.416 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:39 smithi045 ceph-mon[98474]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:39 smithi060 ceph-mon[134136]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:39.887 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:39 smithi181 ceph-mon[134520]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:41.676 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:41 smithi045 ceph-mon[98474]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:41 smithi060 ceph-mon[134136]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:41 smithi181 ceph-mon[134520]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s; 13 B/s, 1 objects/s recovering 2023-10-17T01:13:42.250 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:42.295 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:43.339 INFO:teuthology.orchestra.run.smithi045.stdout:141733920820 2023-10-17T01:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/4036058557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf8073aa0 msgr2=0x7f9bf810d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/4036058557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf8073aa0 0x7f9bf810d070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009990 tx=0x7f9bec009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/4036058557 shutdown_connections 2023-10-17T01:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/4036058557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf810d5b0 0x7f9bf810fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/4036058557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073180 0x7f9bf8073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/4036058557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf8073aa0 0x7f9bf810d070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/4036058557 >> 172.21.15.45:0/4036058557 conn(0x7f9bf80fc840 msgr2=0x7f9bf80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/4036058557 shutdown_connections 2023-10-17T01:13:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.407+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/4036058557 wait complete. 2023-10-17T01:13:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 Processor -- start 2023-10-17T01:13:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 -- start start 2023-10-17T01:13:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf8073180 0x7f9bf81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 0x7f9bf8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf810d5b0 0x7f9bf819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819e080 con 0x7f9bf810d5b0 2023-10-17T01:13:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819e1f0 con 0x7f9bf8073aa0 2023-10-17T01:13:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.408+0000 7f9bfcc9b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819e3b0 con 0x7f9bf8073180 2023-10-17T01:13:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 0x7f9bf8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf810d5b0 0x7f9bf819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf8073180 0x7f9bf81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 0x7f9bf8199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60636/0 (socket says 172.21.15.45:60636) 2023-10-17T01:13:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 -- 172.21.15.45:0/1980753706 learned_addr learned my addr 172.21.15.45:0/1980753706 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 -- 172.21.15.45:0/1980753706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf8073180 msgr2=0x7f9bf81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf8073180 0x7f9bf81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 -- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf810d5b0 msgr2=0x7f9bf819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf810d5b0 0x7f9bf819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 -- 172.21.15.45:0/1980753706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0095f0 con 0x7f9bf8073aa0 2023-10-17T01:13:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf6d9d700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf810d5b0 0x7f9bf819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf659c700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf8073180 0x7f9bf81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.409+0000 7f9bf5d9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 0x7f9bf8199600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bec005910 tx=0x7f9bec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.410+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01d070 con 0x7f9bf8073aa0 2023-10-17T01:13:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.410+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf819e630 con 0x7f9bf8073aa0 2023-10-17T01:13:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.410+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec022470 con 0x7f9bf8073aa0 2023-10-17T01:13:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.410+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec017650 con 0x7f9bf8073aa0 2023-10-17T01:13:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.410+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf81a5ef0 con 0x7f9bf8073aa0 2023-10-17T01:13:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.411+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf804ea50 con 0x7f9bf8073aa0 2023-10-17T01:13:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.413+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f9bec00fad0 con 0x7f9bf8073aa0 2023-10-17T01:13:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.414+0000 7f9be77fe700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9be00a0bb0 0x7f9be00a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.414+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9bec0cde60 con 0x7f9bf8073aa0 2023-10-17T01:13:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.414+0000 7f9bf659c700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9be00a0bb0 0x7f9be00a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.415+0000 7f9bf659c700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9be00a0bb0 0x7f9be00a3070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007fd0 tx=0x7f9be80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.417+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f9bec0976f0 con 0x7f9bf8073aa0 2023-10-17T01:13:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.641+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9bf819e7c0 con 0x7f9bf8073aa0 2023-10-17T01:13:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.642+0000 7f9be77fe700 1 -- 172.21.15.45:0/1980753706 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bec09ad10 con 0x7f9bf8073aa0 2023-10-17T01:13:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9be00a0bb0 msgr2=0x7f9be00a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9be00a0bb0 0x7f9be00a3070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007fd0 tx=0x7f9be80073d0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 msgr2=0x7f9bf8199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 0x7f9bf8199600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bec005910 tx=0x7f9bec004730 comp rx=0 tx=0).stop 2023-10-17T01:13:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 shutdown_connections 2023-10-17T01:13:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9be00a0bb0 0x7f9be00a3070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf810d5b0 0x7f9bf819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf8073180 0x7f9bf81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 --2- 172.21.15.45:0/1980753706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bf8073aa0 0x7f9bf8199600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 >> 172.21.15.45:0/1980753706 conn(0x7f9bf80fc840 msgr2=0x7f9bf810f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:43.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 shutdown_connections 2023-10-17T01:13:43.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:42.644+0000 7f9bfcc9b700 1 -- 172.21.15.45:0/1980753706 wait complete. 2023-10-17T01:13:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:43 smithi045 ceph-mon[98474]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 206 B/s wr, 9 op/s; 11 B/s, 1 objects/s recovering 2023-10-17T01:13:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:43 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1980753706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-17T01:13:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 -- 172.21.15.45:0/1406325190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 msgr2=0x7f7bcc0734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1406325190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc0734d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009990 tx=0x7f7bbc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 -- 172.21.15.45:0/1406325190 shutdown_connections 2023-10-17T01:13:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1406325190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bcc111d80 0x7f7bcc114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1406325190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc0734d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1406325190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bcc073a10 0x7f7bcc111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 -- 172.21.15.45:0/1406325190 >> 172.21.15.45:0/1406325190 conn(0x7f7bcc0fc840 msgr2=0x7f7bcc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 -- 172.21.15.45:0/1406325190 shutdown_connections 2023-10-17T01:13:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.708+0000 7f7bd1503700 1 -- 172.21.15.45:0/1406325190 wait complete. 2023-10-17T01:13:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.709+0000 7f7bd1503700 1 Processor -- start 2023-10-17T01:13:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.709+0000 7f7bd1503700 1 -- start start 2023-10-17T01:13:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bd1503700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bd1503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bcc073a10 0x7f7bcc19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bd1503700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bcc111d80 0x7f7bcc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bd1503700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a2480 con 0x7f7bcc0730f0 2023-10-17T01:13:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bd1503700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a25f0 con 0x7f7bcc111d80 2023-10-17T01:13:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bd1503700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a27b0 con 0x7f7bcc073a10 2023-10-17T01:13:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44976/0 (socket says 172.21.15.45:44976) 2023-10-17T01:13:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 -- 172.21.15.45:0/1599635013 learned_addr learned my addr 172.21.15.45:0/1599635013 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bca7fc700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bcc073a10 0x7f7bcc19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcb7fe700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bcc111d80 0x7f7bcc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 -- 172.21.15.45:0/1599635013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bcc073a10 msgr2=0x7f7bcc19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bcc073a10 0x7f7bcc19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 -- 172.21.15.45:0/1599635013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bcc111d80 msgr2=0x7f7bcc1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bcc111d80 0x7f7bcc1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.710+0000 7f7bcaffd700 1 -- 172.21.15.45:0/1599635013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0095f0 con 0x7f7bcc0730f0 2023-10-17T01:13:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.711+0000 7f7bcaffd700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc19d4c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0095c0 tx=0x7f7bbc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.711+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc01d070 con 0x7f7bcc0730f0 2023-10-17T01:13:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.711+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc1a2a30 con 0x7f7bcc0730f0 2023-10-17T01:13:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.711+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bbc005050 con 0x7f7bcc0730f0 2023-10-17T01:13:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.711+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc004b30 con 0x7f7bcc0730f0 2023-10-17T01:13:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.711+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1aa2f0 con 0x7f7bcc0730f0 2023-10-17T01:13:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.712+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc104260 con 0x7f7bcc0730f0 2023-10-17T01:13:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.714+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f7bbc004c90 con 0x7f7bcc0730f0 2023-10-17T01:13:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.715+0000 7f7bb3fff700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bb40a0bb0 0x7f7bb40a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.716+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7bbc0cd220 con 0x7f7bcc0730f0 2023-10-17T01:13:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.716+0000 7f7bca7fc700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bb40a0bb0 0x7f7bb40a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.716+0000 7f7bca7fc700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bb40a0bb0 0x7f7bb40a3070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc19eae0 tx=0x7f7bc000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.719+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7bbc096ab0 con 0x7f7bcc0730f0 2023-10-17T01:13:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:43 smithi060 ceph-mon[134136]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 206 B/s wr, 9 op/s; 11 B/s, 1 objects/s recovering 2023-10-17T01:13:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:43 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1980753706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-17T01:13:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:43 smithi181 ceph-mon[134520]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 206 B/s wr, 9 op/s; 11 B/s, 1 objects/s recovering 2023-10-17T01:13:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:43 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1980753706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-17T01:13:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.942+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7bcc04ea50 con 0x7f7bcc0730f0 2023-10-17T01:13:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.942+0000 7f7bb3fff700 1 -- 172.21.15.45:0/1599635013 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7bbc09a0d0 con 0x7f7bcc0730f0 2023-10-17T01:13:43.943 INFO:teuthology.orchestra.run.smithi045.stdout:197568495659 2023-10-17T01:13:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bb40a0bb0 msgr2=0x7f7bb40a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bb40a0bb0 0x7f7bb40a3070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc19eae0 tx=0x7f7bc000b410 comp rx=0 tx=0).stop 2023-10-17T01:13:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 msgr2=0x7f7bcc19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc19d4c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0095c0 tx=0x7f7bbc00f690 comp rx=0 tx=0).stop 2023-10-17T01:13:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 shutdown_connections 2023-10-17T01:13:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7bb40a0bb0 0x7f7bb40a3070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bcc111d80 0x7f7bcc1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bcc0730f0 0x7f7bcc19d4c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 --2- 172.21.15.45:0/1599635013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bcc073a10 0x7f7bcc19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.944+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 >> 172.21.15.45:0/1599635013 conn(0x7f7bcc0fc840 msgr2=0x7f7bcc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.945+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 shutdown_connections 2023-10-17T01:13:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:43.945+0000 7f7bd1503700 1 -- 172.21.15.45:0/1599635013 wait complete. 2023-10-17T01:13:44.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920818 got 141733920820 for osd.4 2023-10-17T01:13:44.603 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:44 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1599635013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-17T01:13:44.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:44 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1599635013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-17T01:13:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:44 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1599635013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-17T01:13:45.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.326+0000 7f2c87fff700 1 --2- 172.21.15.45:0/3698890018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c88068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:45.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 -- 172.21.15.45:0/3698890018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88103ce0 msgr2=0x7f2c88104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3698890018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88103ce0 0x7f2c88104160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009990 tx=0x7f2c7c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 -- 172.21.15.45:0/3698890018 shutdown_connections 2023-10-17T01:13:45.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3698890018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c88068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3698890018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88105d10 0x7f2c881037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3698890018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88103ce0 0x7f2c88104160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.328+0000 7f2c8e061700 1 -- 172.21.15.45:0/3698890018 >> 172.21.15.45:0/3698890018 conn(0x7f2c88076d50 msgr2=0x7f2c88077160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.329+0000 7f2c8e061700 1 -- 172.21.15.45:0/3698890018 shutdown_connections 2023-10-17T01:13:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.329+0000 7f2c8e061700 1 -- 172.21.15.45:0/3698890018 wait complete. 2023-10-17T01:13:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.330+0000 7f2c8e061700 1 Processor -- start 2023-10-17T01:13:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.331+0000 7f2c8e061700 1 -- start start 2023-10-17T01:13:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.331+0000 7f2c8e061700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c8810cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.333+0000 7f2c8e061700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88103ce0 0x7f2c8810d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.333+0000 7f2c8e061700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88105d10 0x7f2c881b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.333+0000 7f2c86ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88103ce0 0x7f2c8810d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.333+0000 7f2c86ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88103ce0 0x7f2c8810d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57816/0 (socket says 172.21.15.45:57816) 2023-10-17T01:13:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.335+0000 7f2c877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c8810cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.335+0000 7f2c877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c8810cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44998/0 (socket says 172.21.15.45:44998) 2023-10-17T01:13:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.335+0000 7f2c8e061700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c881b1f80 con 0x7f2c88068870 2023-10-17T01:13:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.335+0000 7f2c8e061700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c881b20f0 con 0x7f2c88105d10 2023-10-17T01:13:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.336+0000 7f2c86ffd700 1 -- 172.21.15.45:0/3752981909 learned_addr learned my addr 172.21.15.45:0/3752981909 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.335+0000 7f2c8e061700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c881b22b0 con 0x7f2c88103ce0 2023-10-17T01:13:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.336+0000 7f2c87fff700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88105d10 0x7f2c881b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.336+0000 7f2c877fe700 1 -- 172.21.15.45:0/3752981909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88103ce0 msgr2=0x7f2c8810d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.336+0000 7f2c877fe700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88103ce0 0x7f2c8810d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.336+0000 7f2c877fe700 1 -- 172.21.15.45:0/3752981909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88105d10 msgr2=0x7f2c881b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.337+0000 7f2c877fe700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88105d10 0x7f2c881b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.337+0000 7f2c877fe700 1 -- 172.21.15.45:0/3752981909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c80009c40 con 0x7f2c88068870 2023-10-17T01:13:45.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.337+0000 7f2c877fe700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c8810cd30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9f0 tx=0x7f2c7800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.337+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800cb70 con 0x7f2c88068870 2023-10-17T01:13:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.338+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c780044e0 con 0x7f2c88068870 2023-10-17T01:13:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.338+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78010430 con 0x7f2c88068870 2023-10-17T01:13:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.338+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c78009520 con 0x7f2c88068870 2023-10-17T01:13:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.338+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c0095f0 con 0x7f2c88068870 2023-10-17T01:13:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.339+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c74005320 con 0x7f2c88068870 2023-10-17T01:13:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.344+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f2c7800ccd0 con 0x7f2c88068870 2023-10-17T01:13:45.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.348+0000 7f2c84ff9700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c700a0cb0 0x7f2c700a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.348+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2c78014070 con 0x7f2c88068870 2023-10-17T01:13:45.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.349+0000 7f2c86ffd700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c700a0cb0 0x7f2c700a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.351+0000 7f2c86ffd700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c700a0cb0 0x7f2c700a3170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c88103b20 tx=0x7f2c7c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.354+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2c78099910 con 0x7f2c88068870 2023-10-17T01:13:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.164+0000 7f50e512b700 1 -- 172.21.15.45:0/3771452007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 msgr2=0x7f50e010d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.164+0000 7f50e512b700 1 --2- 172.21.15.45:0/3771452007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e010d2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f50d40098f0 tx=0x7f50d4009c00 comp rx=0 tx=0).stop 2023-10-17T01:13:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 -- 172.21.15.45:0/3771452007 shutdown_connections 2023-10-17T01:13:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 --2- 172.21.15.45:0/3771452007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e010d7e0 0x7f50e010fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 --2- 172.21.15.45:0/3771452007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e0100770 0x7f50e0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 --2- 172.21.15.45:0/3771452007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e010d2a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 -- 172.21.15.45:0/3771452007 >> 172.21.15.45:0/3771452007 conn(0x7f50e0076d50 msgr2=0x7f50e0077160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 -- 172.21.15.45:0/3771452007 shutdown_connections 2023-10-17T01:13:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.165+0000 7f50e512b700 1 -- 172.21.15.45:0/3771452007 wait complete. 2023-10-17T01:13:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 Processor -- start 2023-10-17T01:13:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 -- start start 2023-10-17T01:13:45.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e0100770 0x7f50e0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e010d7e0 0x7f50e019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e019e0f0 con 0x7f50e0100770 2023-10-17T01:13:45.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e019e260 con 0x7f50e0101090 2023-10-17T01:13:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50e512b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e019e420 con 0x7f50e010d7e0 2023-10-17T01:13:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50de59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50ded9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e0100770 0x7f50e0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50de59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e0199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60684/0 (socket says 172.21.15.45:60684) 2023-10-17T01:13:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.166+0000 7f50de59c700 1 -- 172.21.15.45:0/1819986234 learned_addr learned my addr 172.21.15.45:0/1819986234 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50df59e700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e010d7e0 0x7f50e019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50de59c700 1 -- 172.21.15.45:0/1819986234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e010d7e0 msgr2=0x7f50e019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50de59c700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e010d7e0 0x7f50e019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50de59c700 1 -- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e0100770 msgr2=0x7f50e0199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50de59c700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e0100770 0x7f50e0199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50de59c700 1 -- 172.21.15.45:0/1819986234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009d00 con 0x7f50e0101090 2023-10-17T01:13:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50ded9d700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e0100770 0x7f50e0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50de59c700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e0199670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009fd0 tx=0x7f50d400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.167+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d401c070 con 0x7f50e0101090 2023-10-17T01:13:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.168+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d4009520 con 0x7f50e0101090 2023-10-17T01:13:45.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.168+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e01a5d90 con 0x7f50e0101090 2023-10-17T01:13:45.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.168+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50d400bb20 con 0x7f50e0101090 2023-10-17T01:13:45.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.168+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4022680 con 0x7f50e0101090 2023-10-17T01:13:45.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.169+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e004ea50 con 0x7f50e0101090 2023-10-17T01:13:45.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.171+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f50d4020070 con 0x7f50e0101090 2023-10-17T01:13:45.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.172+0000 7f50c7fff700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50c80a0bb0 0x7f50c80a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.172+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f50d40cd2e0 con 0x7f50e0101090 2023-10-17T01:13:45.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.172+0000 7f50ded9d700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50c80a0bb0 0x7f50c80a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.173+0000 7f50ded9d700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50c80a0bb0 0x7f50c80a3070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f50d0011490 tx=0x7f50d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:45.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.174+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f50d4096b70 con 0x7f50e0101090 2023-10-17T01:13:45.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.495+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f50e01a5f20 con 0x7f50e0101090 2023-10-17T01:13:45.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.496+0000 7f50c7fff700 1 -- 172.21.15.45:0/1819986234 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50d409a190 con 0x7f50e0101090 2023-10-17T01:13:45.496 INFO:teuthology.orchestra.run.smithi045.stdout:55834574913 2023-10-17T01:13:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50c80a0bb0 msgr2=0x7f50c80a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50c80a0bb0 0x7f50c80a3070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f50d0011490 tx=0x7f50d0009260 comp rx=0 tx=0).stop 2023-10-17T01:13:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 msgr2=0x7f50e0199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e0199670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009fd0 tx=0x7f50d400fa00 comp rx=0 tx=0).stop 2023-10-17T01:13:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 shutdown_connections 2023-10-17T01:13:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50c80a0bb0 0x7f50c80a3070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e010d7e0 0x7f50e019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e0100770 0x7f50e0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 --2- 172.21.15.45:0/1819986234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50e0101090 0x7f50e0199670 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009fd0 tx=0x7f50d400fa00 comp rx=0 tx=0).stop 2023-10-17T01:13:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.499+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 >> 172.21.15.45:0/1819986234 conn(0x7f50e0076d50 msgr2=0x7f50e010ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.500+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 shutdown_connections 2023-10-17T01:13:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.500+0000 7f50e512b700 1 -- 172.21.15.45:0/1819986234 wait complete. 2023-10-17T01:13:45.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.652+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2c74005190 con 0x7f2c88068870 2023-10-17T01:13:45.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.653+0000 7f2c84ff9700 1 -- 172.21.15.45:0/3752981909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c780994a0 con 0x7f2c88068870 2023-10-17T01:13:45.656 INFO:teuthology.orchestra.run.smithi045.stdout:219043332135 2023-10-17T01:13:45.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.656+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c700a0cb0 msgr2=0x7f2c700a3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.656+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c700a0cb0 0x7f2c700a3170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c88103b20 tx=0x7f2c7c00b260 comp rx=0 tx=0).stop 2023-10-17T01:13:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.656+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 msgr2=0x7f2c8810cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.656+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c8810cd30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9f0 tx=0x7f2c7800edb0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.657+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 shutdown_connections 2023-10-17T01:13:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.657+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c700a0cb0 0x7f2c700a3170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.657+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c88105d10 0x7f2c881b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.657+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c88068870 0x7f2c8810cd30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.657+0000 7f2c8e061700 1 --2- 172.21.15.45:0/3752981909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c88103ce0 0x7f2c8810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.657+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 >> 172.21.15.45:0/3752981909 conn(0x7f2c88076d50 msgr2=0x7f2c880fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.658+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 shutdown_connections 2023-10-17T01:13:45.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.658+0000 7f2c8e061700 1 -- 172.21.15.45:0/3752981909 wait complete. 2023-10-17T01:13:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 -- 172.21.15.45:0/66187299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 msgr2=0x7f23141059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 --2- 172.21.15.45:0/66187299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f23141059d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2304009990 tx=0x7f2304009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 -- 172.21.15.45:0/66187299 shutdown_connections 2023-10-17T01:13:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 --2- 172.21.15.45:0/66187299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23141060b0 0x7f231410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 --2- 172.21.15.45:0/66187299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2314100fd0 0x7f23141013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 --2- 172.21.15.45:0/66187299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f23141059d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 -- 172.21.15.45:0/66187299 >> 172.21.15.45:0/66187299 conn(0x7f23140fc840 msgr2=0x7f23140fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 -- 172.21.15.45:0/66187299 shutdown_connections 2023-10-17T01:13:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.724+0000 7f231bc18700 1 -- 172.21.15.45:0/66187299 wait complete. 2023-10-17T01:13:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.725+0000 7f231bc18700 1 Processor -- start 2023-10-17T01:13:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 -- start start 2023-10-17T01:13:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100fd0 0x7f2314073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f2314073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23141060b0 0x7f2314073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a9940 con 0x7f2314101980 2023-10-17T01:13:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231419b100 con 0x7f23141060b0 2023-10-17T01:13:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231bc18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231419b2c0 con 0x7f2314100fd0 2023-10-17T01:13:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f23191b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f2314073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f231a1b5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23141060b0 0x7f2314073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f23191b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f2314073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45040/0 (socket says 172.21.15.45:45040) 2023-10-17T01:13:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f23199b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100fd0 0x7f2314073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.726+0000 7f23191b3700 1 -- 172.21.15.45:0/1354007087 learned_addr learned my addr 172.21.15.45:0/1354007087 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f231a1b5700 1 -- 172.21.15.45:0/1354007087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100fd0 msgr2=0x7f2314073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f231a1b5700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100fd0 0x7f2314073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f231a1b5700 1 -- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 msgr2=0x7f2314073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f231a1b5700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f2314073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f231a1b5700 1 -- 172.21.15.45:0/1354007087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23040095f0 con 0x7f23141060b0 2023-10-17T01:13:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f231a1b5700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23141060b0 0x7f2314073ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f230c007f10 tx=0x7f230c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.727+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c004db0 con 0x7f23141060b0 2023-10-17T01:13:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.728+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231419b5a0 con 0x7f23141060b0 2023-10-17T01:13:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.728+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f230c00f040 con 0x7f23141060b0 2023-10-17T01:13:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.728+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c005990 con 0x7f23141060b0 2023-10-17T01:13:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.728+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231419baf0 con 0x7f23141060b0 2023-10-17T01:13:45.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.729+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231404ea50 con 0x7f23141060b0 2023-10-17T01:13:45.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.731+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f230c00db20 con 0x7f23141060b0 2023-10-17T01:13:45.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.733+0000 7f230affd700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23000a0c30 0x7f23000a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:45.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.733+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f230c0cb9e0 con 0x7f23141060b0 2023-10-17T01:13:45.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.733+0000 7f23199b4700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23000a0c30 0x7f23000a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:45.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.734+0000 7f23199b4700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23000a0c30 0x7f23000a30f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2310009c60 tx=0x7f2310009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.737+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f230c04f0b0 con 0x7f23141060b0 2023-10-17T01:13:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:45 smithi045 ceph-mon[98474]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 187 B/s wr, 9 op/s; 10 B/s, 1 objects/s recovering 2023-10-17T01:13:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:45 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1819986234' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-17T01:13:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:45 smithi060 ceph-mon[134136]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 187 B/s wr, 9 op/s; 10 B/s, 1 objects/s recovering 2023-10-17T01:13:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:45 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1819986234' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-17T01:13:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:45 smithi181 ceph-mon[134520]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 187 B/s wr, 9 op/s; 10 B/s, 1 objects/s recovering 2023-10-17T01:13:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:45 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1819986234' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-17T01:13:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.979+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f231419bd10 con 0x7f23141060b0 2023-10-17T01:13:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.979+0000 7f230affd700 1 -- 172.21.15.45:0/1354007087 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f230c095240 con 0x7f23141060b0 2023-10-17T01:13:45.980 INFO:teuthology.orchestra.run.smithi045.stdout:163208757295 2023-10-17T01:13:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23000a0c30 msgr2=0x7f23000a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23000a0c30 0x7f23000a30f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2310009c60 tx=0x7f2310009260 comp rx=0 tx=0).stop 2023-10-17T01:13:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23141060b0 msgr2=0x7f2314073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23141060b0 0x7f2314073ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f230c007f10 tx=0x7f230c00d3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 shutdown_connections 2023-10-17T01:13:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23000a0c30 0x7f23000a30f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23141060b0 0x7f2314073ad0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100fd0 0x7f2314073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 --2- 172.21.15.45:0/1354007087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2314101980 0x7f2314073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 >> 172.21.15.45:0/1354007087 conn(0x7f23140fc840 msgr2=0x7f23141049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 shutdown_connections 2023-10-17T01:13:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.982+0000 7f231bc18700 1 -- 172.21.15.45:0/1354007087 wait complete. 2023-10-17T01:13:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 -- 172.21.15.45:0/1605924993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce50109f60 msgr2=0x7fce50116420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 --2- 172.21.15.45:0/1605924993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce50109f60 0x7fce50116420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce440099c0 tx=0x7fce44009cd0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 -- 172.21.15.45:0/1605924993 shutdown_connections 2023-10-17T01:13:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 --2- 172.21.15.45:0/1605924993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce50109f60 0x7fce50116420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 --2- 172.21.15.45:0/1605924993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5010f650 0x7fce5010fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 --2- 172.21.15.45:0/1605924993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce501095a0 0x7fce50109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 -- 172.21.15.45:0/1605924993 >> 172.21.15.45:0/1605924993 conn(0x7fce5006dc30 msgr2=0x7fce50070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 -- 172.21.15.45:0/1605924993 shutdown_connections 2023-10-17T01:13:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.403+0000 7fce55d8c700 1 -- 172.21.15.45:0/1605924993 wait complete. 2023-10-17T01:13:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.406+0000 7fce55d8c700 1 Processor -- start 2023-10-17T01:13:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.407+0000 7fce55d8c700 1 -- start start 2023-10-17T01:13:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.408+0000 7fce55d8c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce501095a0 0x7fce50112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.408+0000 7fce55d8c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce50109f60 0x7fce50113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.408+0000 7fce55d8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 0x7fce501b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.408+0000 7fce55d8c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501b2020 con 0x7fce5010f650 2023-10-17T01:13:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.408+0000 7fce55d8c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501b2160 con 0x7fce501095a0 2023-10-17T01:13:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.408+0000 7fce55d8c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501b2320 con 0x7fce50109f60 2023-10-17T01:13:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.409+0000 7fce4ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 0x7fce501b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.409+0000 7fce4ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 0x7fce501b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45012/0 (socket says 172.21.15.45:45012) 2023-10-17T01:13:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.409+0000 7fce4ffff700 1 -- 172.21.15.45:0/4249429629 learned_addr learned my addr 172.21.15.45:0/4249429629 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.409+0000 7fce4f7fe700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce501095a0 0x7fce50112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.409+0000 7fce4effd700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce50109f60 0x7fce50113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.410+0000 7fce4ffff700 1 -- 172.21.15.45:0/4249429629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce50109f60 msgr2=0x7fce50113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.410+0000 7fce4ffff700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce50109f60 0x7fce50113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.410+0000 7fce4ffff700 1 -- 172.21.15.45:0/4249429629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce501095a0 msgr2=0x7fce50112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.410+0000 7fce4ffff700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce501095a0 0x7fce50112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.410+0000 7fce4ffff700 1 -- 172.21.15.45:0/4249429629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440095f0 con 0x7fce5010f650 2023-10-17T01:13:46.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.411+0000 7fce4ffff700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 0x7fce501b1a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce44009990 tx=0x7fce4400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:46.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.411+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4401d070 con 0x7fce5010f650 2023-10-17T01:13:46.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.412+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce501b25a0 con 0x7fce5010f650 2023-10-17T01:13:46.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.412+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce44003680 con 0x7fce5010f650 2023-10-17T01:13:46.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.412+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce501b2a30 con 0x7fce5010f650 2023-10-17T01:13:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.413+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400fc80 con 0x7fce5010f650 2023-10-17T01:13:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.414+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c005320 con 0x7fce5010f650 2023-10-17T01:13:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.418+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fce4400f460 con 0x7fce5010f650 2023-10-17T01:13:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.420+0000 7fce4cff9700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce380a0c70 0x7fce380a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:46.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.420+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce440ccfc0 con 0x7fce5010f650 2023-10-17T01:13:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.420+0000 7fce4f7fe700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce380a0c70 0x7fce380a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.421+0000 7fce4f7fe700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce380a0c70 0x7fce380a3130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fce48009fd0 tx=0x7fce4800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.424+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fce4405c0b0 con 0x7fce5010f650 2023-10-17T01:13:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.710+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fce3c0059f0 con 0x7fce5010f650 2023-10-17T01:13:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.710+0000 7fce4cff9700 1 -- 172.21.15.45:0/4249429629 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce44026020 con 0x7fce5010f650 2023-10-17T01:13:46.697 INFO:teuthology.orchestra.run.smithi045.stdout:283467841565 2023-10-17T01:13:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce380a0c70 msgr2=0x7fce380a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce380a0c70 0x7fce380a3130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fce48009fd0 tx=0x7fce4800d040 comp rx=0 tx=0).stop 2023-10-17T01:13:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 msgr2=0x7fce501b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 0x7fce501b1a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce44009990 tx=0x7fce4400bfd0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 shutdown_connections 2023-10-17T01:13:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce380a0c70 0x7fce380a3130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5010f650 0x7fce501b1a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce501095a0 0x7fce50112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 --2- 172.21.15.45:0/4249429629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce50109f60 0x7fce50113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 >> 172.21.15.45:0/4249429629 conn(0x7fce5006dc30 msgr2=0x7fce5010d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 shutdown_connections 2023-10-17T01:13:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:45.714+0000 7fce55d8c700 1 -- 172.21.15.45:0/4249429629 wait complete. 2023-10-17T01:13:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:46 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3752981909' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-17T01:13:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:46 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4249429629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-17T01:13:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:46 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1354007087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-17T01:13:46.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:46 smithi045 ceph-mon[98474]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-10-17T01:13:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:46 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3752981909' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-17T01:13:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:46 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/4249429629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-17T01:13:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:46 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1354007087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-17T01:13:46.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:46 smithi060 ceph-mon[134136]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-10-17T01:13:46.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:46 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3752981909' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-17T01:13:46.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:46 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/4249429629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-17T01:13:46.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:46 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1354007087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-17T01:13:46.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:46 smithi181 ceph-mon[134520]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-10-17T01:13:47.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495659 for osd.6 2023-10-17T01:13:47.024 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 -- 172.21.15.45:0/2666435941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 msgr2=0x7feeec10d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 --2- 172.21.15.45:0/2666435941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec10d070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feee0009990 tx=0x7feee0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 -- 172.21.15.45:0/2666435941 shutdown_connections 2023-10-17T01:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 --2- 172.21.15.45:0/2666435941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeec10d5b0 0x7feeec10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 --2- 172.21.15.45:0/2666435941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec073180 0x7feeec073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 --2- 172.21.15.45:0/2666435941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec10d070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 -- 172.21.15.45:0/2666435941 >> 172.21.15.45:0/2666435941 conn(0x7feeec0fc840 msgr2=0x7feeec0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 -- 172.21.15.45:0/2666435941 shutdown_connections 2023-10-17T01:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.120+0000 7feef0cdb700 1 -- 172.21.15.45:0/2666435941 wait complete. 2023-10-17T01:13:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.122+0000 7feef0cdb700 1 Processor -- start 2023-10-17T01:13:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.122+0000 7feef0cdb700 1 -- start start 2023-10-17T01:13:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.122+0000 7feef0cdb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec073180 0x7feeec1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.122+0000 7feef0cdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.122+0000 7feef0cdb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeec10d5b0 0x7feeec19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feef0cdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec19e060 con 0x7feeec073aa0 2023-10-17T01:13:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feef0cdb700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec19e1d0 con 0x7feeec10d5b0 2023-10-17T01:13:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feef0cdb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec19e390 con 0x7feeec073180 2023-10-17T01:13:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45054/0 (socket says 172.21.15.45:45054) 2023-10-17T01:13:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 -- 172.21.15.45:0/1574636166 learned_addr learned my addr 172.21.15.45:0/1574636166 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feeea59c700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec073180 0x7feeec1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feeead9d700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeec10d5b0 0x7feeec19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 -- 172.21.15.45:0/1574636166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec073180 msgr2=0x7feeec1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec073180 0x7feeec1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 -- 172.21.15.45:0/1574636166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeec10d5b0 msgr2=0x7feeec19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeec10d5b0 0x7feeec19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.123+0000 7feee9d9b700 1 -- 172.21.15.45:0/1574636166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00095f0 con 0x7feeec073aa0 2023-10-17T01:13:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.124+0000 7feee9d9b700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec1995e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feee0005910 tx=0x7feee0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.124+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee001d070 con 0x7feeec073aa0 2023-10-17T01:13:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.124+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee0022470 con 0x7feeec073aa0 2023-10-17T01:13:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.124+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0017650 con 0x7feeec073aa0 2023-10-17T01:13:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.124+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec19e610 con 0x7feeec073aa0 2023-10-17T01:13:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.124+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec1a5ed0 con 0x7feeec073aa0 2023-10-17T01:13:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.126+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec04ea50 con 0x7feeec073aa0 2023-10-17T01:13:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.127+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7feee000fad0 con 0x7feeec073aa0 2023-10-17T01:13:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.129+0000 7feedb7fe700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feed40a0d00 0x7feed40a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.129+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7feee00cd6b0 con 0x7feeec073aa0 2023-10-17T01:13:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.129+0000 7feeea59c700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feed40a0d00 0x7feed40a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.130+0000 7feeea59c700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feed40a0d00 0x7feed40a31c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feedc005950 tx=0x7feedc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:47.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.133+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7feee0096f40 con 0x7feeec073aa0 2023-10-17T01:13:47.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574910 got 55834574913 for osd.1 2023-10-17T01:13:47.194 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.352+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7feeec1a6060 con 0x7feeec073aa0 2023-10-17T01:13:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.353+0000 7feedb7fe700 1 -- 172.21.15.45:0/1574636166 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feee009a560 con 0x7feeec073aa0 2023-10-17T01:13:47.353 INFO:teuthology.orchestra.run.smithi045.stdout:249108103202 2023-10-17T01:13:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feed40a0d00 msgr2=0x7feed40a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feed40a0d00 0x7feed40a31c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feedc005950 tx=0x7feedc00b410 comp rx=0 tx=0).stop 2023-10-17T01:13:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 msgr2=0x7feeec1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec1995e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feee0005910 tx=0x7feee0004730 comp rx=0 tx=0).stop 2023-10-17T01:13:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 shutdown_connections 2023-10-17T01:13:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feed40a0d00 0x7feed40a31c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeec10d5b0 0x7feeec19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeec073180 0x7feeec1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 --2- 172.21.15.45:0/1574636166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec073aa0 0x7feeec1995e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 >> 172.21.15.45:0/1574636166 conn(0x7feeec0fc840 msgr2=0x7feeec10f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.355+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 shutdown_connections 2023-10-17T01:13:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:47.356+0000 7feef0cdb700 1 -- 172.21.15.45:0/1574636166 wait complete. 2023-10-17T01:13:47.799 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:47 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1574636166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-17T01:13:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:47 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1574636166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-17T01:13:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:47 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1574636166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-17T01:13:47.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332132 got 219043332135 for osd.7 2023-10-17T01:13:47.966 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:48.269 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841564 got 283467841565 for osd.9 2023-10-17T01:13:48.269 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.353+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/3184998762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 msgr2=0x7fe1bc0734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.353+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/3184998762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 0x7fe1bc0734d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009990 tx=0x7fe1b0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.353+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/3184998762 shutdown_connections 2023-10-17T01:13:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.354+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/3184998762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 0x7fe1bc114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.354+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/3184998762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 0x7fe1bc0734d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.354+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/3184998762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.354+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/3184998762 >> 172.21.15.45:0/3184998762 conn(0x7fe1bc0fc840 msgr2=0x7fe1bc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:48.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.354+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/3184998762 shutdown_connections 2023-10-17T01:13:48.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.354+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/3184998762 wait complete. 2023-10-17T01:13:48.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.355+0000 7fe1c2f1a700 1 Processor -- start 2023-10-17T01:13:48.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.355+0000 7fe1c2f1a700 1 -- start start 2023-10-17T01:13:48.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c2f1a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 0x7fe1bc19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:48.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c2f1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:48.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c2f1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 0x7fe1bc1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:48.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c2f1a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc1a2470 con 0x7fe1bc073a10 2023-10-17T01:13:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c2f1a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc1a25e0 con 0x7fe1bc0730f0 2023-10-17T01:13:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c2f1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc1a27a0 con 0x7fe1bc111d80 2023-10-17T01:13:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc19d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59228/0 (socket says 172.21.15.45:59228) 2023-10-17T01:13:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1bbfff700 1 -- 172.21.15.45:0/420345731 learned_addr learned my addr 172.21.15.45:0/420345731 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c14b7700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 0x7fe1bc1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.356+0000 7fe1c0cb6700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 0x7fe1bc19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1bbfff700 1 -- 172.21.15.45:0/420345731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 msgr2=0x7fe1bc1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1bbfff700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 0x7fe1bc1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1bbfff700 1 -- 172.21.15.45:0/420345731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 msgr2=0x7fe1bc19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1bbfff700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 0x7fe1bc19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1bbfff700 1 -- 172.21.15.45:0/420345731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00095f0 con 0x7fe1bc073a10 2023-10-17T01:13:48.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1c14b7700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 0x7fe1bc1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:48.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.357+0000 7fe1bbfff700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc19d9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9a0 tx=0x7fe1a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:48.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.358+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8009790 con 0x7fe1bc073a10 2023-10-17T01:13:48.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.358+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1a8010430 con 0x7fe1bc073a10 2023-10-17T01:13:48.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.358+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8018680 con 0x7fe1bc073a10 2023-10-17T01:13:48.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.358+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc1a2a80 con 0x7fe1bc073a10 2023-10-17T01:13:48.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.358+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc1aa2e0 con 0x7fe1bc073a10 2023-10-17T01:13:48.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.359+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc104260 con 0x7fe1bc073a10 2023-10-17T01:13:48.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.362+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fe1a80105a0 con 0x7fe1bc073a10 2023-10-17T01:13:48.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.364+0000 7fe1b9ffb700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe1ac0a0c80 0x7fe1ac0a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:48.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.364+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe1a8014070 con 0x7fe1bc073a10 2023-10-17T01:13:48.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.364+0000 7fe1c0cb6700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe1ac0a0c80 0x7fe1ac0a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:48.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.365+0000 7fe1c0cb6700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe1ac0a0c80 0x7fe1ac0a3140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000b2e0 tx=0x7fe1b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:48.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.368+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe1a80520b0 con 0x7fe1bc073a10 2023-10-17T01:13:48.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757293 got 163208757295 for osd.5 2023-10-17T01:13:48.466 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:48.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.600+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fe1bc02cc30 con 0x7fe1bc073a10 2023-10-17T01:13:48.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.600+0000 7fe1b9ffb700 1 -- 172.21.15.45:0/420345731 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe1a8095cc0 con 0x7fe1bc073a10 2023-10-17T01:13:48.601 INFO:teuthology.orchestra.run.smithi045.stdout:335007449109 2023-10-17T01:13:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.602+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe1ac0a0c80 msgr2=0x7fe1ac0a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.602+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe1ac0a0c80 0x7fe1ac0a3140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000b2e0 tx=0x7fe1b0005fb0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.602+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 msgr2=0x7fe1bc19d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.602+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc19d9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9a0 tx=0x7fe1a800ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 shutdown_connections 2023-10-17T01:13:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe1ac0a0c80 0x7fe1ac0a3140 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1bc111d80 0x7fe1bc1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0730f0 0x7fe1bc19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 --2- 172.21.15.45:0/420345731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc073a10 0x7fe1bc19d9f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:48.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 >> 172.21.15.45:0/420345731 conn(0x7fe1bc0fc840 msgr2=0x7fe1bc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:48.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 shutdown_connections 2023-10-17T01:13:48.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.603+0000 7fe1c2f1a700 1 -- 172.21.15.45:0/420345731 wait complete. 2023-10-17T01:13:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:48 smithi045 ceph-mon[98474]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-10-17T01:13:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:48 smithi060 ceph-mon[134136]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-10-17T01:13:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:48 smithi181 ceph-mon[134520]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-10-17T01:13:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/1549986498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 msgr2=0x7fa49c111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/1549986498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c111680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa488009990 tx=0x7fa488009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/1549986498 shutdown_connections 2023-10-17T01:13:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/1549986498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa49c111bc0 0x7fa49c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/1549986498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa49c0730f0 0x7fa49c0734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/1549986498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c111680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/1549986498 >> 172.21.15.45:0/1549986498 conn(0x7fa49c0fc700 msgr2=0x7fa49c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/1549986498 shutdown_connections 2023-10-17T01:13:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.734+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/1549986498 wait complete. 2023-10-17T01:13:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.735+0000 7fa4a1ab6700 1 Processor -- start 2023-10-17T01:13:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.735+0000 7fa4a1ab6700 1 -- start start 2023-10-17T01:13:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a1ab6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa49c0730f0 0x7fa49c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a1ab6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a1ab6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa49c111bc0 0x7fa49c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a1ab6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a2330 con 0x7fa49c073a10 2023-10-17T01:13:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a1ab6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a24a0 con 0x7fa49c0730f0 2023-10-17T01:13:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a1ab6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a2660 con 0x7fa49c111bc0 2023-10-17T01:13:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59242/0 (socket says 172.21.15.45:59242) 2023-10-17T01:13:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 -- 172.21.15.45:0/510994561 learned_addr learned my addr 172.21.15.45:0/510994561 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a0ab4700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa49c0730f0 0x7fa49c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa4a12b5700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa49c111bc0 0x7fa49c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 -- 172.21.15.45:0/510994561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa49c111bc0 msgr2=0x7fa49c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa49c111bc0 0x7fa49c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 -- 172.21.15.45:0/510994561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa49c0730f0 msgr2=0x7fa49c19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa49c0730f0 0x7fa49c19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.736+0000 7fa49bfff700 1 -- 172.21.15.45:0/510994561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880095f0 con 0x7fa49c073a10 2023-10-17T01:13:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.737+0000 7fa49bfff700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c19d8b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa488005910 tx=0x7fa488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.737+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48801d070 con 0x7fa49c073a10 2023-10-17T01:13:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.737+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa488022470 con 0x7fa49c073a10 2023-10-17T01:13:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.737+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c1a28e0 con 0x7fa49c073a10 2023-10-17T01:13:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.737+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488017650 con 0x7fa49c073a10 2023-10-17T01:13:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.737+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1aa1a0 con 0x7fa49c073a10 2023-10-17T01:13:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.738+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c1040e0 con 0x7fa49c073a10 2023-10-17T01:13:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.740+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fa48800fad0 con 0x7fa49c073a10 2023-10-17T01:13:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.742+0000 7fa499ffb700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa48c0a0c30 0x7fa48c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.743+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa4880cd730 con 0x7fa49c073a10 2023-10-17T01:13:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.743+0000 7fa4a0ab4700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa48c0a0c30 0x7fa48c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.743+0000 7fa4a0ab4700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa48c0a0c30 0x7fa48c0a30f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa490005950 tx=0x7fa49000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.746+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fa488096fc0 con 0x7fa49c073a10 2023-10-17T01:13:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.960+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa49c04f2a0 con 0x7fa49c073a10 2023-10-17T01:13:49.080 INFO:teuthology.orchestra.run.smithi045.stdout:34359738439 2023-10-17T01:13:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.960+0000 7fa499ffb700 1 -- 172.21.15.45:0/510994561 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa48809a5e0 con 0x7fa49c073a10 2023-10-17T01:13:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa48c0a0c30 msgr2=0x7fa48c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa48c0a0c30 0x7fa48c0a30f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa490005950 tx=0x7fa49000b410 comp rx=0 tx=0).stop 2023-10-17T01:13:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 msgr2=0x7fa49c19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c19d8b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa488005910 tx=0x7fa488004730 comp rx=0 tx=0).stop 2023-10-17T01:13:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 shutdown_connections 2023-10-17T01:13:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa48c0a0c30 0x7fa48c0a30f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa49c111bc0 0x7fa49c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa49c0730f0 0x7fa49c19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 --2- 172.21.15.45:0/510994561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c073a10 0x7fa49c19d8b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.962+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 >> 172.21.15.45:0/510994561 conn(0x7fa49c0fc700 msgr2=0x7fa49c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.963+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 shutdown_connections 2023-10-17T01:13:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:48.963+0000 7fa4a1ab6700 1 -- 172.21.15.45:0/510994561 wait complete. 2023-10-17T01:13:49.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103199 got 249108103202 for osd.8 2023-10-17T01:13:49.093 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 -- 172.21.15.45:0/16918381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4103ce0 msgr2=0x7f4ae4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/16918381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4103ce0 0x7f4ae4104160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009990 tx=0x7f4ad4009ca0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 -- 172.21.15.45:0/16918381 shutdown_connections 2023-10-17T01:13:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/16918381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ae4068870 0x7f4ae4068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/16918381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 0x7f4ae41037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/16918381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4103ce0 0x7f4ae4104160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 -- 172.21.15.45:0/16918381 >> 172.21.15.45:0/16918381 conn(0x7f4ae4076d50 msgr2=0x7f4ae4077160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 -- 172.21.15.45:0/16918381 shutdown_connections 2023-10-17T01:13:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.726+0000 7f4aebf1c700 1 -- 172.21.15.45:0/16918381 wait complete. 2023-10-17T01:13:49.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.728+0000 7f4aebf1c700 1 Processor -- start 2023-10-17T01:13:49.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.728+0000 7f4aebf1c700 1 -- start start 2023-10-17T01:13:49.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.728+0000 7f4aebf1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 0x7f4ae419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.728+0000 7f4aebf1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ae4103ce0 0x7f4ae419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4ae9cb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 0x7f4ae419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4ae9cb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 0x7f4ae419d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59262/0 (socket says 172.21.15.45:59262) 2023-10-17T01:13:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4aebf1c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 0x7f4ae41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4aebf1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41a2350 con 0x7f4ae4068870 2023-10-17T01:13:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41a24c0 con 0x7f4ae4105d10 2023-10-17T01:13:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41a2680 con 0x7f4ae4103ce0 2023-10-17T01:13:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4ae9cb8700 1 -- 172.21.15.45:0/2425658895 learned_addr learned my addr 172.21.15.45:0/2425658895 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.729+0000 7f4ae94b7700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ae4103ce0 0x7f4ae419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4aea4b9700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 0x7f4ae41a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4ae9cb8700 1 -- 172.21.15.45:0/2425658895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ae4103ce0 msgr2=0x7f4ae419d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4ae9cb8700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ae4103ce0 0x7f4ae419d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4ae9cb8700 1 -- 172.21.15.45:0/2425658895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 msgr2=0x7f4ae41a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4ae9cb8700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 0x7f4ae41a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4ae9cb8700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad40095f0 con 0x7f4ae4068870 2023-10-17T01:13:49.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4aea4b9700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 0x7f4ae41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:49.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.730+0000 7f4ae9cb8700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 0x7f4ae419d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c9b0 tx=0x7f4ae000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:49.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.731+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0007b00 con 0x7f4ae4068870 2023-10-17T01:13:49.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.731+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ae0007c60 con 0x7f4ae4068870 2023-10-17T01:13:49.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.731+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0018660 con 0x7f4ae4068870 2023-10-17T01:13:49.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.731+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae41a2960 con 0x7f4ae4068870 2023-10-17T01:13:49.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.731+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae41aa1c0 con 0x7f4ae4068870 2023-10-17T01:13:49.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.733+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae404ea50 con 0x7f4ae4068870 2023-10-17T01:13:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.735+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f4ae000f450 con 0x7f4ae4068870 2023-10-17T01:13:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.736+0000 7f4adaffd700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4ad00a0bb0 0x7f4ad00a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.737+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4ae00cc280 con 0x7f4ae4068870 2023-10-17T01:13:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.737+0000 7f4ae94b7700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4ad00a0bb0 0x7f4ad00a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:49.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.737+0000 7f4ae94b7700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4ad00a0bb0 0x7f4ad00a3070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40036c0 tx=0x7f4ad4005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:49.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.741+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f4ae0095a80 con 0x7f4ae4068870 2023-10-17T01:13:49.744 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:49 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/420345731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-17T01:13:49.744 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:49 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/510994561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-17T01:13:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:49 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/420345731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-17T01:13:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:49 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/510994561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-17T01:13:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:49 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/420345731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-17T01:13:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:49 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/510994561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-17T01:13:49.962 INFO:teuthology.orchestra.run.smithi045.stdout:107374182456 2023-10-17T01:13:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.961+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ae41aa350 con 0x7f4ae4068870 2023-10-17T01:13:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.962+0000 7f4adaffd700 1 -- 172.21.15.45:0/2425658895 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ae00990a0 con 0x7f4ae4068870 2023-10-17T01:13:49.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.964+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4ad00a0bb0 msgr2=0x7f4ad00a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.964+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4ad00a0bb0 0x7f4ad00a3070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40036c0 tx=0x7f4ad4005ee0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.964+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 msgr2=0x7f4ae419d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.964+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 0x7f4ae419d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c9b0 tx=0x7f4ae000ccc0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 shutdown_connections 2023-10-17T01:13:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4ad00a0bb0 0x7f4ad00a3070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae4105d10 0x7f4ae41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ae4068870 0x7f4ae419d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 --2- 172.21.15.45:0/2425658895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ae4103ce0 0x7f4ae419d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 >> 172.21.15.45:0/2425658895 conn(0x7f4ae4076d50 msgr2=0x7f4ae4102780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 shutdown_connections 2023-10-17T01:13:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:49.965+0000 7f4aebf1c700 1 -- 172.21.15.45:0/2425658895 wait complete. 2023-10-17T01:13:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 -- 172.21.15.45:0/2544813422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c1060b0 msgr2=0x7f992c10dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 --2- 172.21.15.45:0/2544813422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c1060b0 0x7f992c10dc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f992400b0a0 tx=0x7f992400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 -- 172.21.15.45:0/2544813422 shutdown_connections 2023-10-17T01:13:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 --2- 172.21.15.45:0/2544813422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c1060b0 0x7f992c10dc80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 --2- 172.21.15.45:0/2544813422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 0x7f992c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 --2- 172.21.15.45:0/2544813422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c101980 0x7f992c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 -- 172.21.15.45:0/2544813422 >> 172.21.15.45:0/2544813422 conn(0x7f992c0fc840 msgr2=0x7f992c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 -- 172.21.15.45:0/2544813422 shutdown_connections 2023-10-17T01:13:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.172+0000 7f9933013700 1 -- 172.21.15.45:0/2544813422 wait complete. 2023-10-17T01:13:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.173+0000 7f9933013700 1 Processor -- start 2023-10-17T01:13:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 -- start start 2023-10-17T01:13:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 0x7f992c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 0x7f992c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c1060b0 0x7f992c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c19e150 con 0x7f992c101980 2023-10-17T01:13:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c19e2c0 con 0x7f992c100fd0 2023-10-17T01:13:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.174+0000 7f9933013700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c19e480 con 0x7f992c1060b0 2023-10-17T01:13:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f992bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 0x7f992c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f992bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 0x7f992c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59278/0 (socket says 172.21.15.45:59278) 2023-10-17T01:13:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f992bfff700 1 -- 172.21.15.45:0/2984441976 learned_addr learned my addr 172.21.15.45:0/2984441976 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f9930daf700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 0x7f992c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f99315b0700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c1060b0 0x7f992c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f9930daf700 1 -- 172.21.15.45:0/2984441976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c1060b0 msgr2=0x7f992c19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f9930daf700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c1060b0 0x7f992c19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f9930daf700 1 -- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 msgr2=0x7f992c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f9930daf700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 0x7f992c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f9930daf700 1 -- 172.21.15.45:0/2984441976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924009d00 con 0x7f992c100fd0 2023-10-17T01:13:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f992bfff700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 0x7f992c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.175+0000 7f99315b0700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c1060b0 0x7f992c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.176+0000 7f9930daf700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 0x7f992c199230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f992000e9f0 tx=0x7f992000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.176+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000c9c0 con 0x7f992c100fd0 2023-10-17T01:13:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.176+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9920010430 con 0x7f992c100fd0 2023-10-17T01:13:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.176+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c19e760 con 0x7f992c100fd0 2023-10-17T01:13:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.176+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920018680 con 0x7f992c100fd0 2023-10-17T01:13:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.176+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c1a5fc0 con 0x7f992c100fd0 2023-10-17T01:13:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.177+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c04ea50 con 0x7f992c100fd0 2023-10-17T01:13:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.179+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f9920010860 con 0x7f992c100fd0 2023-10-17T01:13:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.181+0000 7f9929ffb700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f991c0a0c30 0x7f991c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.182+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9920014070 con 0x7f992c100fd0 2023-10-17T01:13:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.182+0000 7f992bfff700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f991c0a0c30 0x7f991c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.182+0000 7f992bfff700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f991c0a0c30 0x7f991c0a30f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f992c19a850 tx=0x7f9918009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.185+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f99200520b0 con 0x7f992c100fd0 2023-10-17T01:13:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.404+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f992c19a590 con 0x7f992c100fd0 2023-10-17T01:13:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.405+0000 7f9929ffb700 1 -- 172.21.15.45:0/2984441976 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9920095bd0 con 0x7f992c100fd0 2023-10-17T01:13:50.406 INFO:teuthology.orchestra.run.smithi045.stdout:77309411390 2023-10-17T01:13:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.407+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f991c0a0c30 msgr2=0x7f991c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.407+0000 7f9933013700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f991c0a0c30 0x7f991c0a30f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f992c19a850 tx=0x7f9918009260 comp rx=0 tx=0).stop 2023-10-17T01:13:50.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.407+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 msgr2=0x7f992c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.407+0000 7f9933013700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 0x7f992c199230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f992000e9f0 tx=0x7f992000ed00 comp rx=0 tx=0).stop 2023-10-17T01:13:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 shutdown_connections 2023-10-17T01:13:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f991c0a0c30 0x7f991c0a30f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f992c1060b0 0x7f992c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f992c100fd0 0x7f992c199230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 --2- 172.21.15.45:0/2984441976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c101980 0x7f992c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 >> 172.21.15.45:0/2984441976 conn(0x7f992c0fc840 msgr2=0x7f992c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 shutdown_connections 2023-10-17T01:13:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:50.408+0000 7f9933013700 1 -- 172.21.15.45:0/2984441976 wait complete. 2023-10-17T01:13:50.657 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449106 got 335007449109 for osd.11 2023-10-17T01:13:50.658 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:50 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2425658895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-17T01:13:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:50 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2984441976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-17T01:13:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:50 smithi060 ceph-mon[134136]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-10-17T01:13:50.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:50 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2425658895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-17T01:13:50.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:50 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2984441976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-17T01:13:50.839 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:50 smithi045 ceph-mon[98474]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-10-17T01:13:50.894 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738437 got 34359738439 for osd.0 2023-10-17T01:13:50.894 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:50.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:50 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2425658895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-17T01:13:50.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:50 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2984441976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-17T01:13:50.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:50 smithi181 ceph-mon[134520]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-10-17T01:13:51.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182454 got 107374182456 for osd.3 2023-10-17T01:13:51.064 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3802259795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 msgr2=0x7ff5e410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:51.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3802259795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e410daa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b0a0 tx=0x7ff5dc00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3802259795 shutdown_connections 2023-10-17T01:13:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3802259795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e410daa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3802259795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5e4100df0 0x7ff5e41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3802259795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e41017a0 0x7ff5e41057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.188+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3802259795 >> 172.21.15.45:0/3802259795 conn(0x7ff5e40fc700 msgr2=0x7ff5e40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.189+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3802259795 shutdown_connections 2023-10-17T01:13:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.189+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3802259795 wait complete. 2023-10-17T01:13:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 Processor -- start 2023-10-17T01:13:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 -- start start 2023-10-17T01:13:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 0x7ff5e4196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5e41017a0 0x7ff5e4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419bda0 con 0x7ff5e4105ed0 2023-10-17T01:13:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419bf10 con 0x7ff5e4100df0 2023-10-17T01:13:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419c0d0 con 0x7ff5e41017a0 2023-10-17T01:13:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 0x7ff5e4196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 0x7ff5e4196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40722/0 (socket says 172.21.15.45:40722) 2023-10-17T01:13:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.190+0000 7ff5e2ffd700 1 -- 172.21.15.45:0/3468974722 learned_addr learned my addr 172.21.15.45:0/3468974722 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e27fc700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5e41017a0 0x7ff5e4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e2ffd700 1 -- 172.21.15.45:0/3468974722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5e41017a0 msgr2=0x7ff5e4197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e2ffd700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5e41017a0 0x7ff5e4197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e2ffd700 1 -- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 msgr2=0x7ff5e419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e2ffd700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e2ffd700 1 -- 172.21.15.45:0/3468974722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc009d00 con 0x7ff5e4100df0 2023-10-17T01:13:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e37fe700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:13:51.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e2ffd700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 0x7ff5e4196de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e9a0 tx=0x7ff5d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4009790 con 0x7ff5e4100df0 2023-10-17T01:13:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d4010430 con 0x7ff5e4100df0 2023-10-17T01:13:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4018680 con 0x7ff5e4100df0 2023-10-17T01:13:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e419c3b0 con 0x7ff5e4100df0 2023-10-17T01:13:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.191+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41a3c10 con 0x7ff5e4100df0 2023-10-17T01:13:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.192+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e404f2a0 con 0x7ff5e4100df0 2023-10-17T01:13:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.195+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7ff5d40105a0 con 0x7ff5e4100df0 2023-10-17T01:13:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.196+0000 7ff5cbfff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5cc0a0bb0 0x7ff5cc0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.196+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff5d4014070 con 0x7ff5e4100df0 2023-10-17T01:13:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.196+0000 7ff5e27fc700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5cc0a0bb0 0x7ff5cc0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.197+0000 7ff5e27fc700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5cc0a0bb0 0x7ff5cc0a3070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4198400 tx=0x7ff5d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.198+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7ff5d4095ed0 con 0x7ff5e4100df0 2023-10-17T01:13:51.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411390 for osd.2 2023-10-17T01:13:51.289 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:51.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.412+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7ff5e41a3ea0 con 0x7ff5e4100df0 2023-10-17T01:13:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.413+0000 7ff5cbfff700 1 -- 172.21.15.45:0/3468974722 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff5d40994f0 con 0x7ff5e4100df0 2023-10-17T01:13:51.415 INFO:teuthology.orchestra.run.smithi045.stdout:304942678042 2023-10-17T01:13:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5cc0a0bb0 msgr2=0x7ff5cc0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5cc0a0bb0 0x7ff5cc0a3070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4198400 tx=0x7ff5d8009260 comp rx=0 tx=0).stop 2023-10-17T01:13:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 msgr2=0x7ff5e4196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 0x7ff5e4196de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e9a0 tx=0x7ff5d400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 shutdown_connections 2023-10-17T01:13:51.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff5cc0a0bb0 0x7ff5cc0a3070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5e4105ed0 0x7ff5e419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e4100df0 0x7ff5e4196de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 --2- 172.21.15.45:0/3468974722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5e41017a0 0x7ff5e4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:51.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 >> 172.21.15.45:0/3468974722 conn(0x7ff5e40fc700 msgr2=0x7ff5e40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:51.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 shutdown_connections 2023-10-17T01:13:51.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:51.415+0000 7ff5e3fff700 1 -- 172.21.15.45:0/3468974722 wait complete. 2023-10-17T01:13:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:51 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3468974722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-17T01:13:51.848 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:51 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3468974722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-17T01:13:51.888 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678040 got 304942678042 for osd.10 2023-10-17T01:13:51.889 DEBUG:teuthology.parallel:result is None 2023-10-17T01:13:51.889 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-17T01:13:51.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-17T01:13:51.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:51 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3468974722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-17T01:13:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:52.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:52 smithi060 ceph-mon[134136]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:13:52.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:52 smithi181 ceph-mon[134520]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:13:52.929 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:52 smithi045 ceph-mon[98474]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:13:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 -- 172.21.15.45:0/4243642352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f41060b0 msgr2=0x7fb2f410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 --2- 172.21.15.45:0/4243642352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f41060b0 0x7fb2f410dc80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b0a0 tx=0x7fb2ec00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 -- 172.21.15.45:0/4243642352 shutdown_connections 2023-10-17T01:13:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 --2- 172.21.15.45:0/4243642352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f41060b0 0x7fb2f410dc80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 --2- 172.21.15.45:0/4243642352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f4100fd0 0x7fb2f41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 --2- 172.21.15.45:0/4243642352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4101980 0x7fb2f41059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 -- 172.21.15.45:0/4243642352 >> 172.21.15.45:0/4243642352 conn(0x7fb2f40fc840 msgr2=0x7fb2f40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 -- 172.21.15.45:0/4243642352 shutdown_connections 2023-10-17T01:13:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.179+0000 7fb2facc0700 1 -- 172.21.15.45:0/4243642352 wait complete. 2023-10-17T01:13:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.180+0000 7fb2facc0700 1 Processor -- start 2023-10-17T01:13:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.180+0000 7fb2facc0700 1 -- start start 2023-10-17T01:13:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2facc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 0x7fb2f4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2facc0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4101980 0x7fb2f4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2facc0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f41060b0 0x7fb2f4073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2facc0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41a5410 con 0x7fb2f4100fd0 2023-10-17T01:13:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2facc0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41a5580 con 0x7fb2f41060b0 2023-10-17T01:13:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2facc0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41a5740 con 0x7fb2f4101980 2023-10-17T01:13:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 0x7fb2f4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 0x7fb2f4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59310/0 (socket says 172.21.15.45:59310) 2023-10-17T01:13:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 -- 172.21.15.45:0/2902947069 learned_addr learned my addr 172.21.15.45:0/2902947069 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f3fff700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4101980 0x7fb2f4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f925d700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f41060b0 0x7fb2f4073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 -- 172.21.15.45:0/2902947069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4101980 msgr2=0x7fb2f4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4101980 0x7fb2f4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 -- 172.21.15.45:0/2902947069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f41060b0 msgr2=0x7fb2f4073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f41060b0 0x7fb2f4073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f8a5c700 1 -- 172.21.15.45:0/2902947069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec009d00 con 0x7fb2f4100fd0 2023-10-17T01:13:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.181+0000 7fb2f925d700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f41060b0 0x7fb2f4073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.182+0000 7fb2f8a5c700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 0x7fb2f4073050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e9a0 tx=0x7fb2e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.182+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8009790 con 0x7fb2f4100fd0 2023-10-17T01:13:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.182+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f41a5a20 con 0x7fb2f4100fd0 2023-10-17T01:13:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.182+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e800cbd0 con 0x7fb2f4100fd0 2023-10-17T01:13:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.182+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8018680 con 0x7fb2f4100fd0 2023-10-17T01:13:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.182+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f41a5f70 con 0x7fb2f4100fd0 2023-10-17T01:13:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.183+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f404ea50 con 0x7fb2f4100fd0 2023-10-17T01:13:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.185+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fb2e800cd40 con 0x7fb2f4100fd0 2023-10-17T01:13:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.187+0000 7fb2f1ffb700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb2e40a0bb0 0x7fb2e40a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.187+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb2e8014070 con 0x7fb2f4100fd0 2023-10-17T01:13:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.187+0000 7fb2f3fff700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb2e40a0bb0 0x7fb2e40a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.188+0000 7fb2f3fff700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb2e40a0bb0 0x7fb2e40a3070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4074840 tx=0x7fb2e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.190+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb2e80520b0 con 0x7fb2f4100fd0 2023-10-17T01:13:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.395+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2f40fd9b0 con 0x7fb2e40a0bb0 2023-10-17T01:13:54.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.407+0000 7fb2f1ffb700 1 -- 172.21.15.45:0/2902947069 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340546 (secure 0 0 0) 0x7fb2f40fd9b0 con 0x7fb2e40a0bb0 2023-10-17T01:13:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb2e40a0bb0 msgr2=0x7fb2e40a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb2e40a0bb0 0x7fb2e40a3070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4074840 tx=0x7fb2e0009260 comp rx=0 tx=0).stop 2023-10-17T01:13:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 msgr2=0x7fb2f4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 0x7fb2f4073050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e9a0 tx=0x7fb2e800ecb0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 shutdown_connections 2023-10-17T01:13:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb2e40a0bb0 0x7fb2e40a3070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f41060b0 0x7fb2f4073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f4100fd0 0x7fb2f4073050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 --2- 172.21.15.45:0/2902947069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2f4101980 0x7fb2f4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 >> 172.21.15.45:0/2902947069 conn(0x7fb2f40fc840 msgr2=0x7fb2f40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 shutdown_connections 2023-10-17T01:13:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:54.412+0000 7fb2facc0700 1 -- 172.21.15.45:0/2902947069 wait complete. 2023-10-17T01:13:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:13:54.819 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-10-17T01:13:52.444237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654064,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":589,"num_read_kb":414,"num_write":412,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334820,"kb_used_data":11396,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607004,"statfs":{"total":1151940427776,"available":1151597572096,"internally_reserved":0,"allocated":11669504,"data_stored":7276588,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":74,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002872"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512010+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:13:32.512010+0000","last_peered":"2023-10-17T01:13:32.512010+0000","last_clean":"2023-10-17T01:13:32.512010+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:13:32.512010+0000","last_undegraded":"2023-10-17T01:13:32.512010+0000","last_fullsized":"2023-10-17T01:13:32.512010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514612+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:13:32.514612+0000","last_peered":"2023-10-17T01:13:32.514612+0000","last_clean":"2023-10-17T01:13:32.514612+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:13:32.514612+0000","last_undegraded":"2023-10-17T01:13:32.514612+0000","last_fullsized":"2023-10-17T01:13:32.514612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514557+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:13:32.514557+0000","last_peered":"2023-10-17T01:13:32.514557+0000","last_clean":"2023-10-17T01:13:32.514557+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:13:32.514557+0000","last_undegraded":"2023-10-17T01:13:32.514557+0000","last_fullsized":"2023-10-17T01:13:32.514557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.519158+0000","last_change":"2023-10-17T01:13:33.514935+0000","last_active":"2023-10-17T01:13:33.519158+0000","last_peered":"2023-10-17T01:13:33.519158+0000","last_clean":"2023-10-17T01:13:33.519158+0000","last_became_active":"2023-10-17T01:13:33.514537+0000","last_became_peered":"2023-10-17T01:13:33.514537+0000","last_unstale":"2023-10-17T01:13:33.519158+0000","last_undegraded":"2023-10-17T01:13:33.519158+0000","last_fullsized":"2023-10-17T01:13:33.519158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:56.247357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515488+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:13:32.515488+0000","last_peered":"2023-10-17T01:13:32.515488+0000","last_clean":"2023-10-17T01:13:32.515488+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:13:32.515488+0000","last_undegraded":"2023-10-17T01:13:32.515488+0000","last_fullsized":"2023-10-17T01:13:32.515488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512118+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:13:32.512118+0000","last_peered":"2023-10-17T01:13:32.512118+0000","last_clean":"2023-10-17T01:13:32.512118+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:13:32.512118+0000","last_undegraded":"2023-10-17T01:13:32.512118+0000","last_fullsized":"2023-10-17T01:13:32.512118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513252+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:13:32.513252+0000","last_peered":"2023-10-17T01:13:32.513252+0000","last_clean":"2023-10-17T01:13:32.513252+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:13:32.513252+0000","last_undegraded":"2023-10-17T01:13:32.513252+0000","last_fullsized":"2023-10-17T01:13:32.513252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512878+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:13:32.512878+0000","last_peered":"2023-10-17T01:13:32.512878+0000","last_clean":"2023-10-17T01:13:32.512878+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:13:32.512878+0000","last_undegraded":"2023-10-17T01:13:32.512878+0000","last_fullsized":"2023-10-17T01:13:32.512878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516735+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:13:32.516735+0000","last_peered":"2023-10-17T01:13:32.516735+0000","last_clean":"2023-10-17T01:13:32.516735+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:13:32.516735+0000","last_undegraded":"2023-10-17T01:13:32.516735+0000","last_fullsized":"2023-10-17T01:13:32.516735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511611+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:13:32.511611+0000","last_peered":"2023-10-17T01:13:32.511611+0000","last_clean":"2023-10-17T01:13:32.511611+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:13:32.511611+0000","last_undegraded":"2023-10-17T01:13:32.511611+0000","last_fullsized":"2023-10-17T01:13:32.511611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513695+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:13:32.513695+0000","last_peered":"2023-10-17T01:13:32.513695+0000","last_clean":"2023-10-17T01:13:32.513695+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:13:32.513695+0000","last_undegraded":"2023-10-17T01:13:32.513695+0000","last_fullsized":"2023-10-17T01:13:32.513695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512481+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:13:32.512481+0000","last_peered":"2023-10-17T01:13:32.512481+0000","last_clean":"2023-10-17T01:13:32.512481+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:13:32.512481+0000","last_undegraded":"2023-10-17T01:13:32.512481+0000","last_fullsized":"2023-10-17T01:13:32.512481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508355+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:13:32.508355+0000","last_peered":"2023-10-17T01:13:32.508355+0000","last_clean":"2023-10-17T01:13:32.508355+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:13:32.508355+0000","last_undegraded":"2023-10-17T01:13:32.508355+0000","last_fullsized":"2023-10-17T01:13:32.508355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512618+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:13:32.512618+0000","last_peered":"2023-10-17T01:13:32.512618+0000","last_clean":"2023-10-17T01:13:32.512618+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:13:32.512618+0000","last_undegraded":"2023-10-17T01:13:32.512618+0000","last_fullsized":"2023-10-17T01:13:32.512618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513523+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:13:32.513523+0000","last_peered":"2023-10-17T01:13:32.513523+0000","last_clean":"2023-10-17T01:13:32.513523+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:13:32.513523+0000","last_undegraded":"2023-10-17T01:13:32.513523+0000","last_fullsized":"2023-10-17T01:13:32.513523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515977+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:13:32.515977+0000","last_peered":"2023-10-17T01:13:32.515977+0000","last_clean":"2023-10-17T01:13:32.515977+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:13:32.515977+0000","last_undegraded":"2023-10-17T01:13:32.515977+0000","last_fullsized":"2023-10-17T01:13:32.515977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512111+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:13:32.512111+0000","last_peered":"2023-10-17T01:13:32.512111+0000","last_clean":"2023-10-17T01:13:32.512111+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:13:32.512111+0000","last_undegraded":"2023-10-17T01:13:32.512111+0000","last_fullsized":"2023-10-17T01:13:32.512111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508497+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:13:32.508497+0000","last_peered":"2023-10-17T01:13:32.508497+0000","last_clean":"2023-10-17T01:13:32.508497+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:13:32.508497+0000","last_undegraded":"2023-10-17T01:13:32.508497+0000","last_fullsized":"2023-10-17T01:13:32.508497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513914+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:13:32.513914+0000","last_peered":"2023-10-17T01:13:32.513914+0000","last_clean":"2023-10-17T01:13:32.513914+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:13:32.513914+0000","last_undegraded":"2023-10-17T01:13:32.513914+0000","last_fullsized":"2023-10-17T01:13:32.513914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:48.518246+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:13:48.518246+0000","last_peered":"2023-10-17T01:13:48.518246+0000","last_clean":"2023-10-17T01:13:48.518246+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:13:48.518246+0000","last_undegraded":"2023-10-17T01:13:48.518246+0000","last_fullsized":"2023-10-17T01:13:48.518246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512614+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:13:32.512614+0000","last_peered":"2023-10-17T01:13:32.512614+0000","last_clean":"2023-10-17T01:13:32.512614+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:13:32.512614+0000","last_undegraded":"2023-10-17T01:13:32.512614+0000","last_fullsized":"2023-10-17T01:13:32.512614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512845+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:32.512845+0000","last_peered":"2023-10-17T01:13:32.512845+0000","last_clean":"2023-10-17T01:13:32.512845+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:32.512845+0000","last_undegraded":"2023-10-17T01:13:32.512845+0000","last_fullsized":"2023-10-17T01:13:32.512845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514581+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:13:32.514581+0000","last_peered":"2023-10-17T01:13:32.514581+0000","last_clean":"2023-10-17T01:13:32.514581+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:13:32.514581+0000","last_undegraded":"2023-10-17T01:13:32.514581+0000","last_fullsized":"2023-10-17T01:13:32.514581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516202+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:32.516202+0000","last_peered":"2023-10-17T01:13:32.516202+0000","last_clean":"2023-10-17T01:13:32.516202+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:32.516202+0000","last_undegraded":"2023-10-17T01:13:32.516202+0000","last_fullsized":"2023-10-17T01:13:32.516202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516186+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:13:32.516186+0000","last_peered":"2023-10-17T01:13:32.516186+0000","last_clean":"2023-10-17T01:13:32.516186+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:13:32.516186+0000","last_undegraded":"2023-10-17T01:13:32.516186+0000","last_fullsized":"2023-10-17T01:13:32.516186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516290+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:32.516290+0000","last_peered":"2023-10-17T01:13:32.516290+0000","last_clean":"2023-10-17T01:13:32.516290+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:32.516290+0000","last_undegraded":"2023-10-17T01:13:32.516290+0000","last_fullsized":"2023-10-17T01:13:32.516290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508296+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:13:32.508296+0000","last_peered":"2023-10-17T01:13:32.508296+0000","last_clean":"2023-10-17T01:13:32.508296+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:13:32.508296+0000","last_undegraded":"2023-10-17T01:13:32.508296+0000","last_fullsized":"2023-10-17T01:13:32.508296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:48.519757+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:13:48.519757+0000","last_peered":"2023-10-17T01:13:48.519757+0000","last_clean":"2023-10-17T01:13:48.519757+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:13:48.519757+0000","last_undegraded":"2023-10-17T01:13:48.519757+0000","last_fullsized":"2023-10-17T01:13:48.519757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515489+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:13:32.515489+0000","last_peered":"2023-10-17T01:13:32.515489+0000","last_clean":"2023-10-17T01:13:32.515489+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:13:32.515489+0000","last_undegraded":"2023-10-17T01:13:32.515489+0000","last_fullsized":"2023-10-17T01:13:32.515489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.029617+0000","last_change":"2023-10-17T01:13:34.029617+0000","last_active":"2023-10-17T01:13:34.029617+0000","last_peered":"2023-10-17T01:13:34.029617+0000","last_clean":"2023-10-17T01:13:34.029617+0000","last_became_active":"2023-10-17T01:13:34.023843+0000","last_became_peered":"2023-10-17T01:13:34.023843+0000","last_unstale":"2023-10-17T01:13:34.029617+0000","last_undegraded":"2023-10-17T01:13:34.029617+0000","last_fullsized":"2023-10-17T01:13:34.029617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:03:37.817216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.131194+0000","last_change":"2023-10-17T01:13:34.130813+0000","last_active":"2023-10-17T01:13:34.131194+0000","last_peered":"2023-10-17T01:13:34.131194+0000","last_clean":"2023-10-17T01:13:34.131194+0000","last_became_active":"2023-10-17T01:13:34.130345+0000","last_became_peered":"2023-10-17T01:13:34.130345+0000","last_unstale":"2023-10-17T01:13:34.131194+0000","last_undegraded":"2023-10-17T01:13:34.131194+0000","last_fullsized":"2023-10-17T01:13:34.131194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:56:47.808654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508250+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:32.508250+0000","last_peered":"2023-10-17T01:13:32.508250+0000","last_clean":"2023-10-17T01:13:32.508250+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:32.508250+0000","last_undegraded":"2023-10-17T01:13:32.508250+0000","last_fullsized":"2023-10-17T01:13:32.508250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512724+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:13:32.512724+0000","last_peered":"2023-10-17T01:13:32.512724+0000","last_clean":"2023-10-17T01:13:32.512724+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:13:32.512724+0000","last_undegraded":"2023-10-17T01:13:32.512724+0000","last_fullsized":"2023-10-17T01:13:32.512724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507782+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:32.507782+0000","last_peered":"2023-10-17T01:13:32.507782+0000","last_clean":"2023-10-17T01:13:32.507782+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:32.507782+0000","last_undegraded":"2023-10-17T01:13:32.507782+0000","last_fullsized":"2023-10-17T01:13:32.507782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.509901+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:13:32.509901+0000","last_peered":"2023-10-17T01:13:32.509901+0000","last_clean":"2023-10-17T01:13:32.509901+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:13:32.509901+0000","last_undegraded":"2023-10-17T01:13:32.509901+0000","last_fullsized":"2023-10-17T01:13:32.509901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":106,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:51.522994+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:13:51.522994+0000","last_peered":"2023-10-17T01:13:51.522994+0000","last_clean":"2023-10-17T01:13:51.522994+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:13:51.522994+0000","last_undegraded":"2023-10-17T01:13:51.522994+0000","last_fullsized":"2023-10-17T01:13:51.522994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516641+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:13:32.516641+0000","last_peered":"2023-10-17T01:13:32.516641+0000","last_clean":"2023-10-17T01:13:32.516641+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:13:32.516641+0000","last_undegraded":"2023-10-17T01:13:32.516641+0000","last_fullsized":"2023-10-17T01:13:32.516641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514532+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:13:32.514532+0000","last_peered":"2023-10-17T01:13:32.514532+0000","last_clean":"2023-10-17T01:13:32.514532+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:13:32.514532+0000","last_undegraded":"2023-10-17T01:13:32.514532+0000","last_fullsized":"2023-10-17T01:13:32.514532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508333+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:32.508333+0000","last_peered":"2023-10-17T01:13:32.508333+0000","last_clean":"2023-10-17T01:13:32.508333+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:32.508333+0000","last_undegraded":"2023-10-17T01:13:32.508333+0000","last_fullsized":"2023-10-17T01:13:32.508333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516291+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:13:32.516291+0000","last_peered":"2023-10-17T01:13:32.516291+0000","last_clean":"2023-10-17T01:13:32.516291+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:13:32.516291+0000","last_undegraded":"2023-10-17T01:13:32.516291+0000","last_fullsized":"2023-10-17T01:13:32.516291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":138,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:51.523103+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:13:51.523103+0000","last_peered":"2023-10-17T01:13:51.523103+0000","last_clean":"2023-10-17T01:13:51.523103+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:13:51.523103+0000","last_undegraded":"2023-10-17T01:13:51.523103+0000","last_fullsized":"2023-10-17T01:13:51.523103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56828,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":102,"num_write":31,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515909+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:13:32.515909+0000","last_peered":"2023-10-17T01:13:32.515909+0000","last_clean":"2023-10-17T01:13:32.515909+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:13:32.515909+0000","last_undegraded":"2023-10-17T01:13:32.515909+0000","last_fullsized":"2023-10-17T01:13:32.515909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513865+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:32.513865+0000","last_peered":"2023-10-17T01:13:32.513865+0000","last_clean":"2023-10-17T01:13:32.513865+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:32.513865+0000","last_undegraded":"2023-10-17T01:13:32.513865+0000","last_fullsized":"2023-10-17T01:13:32.513865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508604+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:13:32.508604+0000","last_peered":"2023-10-17T01:13:32.508604+0000","last_clean":"2023-10-17T01:13:32.508604+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:13:32.508604+0000","last_undegraded":"2023-10-17T01:13:32.508604+0000","last_fullsized":"2023-10-17T01:13:32.508604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508044+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:32.508044+0000","last_peered":"2023-10-17T01:13:32.508044+0000","last_clean":"2023-10-17T01:13:32.508044+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:32.508044+0000","last_undegraded":"2023-10-17T01:13:32.508044+0000","last_fullsized":"2023-10-17T01:13:32.508044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513565+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:13:32.513565+0000","last_peered":"2023-10-17T01:13:32.513565+0000","last_clean":"2023-10-17T01:13:32.513565+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:13:32.513565+0000","last_undegraded":"2023-10-17T01:13:32.513565+0000","last_fullsized":"2023-10-17T01:13:32.513565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:48.519644+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:13:48.519644+0000","last_peered":"2023-10-17T01:13:48.519644+0000","last_clean":"2023-10-17T01:13:48.519644+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:13:48.519644+0000","last_undegraded":"2023-10-17T01:13:48.519644+0000","last_fullsized":"2023-10-17T01:13:48.519644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514268+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:13:32.514268+0000","last_peered":"2023-10-17T01:13:32.514268+0000","last_clean":"2023-10-17T01:13:32.514268+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:13:32.514268+0000","last_undegraded":"2023-10-17T01:13:32.514268+0000","last_fullsized":"2023-10-17T01:13:32.514268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.858283+0000","last_change":"2023-10-17T01:13:33.858283+0000","last_active":"2023-10-17T01:13:33.858283+0000","last_peered":"2023-10-17T01:13:33.858283+0000","last_clean":"2023-10-17T01:13:33.858283+0000","last_became_active":"2023-10-17T01:13:33.844620+0000","last_became_peered":"2023-10-17T01:13:33.844620+0000","last_unstale":"2023-10-17T01:13:33.858283+0000","last_undegraded":"2023-10-17T01:13:33.858283+0000","last_fullsized":"2023-10-17T01:13:33.858283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:48:34.953384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507930+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:13:32.507930+0000","last_peered":"2023-10-17T01:13:32.507930+0000","last_clean":"2023-10-17T01:13:32.507930+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:13:32.507930+0000","last_undegraded":"2023-10-17T01:13:32.507930+0000","last_fullsized":"2023-10-17T01:13:32.507930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516499+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:32.516499+0000","last_peered":"2023-10-17T01:13:32.516499+0000","last_clean":"2023-10-17T01:13:32.516499+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:32.516499+0000","last_undegraded":"2023-10-17T01:13:32.516499+0000","last_fullsized":"2023-10-17T01:13:32.516499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511979+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:13:32.511979+0000","last_peered":"2023-10-17T01:13:32.511979+0000","last_clean":"2023-10-17T01:13:32.511979+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:13:32.511979+0000","last_undegraded":"2023-10-17T01:13:32.511979+0000","last_fullsized":"2023-10-17T01:13:32.511979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521188+0000","last_change":"2023-10-17T01:13:33.521188+0000","last_active":"2023-10-17T01:13:33.521188+0000","last_peered":"2023-10-17T01:13:33.521188+0000","last_clean":"2023-10-17T01:13:33.521188+0000","last_became_active":"2023-10-17T01:13:33.520645+0000","last_became_peered":"2023-10-17T01:13:33.520645+0000","last_unstale":"2023-10-17T01:13:33.521188+0000","last_undegraded":"2023-10-17T01:13:33.521188+0000","last_fullsized":"2023-10-17T01:13:33.521188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:55:42.028467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511374+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:13:32.511374+0000","last_peered":"2023-10-17T01:13:32.511374+0000","last_clean":"2023-10-17T01:13:32.511374+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:13:32.511374+0000","last_undegraded":"2023-10-17T01:13:32.511374+0000","last_fullsized":"2023-10-17T01:13:32.511374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:47.517512+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:13:47.517512+0000","last_peered":"2023-10-17T01:13:47.517512+0000","last_clean":"2023-10-17T01:13:47.517512+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:13:47.517512+0000","last_undegraded":"2023-10-17T01:13:47.517512+0000","last_fullsized":"2023-10-17T01:13:47.517512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511207+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:13:32.511207+0000","last_peered":"2023-10-17T01:13:32.511207+0000","last_clean":"2023-10-17T01:13:32.511207+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:13:32.511207+0000","last_undegraded":"2023-10-17T01:13:32.511207+0000","last_fullsized":"2023-10-17T01:13:32.511207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507878+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:13:32.507878+0000","last_peered":"2023-10-17T01:13:32.507878+0000","last_clean":"2023-10-17T01:13:32.507878+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:13:32.507878+0000","last_undegraded":"2023-10-17T01:13:32.507878+0000","last_fullsized":"2023-10-17T01:13:32.507878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:49.520291+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:13:49.520291+0000","last_peered":"2023-10-17T01:13:49.520291+0000","last_clean":"2023-10-17T01:13:49.520291+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:13:49.520291+0000","last_undegraded":"2023-10-17T01:13:49.520291+0000","last_fullsized":"2023-10-17T01:13:49.520291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513543+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:13:32.513543+0000","last_peered":"2023-10-17T01:13:32.513543+0000","last_clean":"2023-10-17T01:13:32.513543+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:13:32.513543+0000","last_undegraded":"2023-10-17T01:13:32.513543+0000","last_fullsized":"2023-10-17T01:13:32.513543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512805+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:32.512805+0000","last_peered":"2023-10-17T01:13:32.512805+0000","last_clean":"2023-10-17T01:13:32.512805+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:32.512805+0000","last_undegraded":"2023-10-17T01:13:32.512805+0000","last_fullsized":"2023-10-17T01:13:32.512805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514380+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:32.514380+0000","last_peered":"2023-10-17T01:13:32.514380+0000","last_clean":"2023-10-17T01:13:32.514380+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:32.514380+0000","last_undegraded":"2023-10-17T01:13:32.514380+0000","last_fullsized":"2023-10-17T01:13:32.514380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508031+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:13:32.508031+0000","last_peered":"2023-10-17T01:13:32.508031+0000","last_clean":"2023-10-17T01:13:32.508031+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:13:32.508031+0000","last_undegraded":"2023-10-17T01:13:32.508031+0000","last_fullsized":"2023-10-17T01:13:32.508031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521833+0000","last_change":"2023-10-17T01:13:33.521833+0000","last_active":"2023-10-17T01:13:33.521833+0000","last_peered":"2023-10-17T01:13:33.521833+0000","last_clean":"2023-10-17T01:13:33.521833+0000","last_became_active":"2023-10-17T01:13:33.518951+0000","last_became_peered":"2023-10-17T01:13:33.518951+0000","last_unstale":"2023-10-17T01:13:33.521833+0000","last_undegraded":"2023-10-17T01:13:33.521833+0000","last_fullsized":"2023-10-17T01:13:33.521833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:26.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516020+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:32.516020+0000","last_peered":"2023-10-17T01:13:32.516020+0000","last_clean":"2023-10-17T01:13:32.516020+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:32.516020+0000","last_undegraded":"2023-10-17T01:13:32.516020+0000","last_fullsized":"2023-10-17T01:13:32.516020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508086+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:13:32.508086+0000","last_peered":"2023-10-17T01:13:32.508086+0000","last_clean":"2023-10-17T01:13:32.508086+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:13:32.508086+0000","last_undegraded":"2023-10-17T01:13:32.508086+0000","last_fullsized":"2023-10-17T01:13:32.508086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508415+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:13:32.508415+0000","last_peered":"2023-10-17T01:13:32.508415+0000","last_clean":"2023-10-17T01:13:32.508415+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:13:32.508415+0000","last_undegraded":"2023-10-17T01:13:32.508415+0000","last_fullsized":"2023-10-17T01:13:32.508415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514827+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:13:32.514827+0000","last_peered":"2023-10-17T01:13:32.514827+0000","last_clean":"2023-10-17T01:13:32.514827+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:13:32.514827+0000","last_undegraded":"2023-10-17T01:13:32.514827+0000","last_fullsized":"2023-10-17T01:13:32.514827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512549+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:32.512549+0000","last_peered":"2023-10-17T01:13:32.512549+0000","last_clean":"2023-10-17T01:13:32.512549+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:32.512549+0000","last_undegraded":"2023-10-17T01:13:32.512549+0000","last_fullsized":"2023-10-17T01:13:32.512549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514687+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:13:32.514687+0000","last_peered":"2023-10-17T01:13:32.514687+0000","last_clean":"2023-10-17T01:13:32.514687+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:13:32.514687+0000","last_undegraded":"2023-10-17T01:13:32.514687+0000","last_fullsized":"2023-10-17T01:13:32.514687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516068+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:32.516068+0000","last_peered":"2023-10-17T01:13:32.516068+0000","last_clean":"2023-10-17T01:13:32.516068+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:32.516068+0000","last_undegraded":"2023-10-17T01:13:32.516068+0000","last_fullsized":"2023-10-17T01:13:32.516068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515366+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:13:32.515366+0000","last_peered":"2023-10-17T01:13:32.515366+0000","last_clean":"2023-10-17T01:13:32.515366+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:13:32.515366+0000","last_undegraded":"2023-10-17T01:13:32.515366+0000","last_fullsized":"2023-10-17T01:13:32.515366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513931+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:32.513931+0000","last_peered":"2023-10-17T01:13:32.513931+0000","last_clean":"2023-10-17T01:13:32.513931+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:32.513931+0000","last_undegraded":"2023-10-17T01:13:32.513931+0000","last_fullsized":"2023-10-17T01:13:32.513931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511310+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:13:32.511310+0000","last_peered":"2023-10-17T01:13:32.511310+0000","last_clean":"2023-10-17T01:13:32.511310+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:13:32.511310+0000","last_undegraded":"2023-10-17T01:13:32.511310+0000","last_fullsized":"2023-10-17T01:13:32.511310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513716+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:32.513716+0000","last_peered":"2023-10-17T01:13:32.513716+0000","last_clean":"2023-10-17T01:13:32.513716+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:32.513716+0000","last_undegraded":"2023-10-17T01:13:32.513716+0000","last_fullsized":"2023-10-17T01:13:32.513716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514825+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:13:32.514825+0000","last_peered":"2023-10-17T01:13:32.514825+0000","last_clean":"2023-10-17T01:13:32.514825+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:13:32.514825+0000","last_undegraded":"2023-10-17T01:13:32.514825+0000","last_fullsized":"2023-10-17T01:13:32.514825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515566+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:32.515566+0000","last_peered":"2023-10-17T01:13:32.515566+0000","last_clean":"2023-10-17T01:13:32.515566+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:32.515566+0000","last_undegraded":"2023-10-17T01:13:32.515566+0000","last_fullsized":"2023-10-17T01:13:32.515566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513784+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:13:32.513784+0000","last_peered":"2023-10-17T01:13:32.513784+0000","last_clean":"2023-10-17T01:13:32.513784+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:13:32.513784+0000","last_undegraded":"2023-10-17T01:13:32.513784+0000","last_fullsized":"2023-10-17T01:13:32.513784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:46.516670+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:13:46.516670+0000","last_peered":"2023-10-17T01:13:46.516670+0000","last_clean":"2023-10-17T01:13:46.516670+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:13:46.516670+0000","last_undegraded":"2023-10-17T01:13:46.516670+0000","last_fullsized":"2023-10-17T01:13:46.516670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513711+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:13:32.513711+0000","last_peered":"2023-10-17T01:13:32.513711+0000","last_clean":"2023-10-17T01:13:32.513711+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:13:32.513711+0000","last_undegraded":"2023-10-17T01:13:32.513711+0000","last_fullsized":"2023-10-17T01:13:32.513711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508206+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:32.508206+0000","last_peered":"2023-10-17T01:13:32.508206+0000","last_clean":"2023-10-17T01:13:32.508206+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:32.508206+0000","last_undegraded":"2023-10-17T01:13:32.508206+0000","last_fullsized":"2023-10-17T01:13:32.508206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508369+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:32.508369+0000","last_peered":"2023-10-17T01:13:32.508369+0000","last_clean":"2023-10-17T01:13:32.508369+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:32.508369+0000","last_undegraded":"2023-10-17T01:13:32.508369+0000","last_fullsized":"2023-10-17T01:13:32.508369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513826+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:13:32.513826+0000","last_peered":"2023-10-17T01:13:32.513826+0000","last_clean":"2023-10-17T01:13:32.513826+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:13:32.513826+0000","last_undegraded":"2023-10-17T01:13:32.513826+0000","last_fullsized":"2023-10-17T01:13:32.513826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511492+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:13:32.511492+0000","last_peered":"2023-10-17T01:13:32.511492+0000","last_clean":"2023-10-17T01:13:32.511492+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:13:32.511492+0000","last_undegraded":"2023-10-17T01:13:32.511492+0000","last_fullsized":"2023-10-17T01:13:32.511492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513497+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:13:32.513497+0000","last_peered":"2023-10-17T01:13:32.513497+0000","last_clean":"2023-10-17T01:13:32.513497+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:13:32.513497+0000","last_undegraded":"2023-10-17T01:13:32.513497+0000","last_fullsized":"2023-10-17T01:13:32.513497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513633+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:13:32.513633+0000","last_peered":"2023-10-17T01:13:32.513633+0000","last_clean":"2023-10-17T01:13:32.513633+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:13:32.513633+0000","last_undegraded":"2023-10-17T01:13:32.513633+0000","last_fullsized":"2023-10-17T01:13:32.513633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513276+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:13:32.513276+0000","last_peered":"2023-10-17T01:13:32.513276+0000","last_clean":"2023-10-17T01:13:32.513276+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:13:32.513276+0000","last_undegraded":"2023-10-17T01:13:32.513276+0000","last_fullsized":"2023-10-17T01:13:32.513276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508184+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:13:32.508184+0000","last_peered":"2023-10-17T01:13:32.508184+0000","last_clean":"2023-10-17T01:13:32.508184+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:13:32.508184+0000","last_undegraded":"2023-10-17T01:13:32.508184+0000","last_fullsized":"2023-10-17T01:13:32.508184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511795+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:13:32.511795+0000","last_peered":"2023-10-17T01:13:32.511795+0000","last_clean":"2023-10-17T01:13:32.511795+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:13:32.511795+0000","last_undegraded":"2023-10-17T01:13:32.511795+0000","last_fullsized":"2023-10-17T01:13:32.511795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514287+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:13:32.514287+0000","last_peered":"2023-10-17T01:13:32.514287+0000","last_clean":"2023-10-17T01:13:32.514287+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:13:32.514287+0000","last_undegraded":"2023-10-17T01:13:32.514287+0000","last_fullsized":"2023-10-17T01:13:32.514287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514736+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:13:32.514736+0000","last_peered":"2023-10-17T01:13:32.514736+0000","last_clean":"2023-10-17T01:13:32.514736+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:13:32.514736+0000","last_undegraded":"2023-10-17T01:13:32.514736+0000","last_fullsized":"2023-10-17T01:13:32.514736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507853+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:13:32.507853+0000","last_peered":"2023-10-17T01:13:32.507853+0000","last_clean":"2023-10-17T01:13:32.507853+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:13:32.507853+0000","last_undegraded":"2023-10-17T01:13:32.507853+0000","last_fullsized":"2023-10-17T01:13:32.507853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513842+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:13:32.513842+0000","last_peered":"2023-10-17T01:13:32.513842+0000","last_clean":"2023-10-17T01:13:32.513842+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:13:32.513842+0000","last_undegraded":"2023-10-17T01:13:32.513842+0000","last_fullsized":"2023-10-17T01:13:32.513842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516267+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:13:32.516267+0000","last_peered":"2023-10-17T01:13:32.516267+0000","last_clean":"2023-10-17T01:13:32.516267+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:13:32.516267+0000","last_undegraded":"2023-10-17T01:13:32.516267+0000","last_fullsized":"2023-10-17T01:13:32.516267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508271+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:13:32.508271+0000","last_peered":"2023-10-17T01:13:32.508271+0000","last_clean":"2023-10-17T01:13:32.508271+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:13:32.508271+0000","last_undegraded":"2023-10-17T01:13:32.508271+0000","last_fullsized":"2023-10-17T01:13:32.508271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.522423+0000","last_change":"2023-10-17T01:13:33.522423+0000","last_active":"2023-10-17T01:13:33.522423+0000","last_peered":"2023-10-17T01:13:33.522423+0000","last_clean":"2023-10-17T01:13:33.522423+0000","last_became_active":"2023-10-17T01:13:33.521927+0000","last_became_peered":"2023-10-17T01:13:33.521927+0000","last_unstale":"2023-10-17T01:13:33.522423+0000","last_undegraded":"2023-10-17T01:13:33.522423+0000","last_fullsized":"2023-10-17T01:13:33.522423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:07:25.868960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63450,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":463,"num_read_kb":305,"num_write":229,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47099999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.74099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":9,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.72999999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.71199999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":1091974,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40699999999999997}]},{"osd":2,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61199999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.439},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51500000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.60699999999999998}]},{"osd":9,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.73199999999999998}]},{"osd":11,"last update":"Tue Oct 17 01:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.70099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.438},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.58899999999999997,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66000000000000003,"15min":0.67400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.74299999999999999,"15min":0.76100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85499999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.495,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.82499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0600000000000001,"5min":1.216,"15min":1.216},"last":0.629}]},{"osd":3,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89500000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.45300000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.46899999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.90100000000000002,"15min":0.874},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":7.0039999999999996,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.871,"15min":0.89100000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0880000000000001,"5min":1.198,"15min":1.198},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88500000000000001,"15min":0.90900000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0489999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":0.98099999999999998,"15min":0.95499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":6.9379999999999997,"5min":6.9379999999999997,"15min":6.9379999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.071,"5min":1.27,"15min":1.27},"last":0.70799999999999996}]},{"osd":8,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.93700000000000006,"15min":0.95899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2090000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":0.59099999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103203,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75800000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.46899999999999997,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.309},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90200000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.98199999999999998,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.748},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93200000000000005,"5min":1.51,"15min":1.51},"last":0.61499999999999999}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.76900000000000002},"min":{"1min":0.68200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93999999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":1036517,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76000000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.435}]},{"osd":2,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67800000000000005}]},{"osd":3,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.628},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59899999999999998,"15min":0.58599999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.54300000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.70699999999999996}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.04,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69299999999999995}]},{"osd":9,"last update":"Tue Oct 17 01:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.879,"15min":0.89700000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.006,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.038,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.75700000000000001}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.78400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.371,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.626},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.64200000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.411,"5min":1.458,"15min":1.458},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.73199999999999998,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82499999999999996,"15min":0.81299999999999994},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.80400000000000005,"15min":0.78200000000000003},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.90900000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.0580000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.337},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":1.417,"15min":1.417},"last":0.99399999999999999}]},{"osd":9,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.546,"5min":1.657,"15min":1.657},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.80400000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.86299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":501338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1629999999999998,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":0.499}]},{"osd":2,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.53000000000000003},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72299999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88500000000000001,"5min":1.02,"15min":1.02},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68200000000000005}]},{"osd":4,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89700000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.49199999999999999,"15min":0.46500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.59599999999999997},"min":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66000000000000003,"15min":0.64100000000000001},"min":{"1min":0.441,"5min":0.432,"15min":0.432},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.67200000000000004},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.73899999999999999,"15min":0.72899999999999998},"min":{"1min":0.52500000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95499999999999996,"5min":1.262,"15min":1.262},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.018,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74399999999999999}]},{"osd":8,"last update":"Tue Oct 17 01:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77100000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.82799999999999996,"15min":0.82399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.054,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.80300000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58799999999999997,"15min":0.52600000000000002},"min":{"1min":0.40600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78300000000000003,"5min":1.02,"15min":1.02},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.63900000000000001,"15min":0.59099999999999997},"min":{"1min":0.499,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Oct 17 01:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.313,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89100000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.36899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Oct 17 01:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81799999999999995,"5min":1.115,"15min":1.115},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.59999999999999998},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 17 01:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":0.997,"15min":0.997},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Oct 17 01:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.66700000000000004},"min":{"1min":0.39700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.159,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":0.91000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3340000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.82699999999999996,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.117,"5min":1.181,"15min":1.181},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.94699999999999995,"15min":0.96799999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2490000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.79300000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.57699999999999996}]},{"osd":11,"last update":"Tue Oct 17 01:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":501678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.52200000000000002},"min":{"1min":0.376,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.496}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77800000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.40100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72399999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72899999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.40500000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Oct 17 01:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.60599999999999998,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":6.133,"5min":6.133,"15min":6.133},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.66700000000000004,"15min":0.624},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"last":0.625}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.437},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.308},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.38800000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.71799999999999997,"15min":0.66200000000000003},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":0.71599999999999997,"15min":0.65800000000000003},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.41999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.90700000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.42999999999999999}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.59999999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":1.016,"15min":1.016},"last":0.745}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.2060000000000004,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.68799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":443643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.44500000000000001,"15min":0.40899999999999997},"min":{"1min":0.34999999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.433},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44,"15min":0.40400000000000003},"min":{"1min":0.32500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45800000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Oct 17 01:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Oct 17 01:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63600000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.95099999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0860000000000001,"5min":1.161,"15min":1.161},"last":0.69999999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.48999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.006,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68300000000000005,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.026,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Oct 17 01:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.73499999999999999},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82599999999999996,"15min":0.80900000000000005},"min":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.71699999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":0.93500000000000005},"min":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.234,"15min":1.234},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.88400000000000001,"15min":0.90300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.05,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.623}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.56100000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.372},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.57199999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83299999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Oct 17 01:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.56399999999999995},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.621},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.871,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.40500000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.47999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65600000000000003,"15min":0.63900000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.47699999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.018,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.78500000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.749,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.879,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.81200000000000006,"15min":0.78400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":0.64900000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1036622,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73999999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.81100000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.84199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.54100000000000004,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40500000000000003,"15min":0.38900000000000001},"min":{"1min":0.28399999999999997,"5min":0.223,"15min":0.223},"max":{"1min":0.76600000000000001,"5min":1.24,"15min":1.24},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27100000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Oct 17 01:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88200000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.095,"15min":1.095},"last":0.73099999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.622},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68100000000000005}]},{"osd":9,"last update":"Tue Oct 17 01:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.873,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56799999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.83699999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.79200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:13:54.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-17T01:13:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:55 smithi045 ceph-mon[98474]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:55 smithi045 ceph-mon[98474]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:13:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:55 smithi060 ceph-mon[134136]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:55 smithi060 ceph-mon[134136]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:13:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:55 smithi181 ceph-mon[134520]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:55 smithi181 ceph-mon[134520]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:13:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 -- 172.21.15.45:0/3785437899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe041060b0 msgr2=0x7fbe0410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/3785437899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe041060b0 0x7fbe0410dc80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00b0a0 tx=0x7fbdfc00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 -- 172.21.15.45:0/3785437899 shutdown_connections 2023-10-17T01:13:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/3785437899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe041060b0 0x7fbe0410dc80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/3785437899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 0x7fbe041013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/3785437899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe04101980 0x7fbe041059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 -- 172.21.15.45:0/3785437899 >> 172.21.15.45:0/3785437899 conn(0x7fbe040fc840 msgr2=0x7fbe040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 -- 172.21.15.45:0/3785437899 shutdown_connections 2023-10-17T01:13:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.122+0000 7fbe0a13a700 1 -- 172.21.15.45:0/3785437899 wait complete. 2023-10-17T01:13:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.123+0000 7fbe0a13a700 1 Processor -- start 2023-10-17T01:13:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.123+0000 7fbe0a13a700 1 -- start start 2023-10-17T01:13:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe0a13a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 0x7fbe041991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe0a13a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 0x7fbe04199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe0a13a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe041060b0 0x7fbe0419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe0a13a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419e190 con 0x7fbe04101980 2023-10-17T01:13:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe0a13a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419e300 con 0x7fbe041060b0 2023-10-17T01:13:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe0a13a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419e4c0 con 0x7fbe04100fd0 2023-10-17T01:13:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 0x7fbe04199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 0x7fbe041991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:57.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 0x7fbe04199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59316/0 (socket says 172.21.15.45:59316) 2023-10-17T01:13:57.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 -- 172.21.15.45:0/2220385123 learned_addr learned my addr 172.21.15.45:0/2220385123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:13:57.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 0x7fbe041991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46068/0 (socket says 172.21.15.45:46068) 2023-10-17T01:13:57.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe03fff700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe041060b0 0x7fbe0419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:57.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 -- 172.21.15.45:0/2220385123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 msgr2=0x7fbe041991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 0x7fbe041991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 -- 172.21.15.45:0/2220385123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe041060b0 msgr2=0x7fbe0419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe041060b0 0x7fbe0419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe02ffd700 1 -- 172.21.15.45:0/2220385123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc009d00 con 0x7fbe04101980 2023-10-17T01:13:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.124+0000 7fbe03fff700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe041060b0 0x7fbe0419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:13:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.125+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 0x7fbe04199710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e970 tx=0x7fbdf800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.125+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8009790 con 0x7fbe04101980 2023-10-17T01:13:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.125+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf800eec0 con 0x7fbe04101980 2023-10-17T01:13:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.125+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0419e7a0 con 0x7fbe04101980 2023-10-17T01:13:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.125+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8005590 con 0x7fbe04101980 2023-10-17T01:13:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.125+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe041a6000 con 0x7fbe04101980 2023-10-17T01:13:57.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.126+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0404ea50 con 0x7fbe04101980 2023-10-17T01:13:57.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.128+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fbdf800cbd0 con 0x7fbe04101980 2023-10-17T01:13:57.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.130+0000 7fbe00ff9700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbdec0a0bb0 0x7fbdec0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:13:57.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.130+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbdf8014070 con 0x7fbe04101980 2023-10-17T01:13:57.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.130+0000 7fbe037fe700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbdec0a0bb0 0x7fbdec0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:13:57.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.131+0000 7fbe037fe700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbdec0a0bb0 0x7fbdec0a3070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005950 tx=0x7fbdf400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:13:57.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.134+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fbdf8095d30 con 0x7fbe04101980 2023-10-17T01:13:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.341+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe040fd9b0 con 0x7fbdec0a0bb0 2023-10-17T01:13:57.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.352+0000 7fbe00ff9700 1 -- 172.21.15.45:0/2220385123 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340522 (secure 0 0 0) 0x7fbe040fd9b0 con 0x7fbdec0a0bb0 2023-10-17T01:13:57.357 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:13:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.356+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbdec0a0bb0 msgr2=0x7fbdec0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.356+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbdec0a0bb0 0x7fbdec0a3070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005950 tx=0x7fbdf400b410 comp rx=0 tx=0).stop 2023-10-17T01:13:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.356+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 msgr2=0x7fbe04199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:13:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.356+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 0x7fbe04199710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e970 tx=0x7fbdf800ec80 comp rx=0 tx=0).stop 2023-10-17T01:13:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 shutdown_connections 2023-10-17T01:13:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbdec0a0bb0 0x7fbdec0a3070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe041060b0 0x7fbe0419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe04100fd0 0x7fbe041991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 --2- 172.21.15.45:0/2220385123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe04101980 0x7fbe04199710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:13:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 >> 172.21.15.45:0/2220385123 conn(0x7fbe040fc840 msgr2=0x7fbe040fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:13:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 shutdown_connections 2023-10-17T01:13:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:13:57.357+0000 7fbe0a13a700 1 -- 172.21.15.45:0/2220385123 wait complete. 2023-10-17T01:13:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:13:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:57 smithi045 ceph-mon[98474]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:13:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:57 smithi060 ceph-mon[134136]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:13:57.861 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2023-10-17T01:13:56.445184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654064,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":619,"num_read_kb":424,"num_write":412,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334820,"kb_used_data":11396,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607004,"statfs":{"total":1151940427776,"available":1151597572096,"internally_reserved":0,"allocated":11669504,"data_stored":7276588,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002755"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512010+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:13:32.512010+0000","last_peered":"2023-10-17T01:13:32.512010+0000","last_clean":"2023-10-17T01:13:32.512010+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:13:32.512010+0000","last_undegraded":"2023-10-17T01:13:32.512010+0000","last_fullsized":"2023-10-17T01:13:32.512010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514612+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:13:32.514612+0000","last_peered":"2023-10-17T01:13:32.514612+0000","last_clean":"2023-10-17T01:13:32.514612+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:13:32.514612+0000","last_undegraded":"2023-10-17T01:13:32.514612+0000","last_fullsized":"2023-10-17T01:13:32.514612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514557+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:13:32.514557+0000","last_peered":"2023-10-17T01:13:32.514557+0000","last_clean":"2023-10-17T01:13:32.514557+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:13:32.514557+0000","last_undegraded":"2023-10-17T01:13:32.514557+0000","last_fullsized":"2023-10-17T01:13:32.514557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.519158+0000","last_change":"2023-10-17T01:13:33.514935+0000","last_active":"2023-10-17T01:13:33.519158+0000","last_peered":"2023-10-17T01:13:33.519158+0000","last_clean":"2023-10-17T01:13:33.519158+0000","last_became_active":"2023-10-17T01:13:33.514537+0000","last_became_peered":"2023-10-17T01:13:33.514537+0000","last_unstale":"2023-10-17T01:13:33.519158+0000","last_undegraded":"2023-10-17T01:13:33.519158+0000","last_fullsized":"2023-10-17T01:13:33.519158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:56.247357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515488+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:13:32.515488+0000","last_peered":"2023-10-17T01:13:32.515488+0000","last_clean":"2023-10-17T01:13:32.515488+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:13:32.515488+0000","last_undegraded":"2023-10-17T01:13:32.515488+0000","last_fullsized":"2023-10-17T01:13:32.515488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512118+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:13:32.512118+0000","last_peered":"2023-10-17T01:13:32.512118+0000","last_clean":"2023-10-17T01:13:32.512118+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:13:32.512118+0000","last_undegraded":"2023-10-17T01:13:32.512118+0000","last_fullsized":"2023-10-17T01:13:32.512118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513252+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:13:32.513252+0000","last_peered":"2023-10-17T01:13:32.513252+0000","last_clean":"2023-10-17T01:13:32.513252+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:13:32.513252+0000","last_undegraded":"2023-10-17T01:13:32.513252+0000","last_fullsized":"2023-10-17T01:13:32.513252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512878+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:13:32.512878+0000","last_peered":"2023-10-17T01:13:32.512878+0000","last_clean":"2023-10-17T01:13:32.512878+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:13:32.512878+0000","last_undegraded":"2023-10-17T01:13:32.512878+0000","last_fullsized":"2023-10-17T01:13:32.512878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516735+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:13:32.516735+0000","last_peered":"2023-10-17T01:13:32.516735+0000","last_clean":"2023-10-17T01:13:32.516735+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:13:32.516735+0000","last_undegraded":"2023-10-17T01:13:32.516735+0000","last_fullsized":"2023-10-17T01:13:32.516735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511611+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:13:32.511611+0000","last_peered":"2023-10-17T01:13:32.511611+0000","last_clean":"2023-10-17T01:13:32.511611+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:13:32.511611+0000","last_undegraded":"2023-10-17T01:13:32.511611+0000","last_fullsized":"2023-10-17T01:13:32.511611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513695+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:13:32.513695+0000","last_peered":"2023-10-17T01:13:32.513695+0000","last_clean":"2023-10-17T01:13:32.513695+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:13:32.513695+0000","last_undegraded":"2023-10-17T01:13:32.513695+0000","last_fullsized":"2023-10-17T01:13:32.513695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512481+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:13:32.512481+0000","last_peered":"2023-10-17T01:13:32.512481+0000","last_clean":"2023-10-17T01:13:32.512481+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:13:32.512481+0000","last_undegraded":"2023-10-17T01:13:32.512481+0000","last_fullsized":"2023-10-17T01:13:32.512481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508355+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:13:32.508355+0000","last_peered":"2023-10-17T01:13:32.508355+0000","last_clean":"2023-10-17T01:13:32.508355+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:13:32.508355+0000","last_undegraded":"2023-10-17T01:13:32.508355+0000","last_fullsized":"2023-10-17T01:13:32.508355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512618+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:13:32.512618+0000","last_peered":"2023-10-17T01:13:32.512618+0000","last_clean":"2023-10-17T01:13:32.512618+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:13:32.512618+0000","last_undegraded":"2023-10-17T01:13:32.512618+0000","last_fullsized":"2023-10-17T01:13:32.512618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513523+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:13:32.513523+0000","last_peered":"2023-10-17T01:13:32.513523+0000","last_clean":"2023-10-17T01:13:32.513523+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:13:32.513523+0000","last_undegraded":"2023-10-17T01:13:32.513523+0000","last_fullsized":"2023-10-17T01:13:32.513523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515977+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:13:32.515977+0000","last_peered":"2023-10-17T01:13:32.515977+0000","last_clean":"2023-10-17T01:13:32.515977+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:13:32.515977+0000","last_undegraded":"2023-10-17T01:13:32.515977+0000","last_fullsized":"2023-10-17T01:13:32.515977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512111+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:13:32.512111+0000","last_peered":"2023-10-17T01:13:32.512111+0000","last_clean":"2023-10-17T01:13:32.512111+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:13:32.512111+0000","last_undegraded":"2023-10-17T01:13:32.512111+0000","last_fullsized":"2023-10-17T01:13:32.512111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508497+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:13:32.508497+0000","last_peered":"2023-10-17T01:13:32.508497+0000","last_clean":"2023-10-17T01:13:32.508497+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:13:32.508497+0000","last_undegraded":"2023-10-17T01:13:32.508497+0000","last_fullsized":"2023-10-17T01:13:32.508497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513914+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:13:32.513914+0000","last_peered":"2023-10-17T01:13:32.513914+0000","last_clean":"2023-10-17T01:13:32.513914+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:13:32.513914+0000","last_undegraded":"2023-10-17T01:13:32.513914+0000","last_fullsized":"2023-10-17T01:13:32.513914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":75,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:53.524852+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:13:53.524852+0000","last_peered":"2023-10-17T01:13:53.524852+0000","last_clean":"2023-10-17T01:13:53.524852+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:13:53.524852+0000","last_undegraded":"2023-10-17T01:13:53.524852+0000","last_fullsized":"2023-10-17T01:13:53.524852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512614+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:13:32.512614+0000","last_peered":"2023-10-17T01:13:32.512614+0000","last_clean":"2023-10-17T01:13:32.512614+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:13:32.512614+0000","last_undegraded":"2023-10-17T01:13:32.512614+0000","last_fullsized":"2023-10-17T01:13:32.512614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512845+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:32.512845+0000","last_peered":"2023-10-17T01:13:32.512845+0000","last_clean":"2023-10-17T01:13:32.512845+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:32.512845+0000","last_undegraded":"2023-10-17T01:13:32.512845+0000","last_fullsized":"2023-10-17T01:13:32.512845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514581+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:13:32.514581+0000","last_peered":"2023-10-17T01:13:32.514581+0000","last_clean":"2023-10-17T01:13:32.514581+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:13:32.514581+0000","last_undegraded":"2023-10-17T01:13:32.514581+0000","last_fullsized":"2023-10-17T01:13:32.514581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516202+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:32.516202+0000","last_peered":"2023-10-17T01:13:32.516202+0000","last_clean":"2023-10-17T01:13:32.516202+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:32.516202+0000","last_undegraded":"2023-10-17T01:13:32.516202+0000","last_fullsized":"2023-10-17T01:13:32.516202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516186+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:13:32.516186+0000","last_peered":"2023-10-17T01:13:32.516186+0000","last_clean":"2023-10-17T01:13:32.516186+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:13:32.516186+0000","last_undegraded":"2023-10-17T01:13:32.516186+0000","last_fullsized":"2023-10-17T01:13:32.516186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516290+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:32.516290+0000","last_peered":"2023-10-17T01:13:32.516290+0000","last_clean":"2023-10-17T01:13:32.516290+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:32.516290+0000","last_undegraded":"2023-10-17T01:13:32.516290+0000","last_fullsized":"2023-10-17T01:13:32.516290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508296+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:13:32.508296+0000","last_peered":"2023-10-17T01:13:32.508296+0000","last_clean":"2023-10-17T01:13:32.508296+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:13:32.508296+0000","last_undegraded":"2023-10-17T01:13:32.508296+0000","last_fullsized":"2023-10-17T01:13:32.508296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:53.526218+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:13:53.526218+0000","last_peered":"2023-10-17T01:13:53.526218+0000","last_clean":"2023-10-17T01:13:53.526218+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:13:53.526218+0000","last_undegraded":"2023-10-17T01:13:53.526218+0000","last_fullsized":"2023-10-17T01:13:53.526218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515489+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:13:32.515489+0000","last_peered":"2023-10-17T01:13:32.515489+0000","last_clean":"2023-10-17T01:13:32.515489+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:13:32.515489+0000","last_undegraded":"2023-10-17T01:13:32.515489+0000","last_fullsized":"2023-10-17T01:13:32.515489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.029617+0000","last_change":"2023-10-17T01:13:34.029617+0000","last_active":"2023-10-17T01:13:34.029617+0000","last_peered":"2023-10-17T01:13:34.029617+0000","last_clean":"2023-10-17T01:13:34.029617+0000","last_became_active":"2023-10-17T01:13:34.023843+0000","last_became_peered":"2023-10-17T01:13:34.023843+0000","last_unstale":"2023-10-17T01:13:34.029617+0000","last_undegraded":"2023-10-17T01:13:34.029617+0000","last_fullsized":"2023-10-17T01:13:34.029617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:03:37.817216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.131194+0000","last_change":"2023-10-17T01:13:34.130813+0000","last_active":"2023-10-17T01:13:34.131194+0000","last_peered":"2023-10-17T01:13:34.131194+0000","last_clean":"2023-10-17T01:13:34.131194+0000","last_became_active":"2023-10-17T01:13:34.130345+0000","last_became_peered":"2023-10-17T01:13:34.130345+0000","last_unstale":"2023-10-17T01:13:34.131194+0000","last_undegraded":"2023-10-17T01:13:34.131194+0000","last_fullsized":"2023-10-17T01:13:34.131194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:56:47.808654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508250+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:32.508250+0000","last_peered":"2023-10-17T01:13:32.508250+0000","last_clean":"2023-10-17T01:13:32.508250+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:32.508250+0000","last_undegraded":"2023-10-17T01:13:32.508250+0000","last_fullsized":"2023-10-17T01:13:32.508250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512724+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:13:32.512724+0000","last_peered":"2023-10-17T01:13:32.512724+0000","last_clean":"2023-10-17T01:13:32.512724+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:13:32.512724+0000","last_undegraded":"2023-10-17T01:13:32.512724+0000","last_fullsized":"2023-10-17T01:13:32.512724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507782+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:32.507782+0000","last_peered":"2023-10-17T01:13:32.507782+0000","last_clean":"2023-10-17T01:13:32.507782+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:32.507782+0000","last_undegraded":"2023-10-17T01:13:32.507782+0000","last_fullsized":"2023-10-17T01:13:32.507782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.509901+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:13:32.509901+0000","last_peered":"2023-10-17T01:13:32.509901+0000","last_clean":"2023-10-17T01:13:32.509901+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:13:32.509901+0000","last_undegraded":"2023-10-17T01:13:32.509901+0000","last_fullsized":"2023-10-17T01:13:32.509901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":106,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:51.522994+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:13:51.522994+0000","last_peered":"2023-10-17T01:13:51.522994+0000","last_clean":"2023-10-17T01:13:51.522994+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:13:51.522994+0000","last_undegraded":"2023-10-17T01:13:51.522994+0000","last_fullsized":"2023-10-17T01:13:51.522994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516641+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:13:32.516641+0000","last_peered":"2023-10-17T01:13:32.516641+0000","last_clean":"2023-10-17T01:13:32.516641+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:13:32.516641+0000","last_undegraded":"2023-10-17T01:13:32.516641+0000","last_fullsized":"2023-10-17T01:13:32.516641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514532+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:13:32.514532+0000","last_peered":"2023-10-17T01:13:32.514532+0000","last_clean":"2023-10-17T01:13:32.514532+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:13:32.514532+0000","last_undegraded":"2023-10-17T01:13:32.514532+0000","last_fullsized":"2023-10-17T01:13:32.514532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508333+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:32.508333+0000","last_peered":"2023-10-17T01:13:32.508333+0000","last_clean":"2023-10-17T01:13:32.508333+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:32.508333+0000","last_undegraded":"2023-10-17T01:13:32.508333+0000","last_fullsized":"2023-10-17T01:13:32.508333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516291+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:13:32.516291+0000","last_peered":"2023-10-17T01:13:32.516291+0000","last_clean":"2023-10-17T01:13:32.516291+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:13:32.516291+0000","last_undegraded":"2023-10-17T01:13:32.516291+0000","last_fullsized":"2023-10-17T01:13:32.516291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":138,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:51.523103+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:13:51.523103+0000","last_peered":"2023-10-17T01:13:51.523103+0000","last_clean":"2023-10-17T01:13:51.523103+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:13:51.523103+0000","last_undegraded":"2023-10-17T01:13:51.523103+0000","last_fullsized":"2023-10-17T01:13:51.523103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56828,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":102,"num_write":31,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515909+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:13:32.515909+0000","last_peered":"2023-10-17T01:13:32.515909+0000","last_clean":"2023-10-17T01:13:32.515909+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:13:32.515909+0000","last_undegraded":"2023-10-17T01:13:32.515909+0000","last_fullsized":"2023-10-17T01:13:32.515909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513865+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:32.513865+0000","last_peered":"2023-10-17T01:13:32.513865+0000","last_clean":"2023-10-17T01:13:32.513865+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:32.513865+0000","last_undegraded":"2023-10-17T01:13:32.513865+0000","last_fullsized":"2023-10-17T01:13:32.513865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508604+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:13:32.508604+0000","last_peered":"2023-10-17T01:13:32.508604+0000","last_clean":"2023-10-17T01:13:32.508604+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:13:32.508604+0000","last_undegraded":"2023-10-17T01:13:32.508604+0000","last_fullsized":"2023-10-17T01:13:32.508604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508044+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:32.508044+0000","last_peered":"2023-10-17T01:13:32.508044+0000","last_clean":"2023-10-17T01:13:32.508044+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:32.508044+0000","last_undegraded":"2023-10-17T01:13:32.508044+0000","last_fullsized":"2023-10-17T01:13:32.508044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513565+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:13:32.513565+0000","last_peered":"2023-10-17T01:13:32.513565+0000","last_clean":"2023-10-17T01:13:32.513565+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:13:32.513565+0000","last_undegraded":"2023-10-17T01:13:32.513565+0000","last_fullsized":"2023-10-17T01:13:32.513565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:53.526419+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:13:53.526419+0000","last_peered":"2023-10-17T01:13:53.526419+0000","last_clean":"2023-10-17T01:13:53.526419+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:13:53.526419+0000","last_undegraded":"2023-10-17T01:13:53.526419+0000","last_fullsized":"2023-10-17T01:13:53.526419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514268+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:13:32.514268+0000","last_peered":"2023-10-17T01:13:32.514268+0000","last_clean":"2023-10-17T01:13:32.514268+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:13:32.514268+0000","last_undegraded":"2023-10-17T01:13:32.514268+0000","last_fullsized":"2023-10-17T01:13:32.514268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.858283+0000","last_change":"2023-10-17T01:13:33.858283+0000","last_active":"2023-10-17T01:13:33.858283+0000","last_peered":"2023-10-17T01:13:33.858283+0000","last_clean":"2023-10-17T01:13:33.858283+0000","last_became_active":"2023-10-17T01:13:33.844620+0000","last_became_peered":"2023-10-17T01:13:33.844620+0000","last_unstale":"2023-10-17T01:13:33.858283+0000","last_undegraded":"2023-10-17T01:13:33.858283+0000","last_fullsized":"2023-10-17T01:13:33.858283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:48:34.953384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507930+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:13:32.507930+0000","last_peered":"2023-10-17T01:13:32.507930+0000","last_clean":"2023-10-17T01:13:32.507930+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:13:32.507930+0000","last_undegraded":"2023-10-17T01:13:32.507930+0000","last_fullsized":"2023-10-17T01:13:32.507930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516499+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:32.516499+0000","last_peered":"2023-10-17T01:13:32.516499+0000","last_clean":"2023-10-17T01:13:32.516499+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:32.516499+0000","last_undegraded":"2023-10-17T01:13:32.516499+0000","last_fullsized":"2023-10-17T01:13:32.516499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511979+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:13:32.511979+0000","last_peered":"2023-10-17T01:13:32.511979+0000","last_clean":"2023-10-17T01:13:32.511979+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:13:32.511979+0000","last_undegraded":"2023-10-17T01:13:32.511979+0000","last_fullsized":"2023-10-17T01:13:32.511979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521188+0000","last_change":"2023-10-17T01:13:33.521188+0000","last_active":"2023-10-17T01:13:33.521188+0000","last_peered":"2023-10-17T01:13:33.521188+0000","last_clean":"2023-10-17T01:13:33.521188+0000","last_became_active":"2023-10-17T01:13:33.520645+0000","last_became_peered":"2023-10-17T01:13:33.520645+0000","last_unstale":"2023-10-17T01:13:33.521188+0000","last_undegraded":"2023-10-17T01:13:33.521188+0000","last_fullsized":"2023-10-17T01:13:33.521188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:55:42.028467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511374+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:13:32.511374+0000","last_peered":"2023-10-17T01:13:32.511374+0000","last_clean":"2023-10-17T01:13:32.511374+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:13:32.511374+0000","last_undegraded":"2023-10-17T01:13:32.511374+0000","last_fullsized":"2023-10-17T01:13:32.511374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:52.524152+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:13:52.524152+0000","last_peered":"2023-10-17T01:13:52.524152+0000","last_clean":"2023-10-17T01:13:52.524152+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:13:52.524152+0000","last_undegraded":"2023-10-17T01:13:52.524152+0000","last_fullsized":"2023-10-17T01:13:52.524152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511207+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:13:32.511207+0000","last_peered":"2023-10-17T01:13:32.511207+0000","last_clean":"2023-10-17T01:13:32.511207+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:13:32.511207+0000","last_undegraded":"2023-10-17T01:13:32.511207+0000","last_fullsized":"2023-10-17T01:13:32.511207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507878+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:13:32.507878+0000","last_peered":"2023-10-17T01:13:32.507878+0000","last_clean":"2023-10-17T01:13:32.507878+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:13:32.507878+0000","last_undegraded":"2023-10-17T01:13:32.507878+0000","last_fullsized":"2023-10-17T01:13:32.507878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:54.527114+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:13:54.527114+0000","last_peered":"2023-10-17T01:13:54.527114+0000","last_clean":"2023-10-17T01:13:54.527114+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:13:54.527114+0000","last_undegraded":"2023-10-17T01:13:54.527114+0000","last_fullsized":"2023-10-17T01:13:54.527114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513543+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:13:32.513543+0000","last_peered":"2023-10-17T01:13:32.513543+0000","last_clean":"2023-10-17T01:13:32.513543+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:13:32.513543+0000","last_undegraded":"2023-10-17T01:13:32.513543+0000","last_fullsized":"2023-10-17T01:13:32.513543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512805+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:32.512805+0000","last_peered":"2023-10-17T01:13:32.512805+0000","last_clean":"2023-10-17T01:13:32.512805+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:32.512805+0000","last_undegraded":"2023-10-17T01:13:32.512805+0000","last_fullsized":"2023-10-17T01:13:32.512805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514380+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:32.514380+0000","last_peered":"2023-10-17T01:13:32.514380+0000","last_clean":"2023-10-17T01:13:32.514380+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:32.514380+0000","last_undegraded":"2023-10-17T01:13:32.514380+0000","last_fullsized":"2023-10-17T01:13:32.514380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508031+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:13:32.508031+0000","last_peered":"2023-10-17T01:13:32.508031+0000","last_clean":"2023-10-17T01:13:32.508031+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:13:32.508031+0000","last_undegraded":"2023-10-17T01:13:32.508031+0000","last_fullsized":"2023-10-17T01:13:32.508031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521833+0000","last_change":"2023-10-17T01:13:33.521833+0000","last_active":"2023-10-17T01:13:33.521833+0000","last_peered":"2023-10-17T01:13:33.521833+0000","last_clean":"2023-10-17T01:13:33.521833+0000","last_became_active":"2023-10-17T01:13:33.518951+0000","last_became_peered":"2023-10-17T01:13:33.518951+0000","last_unstale":"2023-10-17T01:13:33.521833+0000","last_undegraded":"2023-10-17T01:13:33.521833+0000","last_fullsized":"2023-10-17T01:13:33.521833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:26.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516020+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:32.516020+0000","last_peered":"2023-10-17T01:13:32.516020+0000","last_clean":"2023-10-17T01:13:32.516020+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:32.516020+0000","last_undegraded":"2023-10-17T01:13:32.516020+0000","last_fullsized":"2023-10-17T01:13:32.516020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508086+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:13:32.508086+0000","last_peered":"2023-10-17T01:13:32.508086+0000","last_clean":"2023-10-17T01:13:32.508086+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:13:32.508086+0000","last_undegraded":"2023-10-17T01:13:32.508086+0000","last_fullsized":"2023-10-17T01:13:32.508086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508415+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:13:32.508415+0000","last_peered":"2023-10-17T01:13:32.508415+0000","last_clean":"2023-10-17T01:13:32.508415+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:13:32.508415+0000","last_undegraded":"2023-10-17T01:13:32.508415+0000","last_fullsized":"2023-10-17T01:13:32.508415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514827+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:13:32.514827+0000","last_peered":"2023-10-17T01:13:32.514827+0000","last_clean":"2023-10-17T01:13:32.514827+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:13:32.514827+0000","last_undegraded":"2023-10-17T01:13:32.514827+0000","last_fullsized":"2023-10-17T01:13:32.514827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512549+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:32.512549+0000","last_peered":"2023-10-17T01:13:32.512549+0000","last_clean":"2023-10-17T01:13:32.512549+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:32.512549+0000","last_undegraded":"2023-10-17T01:13:32.512549+0000","last_fullsized":"2023-10-17T01:13:32.512549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514687+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:13:32.514687+0000","last_peered":"2023-10-17T01:13:32.514687+0000","last_clean":"2023-10-17T01:13:32.514687+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:13:32.514687+0000","last_undegraded":"2023-10-17T01:13:32.514687+0000","last_fullsized":"2023-10-17T01:13:32.514687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516068+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:32.516068+0000","last_peered":"2023-10-17T01:13:32.516068+0000","last_clean":"2023-10-17T01:13:32.516068+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:32.516068+0000","last_undegraded":"2023-10-17T01:13:32.516068+0000","last_fullsized":"2023-10-17T01:13:32.516068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515366+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:13:32.515366+0000","last_peered":"2023-10-17T01:13:32.515366+0000","last_clean":"2023-10-17T01:13:32.515366+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:13:32.515366+0000","last_undegraded":"2023-10-17T01:13:32.515366+0000","last_fullsized":"2023-10-17T01:13:32.515366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513931+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:32.513931+0000","last_peered":"2023-10-17T01:13:32.513931+0000","last_clean":"2023-10-17T01:13:32.513931+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:32.513931+0000","last_undegraded":"2023-10-17T01:13:32.513931+0000","last_fullsized":"2023-10-17T01:13:32.513931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511310+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:13:32.511310+0000","last_peered":"2023-10-17T01:13:32.511310+0000","last_clean":"2023-10-17T01:13:32.511310+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:13:32.511310+0000","last_undegraded":"2023-10-17T01:13:32.511310+0000","last_fullsized":"2023-10-17T01:13:32.511310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513716+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:32.513716+0000","last_peered":"2023-10-17T01:13:32.513716+0000","last_clean":"2023-10-17T01:13:32.513716+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:32.513716+0000","last_undegraded":"2023-10-17T01:13:32.513716+0000","last_fullsized":"2023-10-17T01:13:32.513716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514825+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:13:32.514825+0000","last_peered":"2023-10-17T01:13:32.514825+0000","last_clean":"2023-10-17T01:13:32.514825+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:13:32.514825+0000","last_undegraded":"2023-10-17T01:13:32.514825+0000","last_fullsized":"2023-10-17T01:13:32.514825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515566+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:32.515566+0000","last_peered":"2023-10-17T01:13:32.515566+0000","last_clean":"2023-10-17T01:13:32.515566+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:32.515566+0000","last_undegraded":"2023-10-17T01:13:32.515566+0000","last_fullsized":"2023-10-17T01:13:32.515566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513784+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:13:32.513784+0000","last_peered":"2023-10-17T01:13:32.513784+0000","last_clean":"2023-10-17T01:13:32.513784+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:13:32.513784+0000","last_undegraded":"2023-10-17T01:13:32.513784+0000","last_fullsized":"2023-10-17T01:13:32.513784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:51.523495+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:13:51.523495+0000","last_peered":"2023-10-17T01:13:51.523495+0000","last_clean":"2023-10-17T01:13:51.523495+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:13:51.523495+0000","last_undegraded":"2023-10-17T01:13:51.523495+0000","last_fullsized":"2023-10-17T01:13:51.523495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513711+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:13:32.513711+0000","last_peered":"2023-10-17T01:13:32.513711+0000","last_clean":"2023-10-17T01:13:32.513711+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:13:32.513711+0000","last_undegraded":"2023-10-17T01:13:32.513711+0000","last_fullsized":"2023-10-17T01:13:32.513711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508206+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:32.508206+0000","last_peered":"2023-10-17T01:13:32.508206+0000","last_clean":"2023-10-17T01:13:32.508206+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:32.508206+0000","last_undegraded":"2023-10-17T01:13:32.508206+0000","last_fullsized":"2023-10-17T01:13:32.508206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508369+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:32.508369+0000","last_peered":"2023-10-17T01:13:32.508369+0000","last_clean":"2023-10-17T01:13:32.508369+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:32.508369+0000","last_undegraded":"2023-10-17T01:13:32.508369+0000","last_fullsized":"2023-10-17T01:13:32.508369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513826+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:13:32.513826+0000","last_peered":"2023-10-17T01:13:32.513826+0000","last_clean":"2023-10-17T01:13:32.513826+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:13:32.513826+0000","last_undegraded":"2023-10-17T01:13:32.513826+0000","last_fullsized":"2023-10-17T01:13:32.513826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511492+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:13:32.511492+0000","last_peered":"2023-10-17T01:13:32.511492+0000","last_clean":"2023-10-17T01:13:32.511492+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:13:32.511492+0000","last_undegraded":"2023-10-17T01:13:32.511492+0000","last_fullsized":"2023-10-17T01:13:32.511492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513497+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:13:32.513497+0000","last_peered":"2023-10-17T01:13:32.513497+0000","last_clean":"2023-10-17T01:13:32.513497+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:13:32.513497+0000","last_undegraded":"2023-10-17T01:13:32.513497+0000","last_fullsized":"2023-10-17T01:13:32.513497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513633+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:13:32.513633+0000","last_peered":"2023-10-17T01:13:32.513633+0000","last_clean":"2023-10-17T01:13:32.513633+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:13:32.513633+0000","last_undegraded":"2023-10-17T01:13:32.513633+0000","last_fullsized":"2023-10-17T01:13:32.513633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513276+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:13:32.513276+0000","last_peered":"2023-10-17T01:13:32.513276+0000","last_clean":"2023-10-17T01:13:32.513276+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:13:32.513276+0000","last_undegraded":"2023-10-17T01:13:32.513276+0000","last_fullsized":"2023-10-17T01:13:32.513276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508184+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:13:32.508184+0000","last_peered":"2023-10-17T01:13:32.508184+0000","last_clean":"2023-10-17T01:13:32.508184+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:13:32.508184+0000","last_undegraded":"2023-10-17T01:13:32.508184+0000","last_fullsized":"2023-10-17T01:13:32.508184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.511795+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:13:32.511795+0000","last_peered":"2023-10-17T01:13:32.511795+0000","last_clean":"2023-10-17T01:13:32.511795+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:13:32.511795+0000","last_undegraded":"2023-10-17T01:13:32.511795+0000","last_fullsized":"2023-10-17T01:13:32.511795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514287+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:13:32.514287+0000","last_peered":"2023-10-17T01:13:32.514287+0000","last_clean":"2023-10-17T01:13:32.514287+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:13:32.514287+0000","last_undegraded":"2023-10-17T01:13:32.514287+0000","last_fullsized":"2023-10-17T01:13:32.514287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514736+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:13:32.514736+0000","last_peered":"2023-10-17T01:13:32.514736+0000","last_clean":"2023-10-17T01:13:32.514736+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:13:32.514736+0000","last_undegraded":"2023-10-17T01:13:32.514736+0000","last_fullsized":"2023-10-17T01:13:32.514736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507853+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:13:32.507853+0000","last_peered":"2023-10-17T01:13:32.507853+0000","last_clean":"2023-10-17T01:13:32.507853+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:13:32.507853+0000","last_undegraded":"2023-10-17T01:13:32.507853+0000","last_fullsized":"2023-10-17T01:13:32.507853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513842+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:13:32.513842+0000","last_peered":"2023-10-17T01:13:32.513842+0000","last_clean":"2023-10-17T01:13:32.513842+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:13:32.513842+0000","last_undegraded":"2023-10-17T01:13:32.513842+0000","last_fullsized":"2023-10-17T01:13:32.513842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516267+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:13:32.516267+0000","last_peered":"2023-10-17T01:13:32.516267+0000","last_clean":"2023-10-17T01:13:32.516267+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:13:32.516267+0000","last_undegraded":"2023-10-17T01:13:32.516267+0000","last_fullsized":"2023-10-17T01:13:32.516267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508271+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:13:32.508271+0000","last_peered":"2023-10-17T01:13:32.508271+0000","last_clean":"2023-10-17T01:13:32.508271+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:13:32.508271+0000","last_undegraded":"2023-10-17T01:13:32.508271+0000","last_fullsized":"2023-10-17T01:13:32.508271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.522423+0000","last_change":"2023-10-17T01:13:33.522423+0000","last_active":"2023-10-17T01:13:33.522423+0000","last_peered":"2023-10-17T01:13:33.522423+0000","last_clean":"2023-10-17T01:13:33.522423+0000","last_became_active":"2023-10-17T01:13:33.521927+0000","last_became_peered":"2023-10-17T01:13:33.521927+0000","last_unstale":"2023-10-17T01:13:33.522423+0000","last_undegraded":"2023-10-17T01:13:33.522423+0000","last_fullsized":"2023-10-17T01:13:33.522423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:07:25.868960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63450,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":493,"num_read_kb":315,"num_write":229,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449110,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73099999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":9,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.78100000000000003}]},{"osd":10,"last update":"Tue Oct 17 01:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.71899999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":1091974,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40699999999999997}]},{"osd":2,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61199999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.439},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51500000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.60699999999999998}]},{"osd":9,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.73199999999999998}]},{"osd":11,"last update":"Tue Oct 17 01:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.70099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841568,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.58899999999999997,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66800000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66000000000000003,"15min":0.67400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.74299999999999999,"15min":0.76100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85499999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59399999999999997}]},{"osd":2,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.495,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.80500000000000005,"15min":0.82499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0600000000000001,"5min":1.216,"15min":1.216},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89500000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.62},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.46899999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.90100000000000002,"15min":0.874},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":7.0039999999999996,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.89800000000000002}]},{"osd":5,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.871,"15min":0.89100000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0880000000000001,"5min":1.198,"15min":1.198},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88500000000000001,"15min":0.90900000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0489999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":0.98099999999999998,"15min":0.95499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":6.9379999999999997,"5min":6.9379999999999997,"15min":6.9379999999999997},"last":0.754},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.071,"5min":1.27,"15min":1.27},"last":0.82199999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.93700000000000006,"15min":0.95899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2090000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":0.85099999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.93799999999999994},"min":{"1min":0.627,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.077,"5min":1.218,"15min":1.218},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0189999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75800000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.379}]},{"osd":1,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.46899999999999997,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.309,"15min":0.309},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.623}]},{"osd":5,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90200000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.98199999999999998,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93200000000000005,"5min":1.51,"15min":1.51},"last":0.85899999999999999}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.63400000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.76900000000000002},"min":{"1min":0.68200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93999999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76800000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.755,"15min":0.748},"min":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.67700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98799999999999999,"5min":0.996,"15min":0.996},"last":0.67700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":1036517,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64700000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47299999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":1.373,"15min":1.373},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85499999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.621}]},{"osd":3,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83199999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81499999999999995,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84299999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99199999999999999,"5min":1.879,"15min":1.879},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0660000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.71699999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.04,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Tue Oct 17 01:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.879,"15min":0.89700000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.006,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.038,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.84799999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.75900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495662,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.498},"min":{"1min":0.34799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.504},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.371,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.442}]},{"osd":1,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.78900000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.64200000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.411,"5min":1.458,"15min":1.458},"last":0.626},{"interface":"front","average":{"1min":0.752,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.53000000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.52300000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.73199999999999998,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82499999999999996,"15min":0.81299999999999994},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.80500000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.80400000000000005,"15min":0.78200000000000003},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.77400000000000002}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.89600000000000002,"15min":0.90400000000000003},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.0580000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":1.417,"15min":1.417},"last":0.745}]},{"osd":9,"last update":"Tue Oct 17 01:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.546,"5min":1.657,"15min":1.657},"last":0.873},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.64000000000000001}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.82799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":501338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1629999999999998,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":0.56100000000000005}]},{"osd":2,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.53000000000000003},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60299999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.88500000000000001,"5min":1.02,"15min":1.02},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":2.46}]},{"osd":4,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89700000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.49199999999999999,"15min":0.46500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.441}]},{"osd":5,"last update":"Tue Oct 17 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.59599999999999997},"min":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66000000000000003,"15min":0.64100000000000001},"min":{"1min":0.441,"5min":0.432,"15min":0.432},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.67200000000000004},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.504},{"interface":"front","average":{"1min":0.752,"5min":0.73899999999999999,"15min":0.72899999999999998},"min":{"1min":0.52500000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95499999999999996,"5min":1.262,"15min":1.262},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.018,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61599999999999999}]},{"osd":8,"last update":"Tue Oct 17 01:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.79100000000000004}]},{"osd":9,"last update":"Tue Oct 17 01:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":2.4980000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.82799999999999996,"15min":0.82399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.054,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64800000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58799999999999997,"15min":0.52600000000000002},"min":{"1min":0.40600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78300000000000003,"5min":1.02,"15min":1.02},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Oct 17 01:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.63900000000000001,"15min":0.59099999999999997},"min":{"1min":0.499,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Oct 17 01:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.313,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89100000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.36899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Oct 17 01:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81799999999999995,"5min":1.115,"15min":1.115},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.59999999999999998},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 17 01:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":0.997,"15min":0.997},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Oct 17 01:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.66700000000000004},"min":{"1min":0.39700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.159,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":0.91000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3340000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.82699999999999996,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.117,"5min":1.181,"15min":1.181},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.94699999999999995,"15min":0.96799999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2490000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.79300000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.57699999999999996}]},{"osd":11,"last update":"Tue Oct 17 01:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":501678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.52200000000000002},"min":{"1min":0.376,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.871,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.44700000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77800000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.40100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72399999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71499999999999997}]},{"osd":3,"last update":"Tue Oct 17 01:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.40500000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.60599999999999998,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":6.133,"5min":6.133,"15min":6.133},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.66700000000000004,"15min":0.624},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.437},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Oct 17 01:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.71799999999999997,"15min":0.66200000000000003},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.9950000000000001,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.0349999999999999,"5min":0.71599999999999997,"15min":0.65800000000000003},"min":{"1min":0.501,"5min":0.439,"15min":0.439},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.41999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.90700000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75900000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.59999999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":1.016,"15min":1.016},"last":0.74099999999999999}]},{"osd":10,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.2060000000000004,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":443643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.44500000000000001,"15min":0.40899999999999997},"min":{"1min":0.34999999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.435},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44,"15min":0.40400000000000003},"min":{"1min":0.32500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Oct 17 01:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92200000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67600000000000005,"15min":0.64100000000000001},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63600000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.95099999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.47699999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.755,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0860000000000001,"5min":1.161,"15min":1.161},"last":0.73999999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.48999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.006,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68300000000000005,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.026,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Oct 17 01:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.73499999999999999},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82599999999999996,"15min":0.80900000000000005},"min":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66200000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":0.93500000000000005},"min":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.234,"15min":1.234},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.88400000000000001,"15min":0.90300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.05,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.64000000000000001}]},{"osd":10,"last update":"Tue Oct 17 01:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.35599999999999998},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.56100000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.372},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.57199999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83299999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55300000000000005}]},{"osd":3,"last update":"Tue Oct 17 01:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.56399999999999995},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.54600000000000004},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.871,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81799999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77100000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":1.044,"15min":1.044},"last":0.63700000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.755},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.018,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73599999999999999}]},{"osd":9,"last update":"Tue Oct 17 01:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.749,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.879,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.81200000000000006,"15min":0.78400000000000003},"min":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":0.68500000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.624},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1036622,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73999999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.81100000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57299999999999995}]},{"osd":1,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.84199999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.629,"5min":0.54100000000000004,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40500000000000003,"15min":0.38900000000000001},"min":{"1min":0.28399999999999997,"5min":0.223,"15min":0.223},"max":{"1min":0.76600000000000001,"5min":1.24,"15min":1.24},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27100000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77400000000000002,"5min":0.877,"15min":0.877},"last":1.212},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.2250000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.626}]},{"osd":6,"last update":"Tue Oct 17 01:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79900000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.1990000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88200000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.095,"15min":1.095},"last":0.58999999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.622},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.873,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56799999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73499999999999999}]},{"osd":10,"last update":"Tue Oct 17 01:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90300000000000002,"5min":1.01,"15min":1.01},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91300000000000003,"5min":1.081,"15min":1.081},"last":0.70799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:13:57.869 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-17T01:13:57.869 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-17T01:13:57.869 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-17T01:13:57.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph health --format=json 2023-10-17T01:13:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:57 smithi181 ceph-mon[134520]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:13:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:13:58.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:58 smithi045 ceph-mon[98474]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:58.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:58 smithi060 ceph-mon[134136]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:58 smithi181 ceph-mon[134520]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:13:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:13:59 smithi045 ceph-mon[98474]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:13:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:13:59 smithi060 ceph-mon[134136]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:13:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:13:59 smithi181 ceph-mon[134520]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:00.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 -- 172.21.15.45:0/331397867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c100fd0 msgr2=0x7f240c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:00.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 --2- 172.21.15.45:0/331397867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c100fd0 0x7f240c1013b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009990 tx=0x7f23fc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 -- 172.21.15.45:0/331397867 shutdown_connections 2023-10-17T01:14:00.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 --2- 172.21.15.45:0/331397867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c1060b0 0x7f240c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 --2- 172.21.15.45:0/331397867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c100fd0 0x7f240c1013b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 --2- 172.21.15.45:0/331397867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c101980 0x7f240c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 -- 172.21.15.45:0/331397867 >> 172.21.15.45:0/331397867 conn(0x7f240c0fc840 msgr2=0x7f240c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 -- 172.21.15.45:0/331397867 shutdown_connections 2023-10-17T01:14:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.160+0000 7f2412937700 1 -- 172.21.15.45:0/331397867 wait complete. 2023-10-17T01:14:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.161+0000 7f2412937700 1 Processor -- start 2023-10-17T01:14:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 -- start start 2023-10-17T01:14:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c100fd0 0x7f240c196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c101980 0x7f240c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 0x7f240c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19be50 con 0x7f240c1060b0 2023-10-17T01:14:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19bfc0 con 0x7f240c100fd0 2023-10-17T01:14:00.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2412937700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19c180 con 0x7f240c101980 2023-10-17T01:14:00.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2410ed4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 0x7f240c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:00.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f240bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c100fd0 0x7f240c196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:00.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2410ed4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 0x7f240c19b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39166/0 (socket says 172.21.15.45:39166) 2023-10-17T01:14:00.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.162+0000 7f2410ed4700 1 -- 172.21.15.45:0/3201484215 learned_addr learned my addr 172.21.15.45:0/3201484215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:00.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f240b7fe700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c101980 0x7f240c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:00.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2410ed4700 1 -- 172.21.15.45:0/3201484215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c101980 msgr2=0x7f240c1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:00.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2410ed4700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c101980 0x7f240c1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2410ed4700 1 -- 172.21.15.45:0/3201484215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c100fd0 msgr2=0x7f240c196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:00.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2410ed4700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c100fd0 0x7f240c196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2410ed4700 1 -- 172.21.15.45:0/3201484215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc0095f0 con 0x7f240c1060b0 2023-10-17T01:14:00.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f240bfff700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c100fd0 0x7f240c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:00.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2410ed4700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 0x7f240c19b840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f240400ef30 tx=0x7f240400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:00.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404018070 con 0x7f240c1060b0 2023-10-17T01:14:00.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240400f040 con 0x7f240c1060b0 2023-10-17T01:14:00.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c19c400 con 0x7f240c1060b0 2023-10-17T01:14:00.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.163+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400cd00 con 0x7f240c1060b0 2023-10-17T01:14:00.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.164+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c1a3cc0 con 0x7f240c1060b0 2023-10-17T01:14:00.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.165+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c04ea50 con 0x7f240c1060b0 2023-10-17T01:14:00.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.167+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f2404003a00 con 0x7f240c1060b0 2023-10-17T01:14:00.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.168+0000 7f24097fa700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23f40a0bb0 0x7f23f40a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:00.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.168+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f24040cc770 con 0x7f240c1060b0 2023-10-17T01:14:00.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.168+0000 7f240bfff700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23f40a0bb0 0x7f23f40a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:00.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.169+0000 7f240bfff700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23f40a0bb0 0x7f23f40a3070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b2e0 tx=0x7f23fc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:00.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.170+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2404096000 con 0x7f240c1060b0 2023-10-17T01:14:00.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.429+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f240c1a3ef0 con 0x7f240c1060b0 2023-10-17T01:14:00.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.429+0000 7f24097fa700 1 -- 172.21.15.45:0/3201484215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2404099620 con 0x7f240c1060b0 2023-10-17T01:14:00.431 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:00.431 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-17T01:14:00.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.431+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23f40a0bb0 msgr2=0x7f23f40a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:00.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.431+0000 7f2412937700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23f40a0bb0 0x7f23f40a3070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b2e0 tx=0x7f23fc005fd0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.431+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 msgr2=0x7f240c19b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:00.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.431+0000 7f2412937700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 0x7f240c19b840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f240400ef30 tx=0x7f240400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 shutdown_connections 2023-10-17T01:14:00.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f23f40a0bb0 0x7f23f40a3070 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f240c1060b0 0x7f240c19b840 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f240c100fd0 0x7f240c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 --2- 172.21.15.45:0/3201484215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c101980 0x7f240c1973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:00.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 >> 172.21.15.45:0/3201484215 conn(0x7f240c0fc840 msgr2=0x7f240c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:00.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 shutdown_connections 2023-10-17T01:14:00.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:00.432+0000 7f2412937700 1 -- 172.21.15.45:0/3201484215 wait complete. 2023-10-17T01:14:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:00 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3201484215' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-17T01:14:00.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:00 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3201484215' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-17T01:14:00.893 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-17T01:14:00.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:00 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3201484215' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-17T01:14:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 -- 172.21.15.45:0/3180896713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c073750 msgr2=0x7f2b1c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3180896713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c073750 0x7f2b1c073bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009990 tx=0x7f2b0c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 -- 172.21.15.45:0/3180896713 shutdown_connections 2023-10-17T01:14:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3180896713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c108c30 0x7f2b1c10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3180896713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c074d30 0x7f2b1c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3180896713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c073750 0x7f2b1c073bd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 -- 172.21.15.45:0/3180896713 >> 172.21.15.45:0/3180896713 conn(0x7f2b1c0fc290 msgr2=0x7f2b1c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:01.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 -- 172.21.15.45:0/3180896713 shutdown_connections 2023-10-17T01:14:01.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.104+0000 7f2b23dff700 1 -- 172.21.15.45:0/3180896713 wait complete. 2023-10-17T01:14:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.105+0000 7f2b23dff700 1 Processor -- start 2023-10-17T01:14:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.105+0000 7f2b23dff700 1 -- start start 2023-10-17T01:14:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b23dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c073750 0x7f2b1c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b23dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c074d30 0x7f2b1c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b23dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c108c30 0x7f2b1c1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b23dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1a0750 con 0x7f2b1c108c30 2023-10-17T01:14:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b23dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1a0890 con 0x7f2b1c074d30 2023-10-17T01:14:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b23dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1a0a20 con 0x7f2b1c073750 2023-10-17T01:14:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b21b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c073750 0x7f2b1c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b21b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c073750 0x7f2b1c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56824/0 (socket says 172.21.15.45:56824) 2023-10-17T01:14:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b21b9b700 1 -- 172.21.15.45:0/3448433511 learned_addr learned my addr 172.21.15.45:0/3448433511 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2239c700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c108c30 0x7f2b1c1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2139a700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c074d30 0x7f2b1c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2239c700 1 -- 172.21.15.45:0/3448433511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c073750 msgr2=0x7f2b1c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2239c700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c073750 0x7f2b1c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2239c700 1 -- 172.21.15.45:0/3448433511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c074d30 msgr2=0x7f2b1c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2239c700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c074d30 0x7f2b1c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.106+0000 7f2b2239c700 1 -- 172.21.15.45:0/3448433511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c0095f0 con 0x7f2b1c108c30 2023-10-17T01:14:01.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.107+0000 7f2b2239c700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c108c30 0x7f2b1c1a0180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400efc0 tx=0x7f2b1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.107+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14018070 con 0x7f2b1c108c30 2023-10-17T01:14:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.107+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c1a0ca0 con 0x7f2b1c108c30 2023-10-17T01:14:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.107+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1400f040 con 0x7f2b1c108c30 2023-10-17T01:14:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.107+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400cd40 con 0x7f2b1c108c30 2023-10-17T01:14:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.107+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c1a11b0 con 0x7f2b1c108c30 2023-10-17T01:14:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.108+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c107330 con 0x7f2b1c108c30 2023-10-17T01:14:01.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.110+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f2b14003a00 con 0x7f2b1c108c30 2023-10-17T01:14:01.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.111+0000 7f2b12ffd700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2b080a0bf0 0x7f2b080a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.111+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2b140cbed0 con 0x7f2b1c108c30 2023-10-17T01:14:01.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.111+0000 7f2b21b9b700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2b080a0bf0 0x7f2b080a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.112+0000 7f2b21b9b700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2b080a0bf0 0x7f2b080a30b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009d20 tx=0x7f2b18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:01.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.115+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2b14095760 con 0x7f2b1c108c30 2023-10-17T01:14:01.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.329+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b1c02cc30 con 0x7f2b1c108c30 2023-10-17T01:14:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.331+0000 7f2b12ffd700 1 -- 172.21.15.45:0/3448433511 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f2b14098d80 con 0x7f2b1c108c30 2023-10-17T01:14:01.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:01.333 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:01.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2b080a0bf0 msgr2=0x7f2b080a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2b080a0bf0 0x7f2b080a30b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009d20 tx=0x7f2b18009260 comp rx=0 tx=0).stop 2023-10-17T01:14:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c108c30 msgr2=0x7f2b1c1a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c108c30 0x7f2b1c1a0180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400efc0 tx=0x7f2b1400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 shutdown_connections 2023-10-17T01:14:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2b080a0bf0 0x7f2b080a30b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1c108c30 0x7f2b1c1a0180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b1c073750 0x7f2b1c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 --2- 172.21.15.45:0/3448433511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c074d30 0x7f2b1c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 >> 172.21.15.45:0/3448433511 conn(0x7f2b1c0fc290 msgr2=0x7f2b1c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 shutdown_connections 2023-10-17T01:14:01.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.333+0000 7f2b23dff700 1 -- 172.21.15.45:0/3448433511 wait complete. 2023-10-17T01:14:01.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:01.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 -- 172.21.15.45:0/1289458367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40101f50 msgr2=0x7fde401023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 --2- 172.21.15.45:0/1289458367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40101f50 0x7fde401023b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde340099c0 tx=0x7fde34009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 -- 172.21.15.45:0/1289458367 shutdown_connections 2023-10-17T01:14:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 --2- 172.21.15.45:0/1289458367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde401028f0 0x7fde401091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 --2- 172.21.15.45:0/1289458367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40100d60 0x7fde40101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 --2- 172.21.15.45:0/1289458367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40101f50 0x7fde401023b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 -- 172.21.15.45:0/1289458367 >> 172.21.15.45:0/1289458367 conn(0x7fde400fc2d0 msgr2=0x7fde400fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 -- 172.21.15.45:0/1289458367 shutdown_connections 2023-10-17T01:14:01.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.562+0000 7fde455ff700 1 -- 172.21.15.45:0/1289458367 wait complete. 2023-10-17T01:14:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.563+0000 7fde455ff700 1 Processor -- start 2023-10-17T01:14:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.563+0000 7fde455ff700 1 -- start start 2023-10-17T01:14:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde455ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 0x7fde40074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde455ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40101f50 0x7fde40073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde455ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde401028f0 0x7fde40073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde455ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40073e20 con 0x7fde40100d60 2023-10-17T01:14:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde455ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40073f90 con 0x7fde401028f0 2023-10-17T01:14:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde455ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde401a0b50 con 0x7fde40101f50 2023-10-17T01:14:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 0x7fde40074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40101f50 0x7fde40073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 0x7fde40074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39192/0 (socket says 172.21.15.45:39192) 2023-10-17T01:14:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3effd700 1 -- 172.21.15.45:0/1213068742 learned_addr learned my addr 172.21.15.45:0/1213068742 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3f7fe700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde401028f0 0x7fde40073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3e7fc700 1 -- 172.21.15.45:0/1213068742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde401028f0 msgr2=0x7fde40073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3e7fc700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde401028f0 0x7fde40073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3e7fc700 1 -- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 msgr2=0x7fde40074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.564+0000 7fde3e7fc700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 0x7fde40074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde3e7fc700 1 -- 172.21.15.45:0/1213068742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340095f0 con 0x7fde40101f50 2023-10-17T01:14:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde3effd700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 0x7fde40074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:01.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde3e7fc700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40101f50 0x7fde40073140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fde340036f0 tx=0x7fde34009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3401d070 con 0x7fde40101f50 2023-10-17T01:14:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde401a0dd0 con 0x7fde40101f50 2023-10-17T01:14:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde34004010 con 0x7fde40101f50 2023-10-17T01:14:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400fe10 con 0x7fde40101f50 2023-10-17T01:14:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.565+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde401a11f0 con 0x7fde40101f50 2023-10-17T01:14:01.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.566+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde40108340 con 0x7fde40101f50 2023-10-17T01:14:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.568+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fde3400f560 con 0x7fde40101f50 2023-10-17T01:14:01.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.569+0000 7fde27fff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fde280a0bb0 0x7fde280a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.569+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fde340cd090 con 0x7fde40101f50 2023-10-17T01:14:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.570+0000 7fde3effd700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fde280a0bb0 0x7fde280a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.570+0000 7fde3effd700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fde280a0bb0 0x7fde280a3070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde300105c0 tx=0x7fde3000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:01.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.574+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fde34096920 con 0x7fde40101f50 2023-10-17T01:14:01.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.788+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde4002ce50 con 0x7fde40101f50 2023-10-17T01:14:01.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.788+0000 7fde27fff700 1 -- 172.21.15.45:0/1213068742 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde34099f40 con 0x7fde40101f50 2023-10-17T01:14:01.790 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fde280a0bb0 msgr2=0x7fde280a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fde280a0bb0 0x7fde280a3070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde300105c0 tx=0x7fde3000b4a0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40101f50 msgr2=0x7fde40073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40101f50 0x7fde40073140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fde340036f0 tx=0x7fde34009580 comp rx=0 tx=0).stop 2023-10-17T01:14:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 shutdown_connections 2023-10-17T01:14:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fde280a0bb0 0x7fde280a3070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde401028f0 0x7fde40073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde40100d60 0x7fde40074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 --2- 172.21.15.45:0/1213068742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde40101f50 0x7fde40073140 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 >> 172.21.15.45:0/1213068742 conn(0x7fde400fc2d0 msgr2=0x7fde40105480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 shutdown_connections 2023-10-17T01:14:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:01.791+0000 7fde455ff700 1 -- 172.21.15.45:0/1213068742 wait complete. 2023-10-17T01:14:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:01 smithi045 ceph-mon[98474]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:01 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3448433511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:01.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:01 smithi060 ceph-mon[134136]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:01 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3448433511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:01 smithi181 ceph-mon[134520]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:01 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3448433511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 -- 172.21.15.45:0/147582748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c0737b0 msgr2=0x7f686c073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 --2- 172.21.15.45:0/147582748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c0737b0 0x7f686c073c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6860009990 tx=0x7f6860009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 -- 172.21.15.45:0/147582748 shutdown_connections 2023-10-17T01:14:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 --2- 172.21.15.45:0/147582748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f686c108c00 0x7f686c10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 --2- 172.21.15.45:0/147582748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c074d90 0x7f686c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 --2- 172.21.15.45:0/147582748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c0737b0 0x7f686c073c30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 -- 172.21.15.45:0/147582748 >> 172.21.15.45:0/147582748 conn(0x7f686c0fc290 msgr2=0x7f686c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 -- 172.21.15.45:0/147582748 shutdown_connections 2023-10-17T01:14:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.018+0000 7f6871fff700 1 -- 172.21.15.45:0/147582748 wait complete. 2023-10-17T01:14:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.019+0000 7f6871fff700 1 Processor -- start 2023-10-17T01:14:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.019+0000 7f6871fff700 1 -- start start 2023-10-17T01:14:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f6871fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0737b0 0x7f686c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f6871fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f686c074d90 0x7f686c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f6871fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 0x7f686c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f6871fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1a0870 con 0x7f686c108c00 2023-10-17T01:14:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f6871fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1a09e0 con 0x7f686c0737b0 2023-10-17T01:14:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f6871fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1a0ba0 con 0x7f686c074d90 2023-10-17T01:14:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 0x7f686c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 0x7f686c1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39210/0 (socket says 172.21.15.45:39210) 2023-10-17T01:14:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 -- 172.21.15.45:0/2736954791 learned_addr learned my addr 172.21.15.45:0/2736954791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686affd700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f686c074d90 0x7f686c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686b7fe700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0737b0 0x7f686c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 -- 172.21.15.45:0/2736954791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f686c074d90 msgr2=0x7f686c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f686c074d90 0x7f686c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 -- 172.21.15.45:0/2736954791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0737b0 msgr2=0x7f686c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0737b0 0x7f686c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686bfff700 1 -- 172.21.15.45:0/2736954791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600095f0 con 0x7f686c108c00 2023-10-17T01:14:02.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.020+0000 7f686b7fe700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0737b0 0x7f686c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:02.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.021+0000 7f686bfff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 0x7f686c1a02a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f686400ef30 tx=0x7f686400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:02.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.021+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864018070 con 0x7f686c108c00 2023-10-17T01:14:02.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.021+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f686400f040 con 0x7f686c108c00 2023-10-17T01:14:02.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.021+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400cdb0 con 0x7f686c108c00 2023-10-17T01:14:02.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.021+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c1a0e20 con 0x7f686c108c00 2023-10-17T01:14:02.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.021+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c1a11f0 con 0x7f686c108c00 2023-10-17T01:14:02.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.022+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c1072b0 con 0x7f686c108c00 2023-10-17T01:14:02.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.024+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f6864003a00 con 0x7f686c108c00 2023-10-17T01:14:02.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.026+0000 7f6868ff9700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68540a0c30 0x7f68540a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.026+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68640cbde0 con 0x7f686c108c00 2023-10-17T01:14:02.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.026+0000 7f686b7fe700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68540a0c30 0x7f68540a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.026+0000 7f686b7fe700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68540a0c30 0x7f68540a30f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f686c19a200 tx=0x7f685c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:02.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.029+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f6864095670 con 0x7f686c108c00 2023-10-17T01:14:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.238+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f686c02ce70 con 0x7f686c108c00 2023-10-17T01:14:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.238+0000 7f6868ff9700 1 -- 172.21.15.45:0/2736954791 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6864098c90 con 0x7f686c108c00 2023-10-17T01:14:02.240 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68540a0c30 msgr2=0x7f68540a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68540a0c30 0x7f68540a30f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f686c19a200 tx=0x7f685c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 msgr2=0x7f686c1a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 0x7f686c1a02a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f686400ef30 tx=0x7f686400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 shutdown_connections 2023-10-17T01:14:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68540a0c30 0x7f68540a30f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f686c108c00 0x7f686c1a02a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0737b0 0x7f686c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 --2- 172.21.15.45:0/2736954791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f686c074d90 0x7f686c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 >> 172.21.15.45:0/2736954791 conn(0x7f686c0fc290 msgr2=0x7f686c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 shutdown_connections 2023-10-17T01:14:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.241+0000 7f6871fff700 1 -- 172.21.15.45:0/2736954791 wait complete. 2023-10-17T01:14:02.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 -- 172.21.15.45:0/1315259103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 msgr2=0x7f8644101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 --2- 172.21.15.45:0/1315259103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644101160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86340099c0 tx=0x7f8634009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 -- 172.21.15.45:0/1315259103 shutdown_connections 2023-10-17T01:14:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 --2- 172.21.15.45:0/1315259103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86441028d0 0x7f8644109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 --2- 172.21.15.45:0/1315259103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644101160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 --2- 172.21.15.45:0/1315259103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8644101f30 0x7f8644102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 -- 172.21.15.45:0/1315259103 >> 172.21.15.45:0/1315259103 conn(0x7f86440fc290 msgr2=0x7f86440fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 -- 172.21.15.45:0/1315259103 shutdown_connections 2023-10-17T01:14:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.468+0000 7f8648fff700 1 -- 172.21.15.45:0/1315259103 wait complete. 2023-10-17T01:14:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.469+0000 7f8648fff700 1 Processor -- start 2023-10-17T01:14:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 -- start start 2023-10-17T01:14:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 0x7f8644073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86441028d0 0x7f8644073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8644073e20 con 0x7f8644100d40 2023-10-17T01:14:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8644073f90 con 0x7f86441028d0 2023-10-17T01:14:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8648fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86441a0ac0 con 0x7f8644101f30 2023-10-17T01:14:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f864259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8641d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 0x7f8644073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8641d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 0x7f8644073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56850/0 (socket says 172.21.15.45:56850) 2023-10-17T01:14:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8641d9b700 1 -- 172.21.15.45:0/3776636600 learned_addr learned my addr 172.21.15.45:0/3776636600 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.470+0000 7f8642d9d700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86441028d0 0x7f8644073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8641d9b700 1 -- 172.21.15.45:0/3776636600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86441028d0 msgr2=0x7f8644073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8641d9b700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86441028d0 0x7f8644073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8641d9b700 1 -- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 msgr2=0x7f8644074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8641d9b700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8641d9b700 1 -- 172.21.15.45:0/3776636600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340095f0 con 0x7f8644101f30 2023-10-17T01:14:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f864259c700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8641d9b700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 0x7f8644073140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f863800e9a0 tx=0x7f863800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638009790 con 0x7f8644101f30 2023-10-17T01:14:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86441a0da0 con 0x7f8644101f30 2023-10-17T01:14:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.471+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8638010430 con 0x7f8644101f30 2023-10-17T01:14:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.472+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638018680 con 0x7f8644101f30 2023-10-17T01:14:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.472+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86441a1270 con 0x7f8644101f30 2023-10-17T01:14:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.473+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864410c5a0 con 0x7f8644101f30 2023-10-17T01:14:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.475+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f863800ca10 con 0x7f8644101f30 2023-10-17T01:14:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.476+0000 7f86337fe700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f862c0a0bb0 0x7f862c0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.476+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8638014070 con 0x7f8644101f30 2023-10-17T01:14:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.476+0000 7f864259c700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f862c0a0bb0 0x7f862c0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.477+0000 7f864259c700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f862c0a0bb0 0x7f862c0a3070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8634009990 tx=0x7f8634005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.479+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f86380520b0 con 0x7f8644101f30 2023-10-17T01:14:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.696+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8644107c70 con 0x7f8644101f30 2023-10-17T01:14:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.697+0000 7f86337fe700 1 -- 172.21.15.45:0/3776636600 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86380996d0 con 0x7f8644101f30 2023-10-17T01:14:02.698 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f862c0a0bb0 msgr2=0x7f862c0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f862c0a0bb0 0x7f862c0a3070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8634009990 tx=0x7f8634005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 msgr2=0x7f8644073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 0x7f8644073140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f863800e9a0 tx=0x7f863800ecb0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 shutdown_connections 2023-10-17T01:14:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f862c0a0bb0 0x7f862c0a3070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86441028d0 0x7f8644073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8644100d40 0x7f8644074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 --2- 172.21.15.45:0/3776636600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8644101f30 0x7f8644073140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 >> 172.21.15.45:0/3776636600 conn(0x7f86440fc290 msgr2=0x7f86440fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 shutdown_connections 2023-10-17T01:14:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.699+0000 7f8648fff700 1 -- 172.21.15.45:0/3776636600 wait complete. 2023-10-17T01:14:02.719 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:02.781 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:02 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1213068742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:02.781 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:02 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2736954791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:02 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1213068742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:02.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:02 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2736954791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:02 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1213068742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:02.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:02 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2736954791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:02.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 -- 172.21.15.45:0/1945785209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 msgr2=0x7fc758101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/1945785209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758101180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7480099c0 tx=0x7fc748009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 -- 172.21.15.45:0/1945785209 shutdown_connections 2023-10-17T01:14:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/1945785209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7581028f0 0x7fc7581091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/1945785209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758101180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/1945785209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc758101f50 0x7fc7581023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 -- 172.21.15.45:0/1945785209 >> 172.21.15.45:0/1945785209 conn(0x7fc7580fc290 msgr2=0x7fc7580fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 -- 172.21.15.45:0/1945785209 shutdown_connections 2023-10-17T01:14:02.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.924+0000 7fc75d7ff700 1 -- 172.21.15.45:0/1945785209 wait complete. 2023-10-17T01:14:02.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.925+0000 7fc75d7ff700 1 Processor -- start 2023-10-17T01:14:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.925+0000 7fc75d7ff700 1 -- start start 2023-10-17T01:14:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc75d7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc75d7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc758101f50 0x7fc758196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc75d7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7581028f0 0x7fc75819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc75d7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819e5a0 con 0x7fc758100d60 2023-10-17T01:14:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc75d7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819e6e0 con 0x7fc7581028f0 2023-10-17T01:14:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc75d7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819e870 con 0x7fc758101f50 2023-10-17T01:14:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39240/0 (socket says 172.21.15.45:39240) 2023-10-17T01:14:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 -- 172.21.15.45:0/3177283261 learned_addr learned my addr 172.21.15.45:0/3177283261 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc7567fc700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc758101f50 0x7fc758196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc7577fe700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7581028f0 0x7fc75819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 -- 172.21.15.45:0/3177283261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc758101f50 msgr2=0x7fc758196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc758101f50 0x7fc758196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 -- 172.21.15.45:0/3177283261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7581028f0 msgr2=0x7fc75819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7581028f0 0x7fc75819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.926+0000 7fc756ffd700 1 -- 172.21.15.45:0/3177283261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480095f0 con 0x7fc758100d60 2023-10-17T01:14:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.927+0000 7fc756ffd700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758196a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc748009990 tx=0x7fc74800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.927+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74801d070 con 0x7fc758100d60 2023-10-17T01:14:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.927+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7480054f0 con 0x7fc758100d60 2023-10-17T01:14:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.927+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75819eaf0 con 0x7fc758100d60 2023-10-17T01:14:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.927+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748004b00 con 0x7fc758100d60 2023-10-17T01:14:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.927+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75819ef70 con 0x7fc758100d60 2023-10-17T01:14:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.928+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75810c5c0 con 0x7fc758100d60 2023-10-17T01:14:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.930+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fc74800fb80 con 0x7fc758100d60 2023-10-17T01:14:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.931+0000 7fc73ffff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7400a0c70 0x7fc7400a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.931+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc7480cd0c0 con 0x7fc758100d60 2023-10-17T01:14:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.931+0000 7fc7567fc700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7400a0c70 0x7fc7400a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.932+0000 7fc7567fc700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7400a0c70 0x7fc7400a3130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009c60 tx=0x7fc74c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:02.935+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fc748096950 con 0x7fc758100d60 2023-10-17T01:14:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.197+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc758107c10 con 0x7fc758100d60 2023-10-17T01:14:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.198+0000 7fc73ffff700 1 -- 172.21.15.45:0/3177283261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7fc748099f70 con 0x7fc758100d60 2023-10-17T01:14:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:03.200 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:03.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7400a0c70 msgr2=0x7fc7400a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7400a0c70 0x7fc7400a3130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009c60 tx=0x7fc74c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:03.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 msgr2=0x7fc758196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758196a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc748009990 tx=0x7fc74800f690 comp rx=0 tx=0).stop 2023-10-17T01:14:03.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 shutdown_connections 2023-10-17T01:14:03.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7400a0c70 0x7fc7400a3130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7581028f0 0x7fc75819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc758100d60 0x7fc758196a20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 --2- 172.21.15.45:0/3177283261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc758101f50 0x7fc758196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 >> 172.21.15.45:0/3177283261 conn(0x7fc7580fc290 msgr2=0x7fc7580fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 shutdown_connections 2023-10-17T01:14:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.200+0000 7fc75d7ff700 1 -- 172.21.15.45:0/3177283261 wait complete. 2023-10-17T01:14:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:03.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:03.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 -- 172.21.15.45:0/3218632097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90073750 msgr2=0x7f9e90073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 --2- 172.21.15.45:0/3218632097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90073750 0x7f9e90073bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009990 tx=0x7f9e84009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 -- 172.21.15.45:0/3218632097 shutdown_connections 2023-10-17T01:14:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 --2- 172.21.15.45:0/3218632097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90108c40 0x7f9e9010b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 --2- 172.21.15.45:0/3218632097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e90074d30 0x7f9e90073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 --2- 172.21.15.45:0/3218632097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90073750 0x7f9e90073bd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 -- 172.21.15.45:0/3218632097 >> 172.21.15.45:0/3218632097 conn(0x7f9e900fc2c0 msgr2=0x7f9e900fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 -- 172.21.15.45:0/3218632097 shutdown_connections 2023-10-17T01:14:03.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.426+0000 7f9e957ff700 1 -- 172.21.15.45:0/3218632097 wait complete. 2023-10-17T01:14:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.427+0000 7f9e957ff700 1 Processor -- start 2023-10-17T01:14:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.427+0000 7f9e957ff700 1 -- start start 2023-10-17T01:14:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.427+0000 7f9e957ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e90073750 0x7f9e90198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e957ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 0x7f9e90199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e957ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90108c40 0x7f9e901a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e957ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e901a0770 con 0x7f9e90108c40 2023-10-17T01:14:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e957ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e901a08b0 con 0x7f9e90073750 2023-10-17T01:14:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e957ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e901a0a40 con 0x7f9e90074d30 2023-10-17T01:14:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 0x7f9e90199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90108c40 0x7f9e901a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 0x7f9e90199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56892/0 (socket says 172.21.15.45:56892) 2023-10-17T01:14:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 -- 172.21.15.45:0/1269754016 learned_addr learned my addr 172.21.15.45:0/1269754016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8effd700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e90073750 0x7f9e90198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 -- 172.21.15.45:0/1269754016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e90073750 msgr2=0x7f9e90198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e90073750 0x7f9e90198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 -- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90108c40 msgr2=0x7f9e901a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90108c40 0x7f9e901a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 -- 172.21.15.45:0/1269754016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80009520 con 0x7f9e90074d30 2023-10-17T01:14:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8f7fe700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90108c40 0x7f9e901a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.428+0000 7f9e8e7fc700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 0x7f9e90199130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b680 tx=0x7f9e84005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.429+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8401d070 con 0x7f9e90074d30 2023-10-17T01:14:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.429+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e88009c40 con 0x7f9e90074d30 2023-10-17T01:14:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.429+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e84022470 con 0x7f9e90074d30 2023-10-17T01:14:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.429+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84017650 con 0x7f9e90074d30 2023-10-17T01:14:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.429+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e840095f0 con 0x7f9e90074d30 2023-10-17T01:14:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.430+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e90107340 con 0x7f9e90074d30 2023-10-17T01:14:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.432+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f9e840225e0 con 0x7f9e90074d30 2023-10-17T01:14:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.433+0000 7f9e77fff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e780a0bf0 0x7f9e780a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.434+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9e840cdf70 con 0x7f9e90074d30 2023-10-17T01:14:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.434+0000 7f9e8effd700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e780a0bf0 0x7f9e780a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.434+0000 7f9e8effd700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e780a0bf0 0x7f9e780a30b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009df0 tx=0x7f9e80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:03.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.436+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f9e84097800 con 0x7f9e90074d30 2023-10-17T01:14:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.690+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e9002ce00 con 0x7f9e90074d30 2023-10-17T01:14:03.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.691+0000 7f9e77fff700 1 -- 172.21.15.45:0/1269754016 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f9e8409ae20 con 0x7f9e90074d30 2023-10-17T01:14:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:03.693 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e780a0bf0 msgr2=0x7f9e780a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e780a0bf0 0x7f9e780a30b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009df0 tx=0x7f9e80009260 comp rx=0 tx=0).stop 2023-10-17T01:14:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 msgr2=0x7f9e90199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 0x7f9e90199130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b680 tx=0x7f9e84005740 comp rx=0 tx=0).stop 2023-10-17T01:14:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 shutdown_connections 2023-10-17T01:14:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e780a0bf0 0x7f9e780a30b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e90108c40 0x7f9e901a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e90073750 0x7f9e90198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 --2- 172.21.15.45:0/1269754016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e90074d30 0x7f9e90199130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 >> 172.21.15.45:0/1269754016 conn(0x7f9e900fc2c0 msgr2=0x7f9e900fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 shutdown_connections 2023-10-17T01:14:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.694+0000 7f9e957ff700 1 -- 172.21.15.45:0/1269754016 wait complete. 2023-10-17T01:14:03.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:03.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:03.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:03 smithi045 ceph-mon[98474]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:03.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:03 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3776636600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:03.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:03 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3177283261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:03 smithi060 ceph-mon[134136]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:03 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3776636600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:03.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:03 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3177283261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:03 smithi181 ceph-mon[134520]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:03 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3776636600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:03 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3177283261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 -- 172.21.15.45:0/424388476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0101800 msgr2=0x7f8fd0103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/424388476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0101800 0x7f8fd0103bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0098f0 tx=0x7f8fcc009c00 comp rx=0 tx=0).stop 2023-10-17T01:14:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 -- 172.21.15.45:0/424388476 shutdown_connections 2023-10-17T01:14:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/424388476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0106a60 0x7f8fd0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/424388476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0101800 0x7f8fd0103bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/424388476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0104130 0x7f8fd0106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 -- 172.21.15.45:0/424388476 >> 172.21.15.45:0/424388476 conn(0x7f8fd00fb1b0 msgr2=0x7f8fd00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 -- 172.21.15.45:0/424388476 shutdown_connections 2023-10-17T01:14:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.930+0000 7f8fd85ff700 1 -- 172.21.15.45:0/424388476 wait complete. 2023-10-17T01:14:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.931+0000 7f8fd85ff700 1 Processor -- start 2023-10-17T01:14:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 -- start start 2023-10-17T01:14:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101800 0x7f8fd0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0104130 0x7f8fd0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 0x7f8fd01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01a0850 con 0x7f8fd0101800 2023-10-17T01:14:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01a09c0 con 0x7f8fd0104130 2023-10-17T01:14:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd85ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01a0b80 con 0x7f8fd0106a60 2023-10-17T01:14:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd639b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101800 0x7f8fd0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd6b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 0x7f8fd01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd6b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 0x7f8fd01a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56912/0 (socket says 172.21.15.45:56912) 2023-10-17T01:14:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd6b9c700 1 -- 172.21.15.45:0/47473549 learned_addr learned my addr 172.21.15.45:0/47473549 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd639b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101800 0x7f8fd0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39280/0 (socket says 172.21.15.45:39280) 2023-10-17T01:14:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd5b9a700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0104130 0x7f8fd0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd6b9c700 1 -- 172.21.15.45:0/47473549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0104130 msgr2=0x7f8fd0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.932+0000 7f8fd6b9c700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0104130 0x7f8fd0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fd6b9c700 1 -- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101800 msgr2=0x7f8fd0198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fd6b9c700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101800 0x7f8fd0198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fd6b9c700 1 -- 172.21.15.45:0/47473549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc009520 con 0x7f8fd0106a60 2023-10-17T01:14:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fd6b9c700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 0x7f8fd01a0280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800ef30 tx=0x7f8fc800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8018070 con 0x7f8fd0106a60 2023-10-17T01:14:03.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc800f040 con 0x7f8fd0106a60 2023-10-17T01:14:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800cdb0 con 0x7f8fd0106a60 2023-10-17T01:14:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd01a0e00 con 0x7f8fd0106a60 2023-10-17T01:14:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.933+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01a11d0 con 0x7f8fd0106a60 2023-10-17T01:14:03.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.934+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd0108620 con 0x7f8fd0106a60 2023-10-17T01:14:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.936+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f8fc8003a00 con 0x7f8fd0106a60 2023-10-17T01:14:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.937+0000 7f8fc77fe700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0bb0 0x7f8fbc0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.938+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8fc80cbf30 con 0x7f8fd0106a60 2023-10-17T01:14:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.938+0000 7f8fd639b700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0bb0 0x7f8fbc0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.938+0000 7f8fd639b700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0bb0 0x7f8fbc0a3070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc005e90 tx=0x7f8fcc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:03.942+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f8fc80957c0 con 0x7f8fd0106a60 2023-10-17T01:14:04.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.158+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fd002ce80 con 0x7f8fd0106a60 2023-10-17T01:14:04.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.159+0000 7f8fc77fe700 1 -- 172.21.15.45:0/47473549 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fc8098de0 con 0x7f8fd0106a60 2023-10-17T01:14:04.161 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:04.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0bb0 msgr2=0x7f8fbc0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0bb0 0x7f8fbc0a3070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc005e90 tx=0x7f8fcc019040 comp rx=0 tx=0).stop 2023-10-17T01:14:04.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 msgr2=0x7f8fd01a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 0x7f8fd01a0280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800ef30 tx=0x7f8fc800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 shutdown_connections 2023-10-17T01:14:04.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0bb0 0x7f8fbc0a3070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0106a60 0x7f8fd01a0280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101800 0x7f8fd0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 --2- 172.21.15.45:0/47473549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0104130 0x7f8fd0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 >> 172.21.15.45:0/47473549 conn(0x7f8fd00fb1b0 msgr2=0x7f8fd0102470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 shutdown_connections 2023-10-17T01:14:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.162+0000 7f8fd85ff700 1 -- 172.21.15.45:0/47473549 wait complete. 2023-10-17T01:14:04.183 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 -- 172.21.15.45:0/263181196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 msgr2=0x7fd298102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/263181196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298102390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009920 tx=0x7fd28c009c30 comp rx=0 tx=0).stop 2023-10-17T01:14:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 -- 172.21.15.45:0/263181196 shutdown_connections 2023-10-17T01:14:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/263181196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2981028d0 0x7fd298109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/263181196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd298100d40 0x7fd298101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/263181196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298102390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.415+0000 7fd29c9ff700 1 -- 172.21.15.45:0/263181196 >> 172.21.15.45:0/263181196 conn(0x7fd2980fc290 msgr2=0x7fd2980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.416+0000 7fd29c9ff700 1 -- 172.21.15.45:0/263181196 shutdown_connections 2023-10-17T01:14:04.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.416+0000 7fd29c9ff700 1 -- 172.21.15.45:0/263181196 wait complete. 2023-10-17T01:14:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.416+0000 7fd29c9ff700 1 Processor -- start 2023-10-17T01:14:04.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 -- start start 2023-10-17T01:14:04.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd298100d40 0x7fd298074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2981028d0 0x7fd298073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29819aa50 con 0x7fd298100d40 2023-10-17T01:14:04.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298073bc0 con 0x7fd298101f30 2023-10-17T01:14:04.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29c9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298073d80 con 0x7fd2981028d0 2023-10-17T01:14:04.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd295d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd295d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46814/0 (socket says 172.21.15.45:46814) 2023-10-17T01:14:04.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd29659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd298100d40 0x7fd298074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd295d9b700 1 -- 172.21.15.45:0/3481559721 learned_addr learned my addr 172.21.15.45:0/3481559721 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.417+0000 7fd296d9d700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2981028d0 0x7fd298073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd295d9b700 1 -- 172.21.15.45:0/3481559721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2981028d0 msgr2=0x7fd298073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd295d9b700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2981028d0 0x7fd298073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd295d9b700 1 -- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd298100d40 msgr2=0x7fd298074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd295d9b700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd298100d40 0x7fd298074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd295d9b700 1 -- 172.21.15.45:0/3481559721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009520 con 0x7fd298101f30 2023-10-17T01:14:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd295d9b700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298073140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009230 tx=0x7fd28c00bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c01c070 con 0x7fd298101f30 2023-10-17T01:14:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd28c005040 con 0x7fd298101f30 2023-10-17T01:14:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2981a5400 con 0x7fd298101f30 2023-10-17T01:14:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0039b0 con 0x7fd298101f30 2023-10-17T01:14:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.418+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981a5770 con 0x7fd298101f30 2023-10-17T01:14:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.419+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2981084c0 con 0x7fd298101f30 2023-10-17T01:14:04.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.421+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fd28c020030 con 0x7fd298101f30 2023-10-17T01:14:04.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.422+0000 7fd2877fe700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2800a0bb0 0x7fd2800a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.422+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd28c0cdaf0 con 0x7fd298101f30 2023-10-17T01:14:04.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.423+0000 7fd29659c700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2800a0bb0 0x7fd2800a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.423+0000 7fd29659c700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2800a0bb0 0x7fd2800a3070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd298101d90 tx=0x7fd288009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:04.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.425+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd28c0974a0 con 0x7fd298101f30 2023-10-17T01:14:04.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.652+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd29802ce80 con 0x7fd298101f30 2023-10-17T01:14:04.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.653+0000 7fd2877fe700 1 -- 172.21.15.45:0/3481559721 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd28c0051b0 con 0x7fd298101f30 2023-10-17T01:14:04.654 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.655+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2800a0bb0 msgr2=0x7fd2800a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.655+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2800a0bb0 0x7fd2800a3070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd298101d90 tx=0x7fd288009310 comp rx=0 tx=0).stop 2023-10-17T01:14:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.655+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 msgr2=0x7fd298073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.655+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298073140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009230 tx=0x7fd28c00bf80 comp rx=0 tx=0).stop 2023-10-17T01:14:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 shutdown_connections 2023-10-17T01:14:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2800a0bb0 0x7fd2800a3070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2981028d0 0x7fd298073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd298100d40 0x7fd298074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 --2- 172.21.15.45:0/3481559721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298101f30 0x7fd298073140 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 >> 172.21.15.45:0/3481559721 conn(0x7fd2980fc290 msgr2=0x7fd298105600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 shutdown_connections 2023-10-17T01:14:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.656+0000 7fd29c9ff700 1 -- 172.21.15.45:0/3481559721 wait complete. 2023-10-17T01:14:04.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:04.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:04 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:14:04.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:04 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1269754016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:04.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:04 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/47473549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:04.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:04 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:14:04.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:04 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1269754016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:04.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:04 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/47473549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:04.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 -- 172.21.15.45:0/2514341335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340fff10 msgr2=0x7f0834100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 --2- 172.21.15.45:0/2514341335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340fff10 0x7f0834100390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0828009990 tx=0x7f0828009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 -- 172.21.15.45:0/2514341335 shutdown_connections 2023-10-17T01:14:04.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 --2- 172.21.15.45:0/2514341335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08341008d0 0x7f0834100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 --2- 172.21.15.45:0/2514341335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08340ff5b0 0x7f08340ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 --2- 172.21.15.45:0/2514341335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340fff10 0x7f0834100390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 -- 172.21.15.45:0/2514341335 >> 172.21.15.45:0/2514341335 conn(0x7f08340fb1b0 msgr2=0x7f08340fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 -- 172.21.15.45:0/2514341335 shutdown_connections 2023-10-17T01:14:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.898+0000 7f0839fff700 1 -- 172.21.15.45:0/2514341335 wait complete. 2023-10-17T01:14:04.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.900+0000 7f0839fff700 1 Processor -- start 2023-10-17T01:14:04.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.900+0000 7f0839fff700 1 -- start start 2023-10-17T01:14:04.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0839fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 0x7f0834198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0839fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340fff10 0x7f08341991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0839fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08341008d0 0x7f08341a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0839fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08341a07c0 con 0x7f08340fff10 2023-10-17T01:14:04.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0839fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08341a0930 con 0x7f08341008d0 2023-10-17T01:14:04.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0839fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08341a0af0 con 0x7f08340ff5b0 2023-10-17T01:14:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 0x7f0834198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 0x7f0834198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56954/0 (socket says 172.21.15.45:56954) 2023-10-17T01:14:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 -- 172.21.15.45:0/492431819 learned_addr learned my addr 172.21.15.45:0/492431819 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0832ffd700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340fff10 0x7f08341991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f0833fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08341008d0 0x7f08341a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 -- 172.21.15.45:0/492431819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08341008d0 msgr2=0x7f08341a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08341008d0 0x7f08341a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 -- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340fff10 msgr2=0x7f08341991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340fff10 0x7f08341991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 -- 172.21.15.45:0/492431819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280095f0 con 0x7f08340ff5b0 2023-10-17T01:14:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.901+0000 7f08337fe700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 0x7f0834198c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f082400e9a0 tx=0x7f082400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.902+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824009790 con 0x7f08340ff5b0 2023-10-17T01:14:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.902+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0824010430 con 0x7f08340ff5b0 2023-10-17T01:14:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.902+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824018680 con 0x7f08340ff5b0 2023-10-17T01:14:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.902+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08341a0dd0 con 0x7f08340ff5b0 2023-10-17T01:14:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.902+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08341a1220 con 0x7f08340ff5b0 2023-10-17T01:14:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.903+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083410a390 con 0x7f08340ff5b0 2023-10-17T01:14:04.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:04 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:14:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:04 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1269754016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:04.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:04 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/47473549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.905+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f08240105a0 con 0x7f08340ff5b0 2023-10-17T01:14:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.906+0000 7f0830ff9700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f081c0a0c30 0x7f081c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.906+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0824014070 con 0x7f08340ff5b0 2023-10-17T01:14:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.907+0000 7f0832ffd700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f081c0a0c30 0x7f081c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.907+0000 7f0832ffd700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f081c0a0c30 0x7f081c0a30f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f082800b2e0 tx=0x7f0828005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:04.908+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0824096080 con 0x7f08340ff5b0 2023-10-17T01:14:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.131+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f083402ce80 con 0x7f08340ff5b0 2023-10-17T01:14:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.132+0000 7f0830ff9700 1 -- 172.21.15.45:0/492431819 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08240996a0 con 0x7f08340ff5b0 2023-10-17T01:14:05.133 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.134+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f081c0a0c30 msgr2=0x7f081c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.134+0000 7f0839fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f081c0a0c30 0x7f081c0a30f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f082800b2e0 tx=0x7f0828005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.134+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 msgr2=0x7f0834198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.134+0000 7f0839fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 0x7f0834198c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f082400e9a0 tx=0x7f082400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.134+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 shutdown_connections 2023-10-17T01:14:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f081c0a0c30 0x7f081c0a30f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08341008d0 0x7f08341a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08340ff5b0 0x7f0834198c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 --2- 172.21.15.45:0/492431819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340fff10 0x7f08341991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 >> 172.21.15.45:0/492431819 conn(0x7f08340fb1b0 msgr2=0x7f08340fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 shutdown_connections 2023-10-17T01:14:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.135+0000 7f0839fff700 1 -- 172.21.15.45:0/492431819 wait complete. 2023-10-17T01:14:05.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/5260096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb8101f40 msgr2=0x7f3cb81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/5260096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb8101f40 0x7f3cb81023a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0099c0 tx=0x7f3cac009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/5260096 shutdown_connections 2023-10-17T01:14:05.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/5260096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cb81028e0 0x7f3cb8109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/5260096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/5260096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb8101f40 0x7f3cb81023a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/5260096 >> 172.21.15.45:0/5260096 conn(0x7f3cb80fc280 msgr2=0x7f3cb80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:05.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/5260096 shutdown_connections 2023-10-17T01:14:05.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.374+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/5260096 wait complete. 2023-10-17T01:14:05.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.375+0000 7f3cbd1ff700 1 Processor -- start 2023-10-17T01:14:05.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 -- start start 2023-10-17T01:14:05.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cb8101f40 0x7f3cb8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb81028e0 0x7f3cb819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb819e610 con 0x7f3cb81028e0 2023-10-17T01:14:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb819e750 con 0x7f3cb8100d50 2023-10-17T01:14:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cbd1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb819e8e0 con 0x7f3cb8101f40 2023-10-17T01:14:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cb6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cb759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb81028e0 0x7f3cb819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cb6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46846/0 (socket says 172.21.15.45:46846) 2023-10-17T01:14:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cb6d9d700 1 -- 172.21.15.45:0/4245153384 learned_addr learned my addr 172.21.15.45:0/4245153384 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.376+0000 7f3cb659c700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cb8101f40 0x7f3cb8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cb6d9d700 1 -- 172.21.15.45:0/4245153384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cb8101f40 msgr2=0x7f3cb8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cb6d9d700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cb8101f40 0x7f3cb8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cb6d9d700 1 -- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb81028e0 msgr2=0x7f3cb819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cb6d9d700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb81028e0 0x7f3cb819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cb6d9d700 1 -- 172.21.15.45:0/4245153384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0095f0 con 0x7f3cb8100d50 2023-10-17T01:14:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cb6d9d700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8196a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800d980 tx=0x7f3ca800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8004020 con 0x7f3cb8100d50 2023-10-17T01:14:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca800f690 con 0x7f3cb8100d50 2023-10-17T01:14:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8003a60 con 0x7f3cb8100d50 2023-10-17T01:14:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb819ebc0 con 0x7f3cb8100d50 2023-10-17T01:14:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.377+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb819f080 con 0x7f3cb8100d50 2023-10-17T01:14:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.378+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb81082f0 con 0x7f3cb8100d50 2023-10-17T01:14:05.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.380+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f3ca800fb00 con 0x7f3cb8100d50 2023-10-17T01:14:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.382+0000 7f3c9ffff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ca00a0c70 0x7f3ca00a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.382+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3ca80cc1e0 con 0x7f3cb8100d50 2023-10-17T01:14:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.382+0000 7f3cb659c700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ca00a0c70 0x7f3ca00a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.383+0000 7f3cb659c700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ca00a0c70 0x7f3ca00a3130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3cac006010 tx=0x7f3cac005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:05.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.384+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3ca8095a60 con 0x7f3cb8100d50 2023-10-17T01:14:05.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.666+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cb802cc30 con 0x7f3cb8100d50 2023-10-17T01:14:05.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.667+0000 7f3c9ffff700 1 -- 172.21.15.45:0/4245153384 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f3ca8099080 con 0x7f3cb8100d50 2023-10-17T01:14:05.668 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:05.669 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ca00a0c70 msgr2=0x7f3ca00a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ca00a0c70 0x7f3ca00a3130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3cac006010 tx=0x7f3cac005bc0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 msgr2=0x7f3cb8196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8196a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800d980 tx=0x7f3ca800dc90 comp rx=0 tx=0).stop 2023-10-17T01:14:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 shutdown_connections 2023-10-17T01:14:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3ca00a0c70 0x7f3ca00a3130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb81028e0 0x7f3cb819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cb8100d50 0x7f3cb8196a90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 --2- 172.21.15.45:0/4245153384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cb8101f40 0x7f3cb8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.669+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 >> 172.21.15.45:0/4245153384 conn(0x7f3cb80fc280 msgr2=0x7f3cb8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.670+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 shutdown_connections 2023-10-17T01:14:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.670+0000 7f3cbd1ff700 1 -- 172.21.15.45:0/4245153384 wait complete. 2023-10-17T01:14:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:05.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:05 smithi045 ceph-mon[98474]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:05 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3481559721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:05 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/492431819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:05 smithi060 ceph-mon[134136]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:05 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3481559721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:05.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:05 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/492431819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 -- 172.21.15.45:0/847716922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f0104aa0 msgr2=0x7fd3f010b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/847716922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f0104aa0 0x7fd3f010b3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800b0a0 tx=0x7fd3e800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 -- 172.21.15.45:0/847716922 shutdown_connections 2023-10-17T01:14:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/847716922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f0104aa0 0x7fd3f010b3c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/847716922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0102f10 0x7fd3f0103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/847716922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f01040e0 0x7fd3f0104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 -- 172.21.15.45:0/847716922 >> 172.21.15.45:0/847716922 conn(0x7fd3f00fe4d0 msgr2=0x7fd3f0100910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 -- 172.21.15.45:0/847716922 shutdown_connections 2023-10-17T01:14:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.898+0000 7fd3f61ff700 1 -- 172.21.15.45:0/847716922 wait complete. 2023-10-17T01:14:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 Processor -- start 2023-10-17T01:14:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 -- start start 2023-10-17T01:14:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 0x7fd3f0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f01040e0 0x7fd3f01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0104aa0 0x7fd3f01a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01a0930 con 0x7fd3f01040e0 2023-10-17T01:14:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01a0a70 con 0x7fd3f0104aa0 2023-10-17T01:14:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.900+0000 7fd3f61ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f01a0c00 con 0x7fd3f0102f10 2023-10-17T01:14:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 0x7fd3f0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3eeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f01040e0 0x7fd3f01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 0x7fd3f0198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57000/0 (socket says 172.21.15.45:57000) 2023-10-17T01:14:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 -- 172.21.15.45:0/3005440616 learned_addr learned my addr 172.21.15.45:0/3005440616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3effff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0104aa0 0x7fd3f01a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 -- 172.21.15.45:0/3005440616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0104aa0 msgr2=0x7fd3f01a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0104aa0 0x7fd3f01a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 -- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f01040e0 msgr2=0x7fd3f01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f01040e0 0x7fd3f01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 -- 172.21.15.45:0/3005440616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8009d00 con 0x7fd3f0102f10 2023-10-17T01:14:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3eeffd700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f01040e0 0x7fd3f01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3effff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0104aa0 0x7fd3f01a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:05.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.901+0000 7fd3ef7fe700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 0x7fd3f0198db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e970 tx=0x7fd3e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:05.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.902+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0009790 con 0x7fd3f0102f10 2023-10-17T01:14:05.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.902+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e000c7e0 con 0x7fd3f0102f10 2023-10-17T01:14:05.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.902+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f01a0ee0 con 0x7fd3f0102f10 2023-10-17T01:14:05.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.902+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0003c70 con 0x7fd3f0102f10 2023-10-17T01:14:05.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.902+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f01a1300 con 0x7fd3f0102f10 2023-10-17T01:14:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:05 smithi181 ceph-mon[134520]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:05.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:05 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3481559721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:05.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:05 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/492431819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:05.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.903+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f010e840 con 0x7fd3f0102f10 2023-10-17T01:14:05.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.905+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fd3e000cc50 con 0x7fd3f0102f10 2023-10-17T01:14:05.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.906+0000 7fd3ecff9700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3d80a0bf0 0x7fd3d80a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:05.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.906+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd3e0014070 con 0x7fd3f0102f10 2023-10-17T01:14:05.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.906+0000 7fd3eeffd700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3d80a0bf0 0x7fd3d80a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:05.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.907+0000 7fd3eeffd700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3d80a0bf0 0x7fd3d80a30b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009fd0 tx=0x7fd3e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:05.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:05.910+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd3e0095d00 con 0x7fd3f0102f10 2023-10-17T01:14:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.137+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f010ea00 con 0x7fd3f0102f10 2023-10-17T01:14:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.139+0000 7fd3ecff9700 1 -- 172.21.15.45:0/3005440616 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7fd3e0003dd0 con 0x7fd3f0102f10 2023-10-17T01:14:06.141 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:06.141 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3d80a0bf0 msgr2=0x7fd3d80a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3d80a0bf0 0x7fd3d80a30b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009fd0 tx=0x7fd3e4009260 comp rx=0 tx=0).stop 2023-10-17T01:14:06.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 msgr2=0x7fd3f0198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 0x7fd3f0198db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e970 tx=0x7fd3e000ec80 comp rx=0 tx=0).stop 2023-10-17T01:14:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 shutdown_connections 2023-10-17T01:14:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3d80a0bf0 0x7fd3d80a30b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3f0104aa0 0x7fd3f01a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f0102f10 0x7fd3f0198db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 --2- 172.21.15.45:0/3005440616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f01040e0 0x7fd3f01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 >> 172.21.15.45:0/3005440616 conn(0x7fd3f00fe4d0 msgr2=0x7fd3f0100760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 shutdown_connections 2023-10-17T01:14:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.142+0000 7fd3f61ff700 1 -- 172.21.15.45:0/3005440616 wait complete. 2023-10-17T01:14:06.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:06.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.379+0000 7fce23bff700 1 -- 172.21.15.45:0/1572213047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 msgr2=0x7fce1c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.379+0000 7fce23bff700 1 --2- 172.21.15.45:0/1572213047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 0x7fce1c0731e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce18009990 tx=0x7fce18009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.379+0000 7fce23bff700 1 -- 172.21.15.45:0/1572213047 shutdown_connections 2023-10-17T01:14:06.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.379+0000 7fce23bff700 1 --2- 172.21.15.45:0/1572213047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c108c20 0x7fce1c10b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.379+0000 7fce23bff700 1 --2- 172.21.15.45:0/1572213047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 0x7fce1c0731e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.379+0000 7fce23bff700 1 --2- 172.21.15.45:0/1572213047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c0737b0 0x7fce1c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.380+0000 7fce23bff700 1 -- 172.21.15.45:0/1572213047 >> 172.21.15.45:0/1572213047 conn(0x7fce1c0fc290 msgr2=0x7fce1c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.380+0000 7fce23bff700 1 -- 172.21.15.45:0/1572213047 shutdown_connections 2023-10-17T01:14:06.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.380+0000 7fce23bff700 1 -- 172.21.15.45:0/1572213047 wait complete. 2023-10-17T01:14:06.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.381+0000 7fce23bff700 1 Processor -- start 2023-10-17T01:14:06.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.381+0000 7fce23bff700 1 -- start start 2023-10-17T01:14:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce23bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 0x7fce1c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce23bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 0x7fce1c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce23bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c108c20 0x7fce1c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce23bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c1a07a0 con 0x7fce1c108c20 2023-10-17T01:14:06.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce23bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c1a0910 con 0x7fce1c074d90 2023-10-17T01:14:06.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce23bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c1a0ad0 con 0x7fce1c0737b0 2023-10-17T01:14:06.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce2199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 0x7fce1c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce2199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 0x7fce1c198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57032/0 (socket says 172.21.15.45:57032) 2023-10-17T01:14:06.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce2199b700 1 -- 172.21.15.45:0/3539982202 learned_addr learned my addr 172.21.15.45:0/3539982202 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:06.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce2119a700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 0x7fce1c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.382+0000 7fce2219c700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c108c20 0x7fce1c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2199b700 1 -- 172.21.15.45:0/3539982202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 msgr2=0x7fce1c1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2199b700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 0x7fce1c1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2199b700 1 -- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c108c20 msgr2=0x7fce1c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2199b700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c108c20 0x7fce1c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2199b700 1 -- 172.21.15.45:0/3539982202 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180095f0 con 0x7fce1c0737b0 2023-10-17T01:14:06.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2219c700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c108c20 0x7fce1c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:14:06.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce2199b700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 0x7fce1c198c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce18004740 tx=0x7fce18009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1801d070 con 0x7fce1c0737b0 2023-10-17T01:14:06.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce18004830 con 0x7fce1c0737b0 2023-10-17T01:14:06.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c1a0d50 con 0x7fce1c0737b0 2023-10-17T01:14:06.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18005450 con 0x7fce1c0737b0 2023-10-17T01:14:06.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.383+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c1a1170 con 0x7fce1c0737b0 2023-10-17T01:14:06.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.385+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c1072d0 con 0x7fce1c0737b0 2023-10-17T01:14:06.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.387+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fce1800fb00 con 0x7fce1c0737b0 2023-10-17T01:14:06.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.389+0000 7fce12ffd700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce080a0c70 0x7fce080a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.389+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce180cd510 con 0x7fce1c0737b0 2023-10-17T01:14:06.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.389+0000 7fce2119a700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce080a0c70 0x7fce080a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.390+0000 7fce2119a700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce080a0c70 0x7fce080a3130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce1c074ab0 tx=0x7fce0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.392+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fce1805b0b0 con 0x7fce1c0737b0 2023-10-17T01:14:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.709+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce1c02cc30 con 0x7fce1c0737b0 2023-10-17T01:14:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.710+0000 7fce12ffd700 1 -- 172.21.15.45:0/3539982202 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7fce18096da0 con 0x7fce1c0737b0 2023-10-17T01:14:06.712 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:06.712 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce080a0c70 msgr2=0x7fce080a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce080a0c70 0x7fce080a3130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce1c074ab0 tx=0x7fce0c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 msgr2=0x7fce1c198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 0x7fce1c198c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce18004740 tx=0x7fce18009580 comp rx=0 tx=0).stop 2023-10-17T01:14:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 shutdown_connections 2023-10-17T01:14:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce080a0c70 0x7fce080a3130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce1c108c20 0x7fce1c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0737b0 0x7fce1c198c60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 --2- 172.21.15.45:0/3539982202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce1c074d90 0x7fce1c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 >> 172.21.15.45:0/3539982202 conn(0x7fce1c0fc290 msgr2=0x7fce1c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 shutdown_connections 2023-10-17T01:14:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.713+0000 7fce23bff700 1 -- 172.21.15.45:0/3539982202 wait complete. 2023-10-17T01:14:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:06.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-17T01:13:01.038089+0000', 'modified': '2023-10-17T01:13:08.449854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24587, 'mds_1': 24550, 'mds_2': 24539, 'mds_3': 24526, 'mds_4': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14544': {'gid': 14544, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/931419854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 931419854}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 931419854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14556': {'gid': 14556, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6841/1667366220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 1667366220}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 1667366220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/2648148860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2648148860}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2648148860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24473': {'gid': 24473, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6833/3345979209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 3345979209}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 3345979209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/1725566254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1725566254}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1725566254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6835/1888494346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1888494346}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 1888494346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6837/1396793294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1396793294}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1396793294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.60:6837/1246290891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6836', 'nonce': 1246290891}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 1246290891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/1273587217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1273587217}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1273587217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6839/165785544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6838', 'nonce': 165785544}, {'type': 'v1', 'addr': '172.21.15.60:6839', 'nonce': 165785544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-17T01:14:06.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-17T01:14:06.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-17T01:14:06.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:06 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4245153384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:06.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:06 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3005440616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:06.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.796 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:06 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:06 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/4245153384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:06.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:06 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3005440616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:06.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:06 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:06 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/4245153384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:06 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3005440616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:06 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:06.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 -- 172.21.15.45:0/43854508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 msgr2=0x7fa914101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/43854508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914101180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa910009920 tx=0x7fa910009c30 comp rx=0 tx=0).stop 2023-10-17T01:14:06.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 -- 172.21.15.45:0/43854508 shutdown_connections 2023-10-17T01:14:06.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/43854508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9141028f0 0x7fa9141091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/43854508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914101180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/43854508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa914101f50 0x7fa9141023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 -- 172.21.15.45:0/43854508 >> 172.21.15.45:0/43854508 conn(0x7fa9140fc2d0 msgr2=0x7fa9140fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 -- 172.21.15.45:0/43854508 shutdown_connections 2023-10-17T01:14:06.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.945+0000 7fa91b3ff700 1 -- 172.21.15.45:0/43854508 wait complete. 2023-10-17T01:14:06.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 Processor -- start 2023-10-17T01:14:06.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 -- start start 2023-10-17T01:14:06.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914101f50 0x7fa914199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 0x7fa9141a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141a0890 con 0x7fa914101f50 2023-10-17T01:14:06.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141a0a00 con 0x7fa914100d60 2023-10-17T01:14:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.946+0000 7fa91b3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141a0bc0 con 0x7fa9141028f0 2023-10-17T01:14:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 0x7fa9141a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 0x7fa9141a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57052/0 (socket says 172.21.15.45:57052) 2023-10-17T01:14:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 -- 172.21.15.45:0/243551990 learned_addr learned my addr 172.21.15.45:0/243551990 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91899a700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914101f50 0x7fa914199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91919b700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 -- 172.21.15.45:0/243551990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 msgr2=0x7fa914198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 -- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914101f50 msgr2=0x7fa914199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914101f50 0x7fa914199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 -- 172.21.15.45:0/243551990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009520 con 0x7fa9141028f0 2023-10-17T01:14:06.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91919b700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:06.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa91999c700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 0x7fa9141a02c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c110 tx=0x7fa90c00c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.947+0000 7fa90a7fc700 1 -- 172.21.15.45:0/243551990 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c017070 con 0x7fa9141028f0 2023-10-17T01:14:06.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.948+0000 7fa90a7fc700 1 -- 172.21.15.45:0/243551990 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90c00e050 con 0x7fa9141028f0 2023-10-17T01:14:06.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.948+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9141a0e40 con 0x7fa9141028f0 2023-10-17T01:14:06.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.948+0000 7fa90a7fc700 1 -- 172.21.15.45:0/243551990 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c007940 con 0x7fa9141028f0 2023-10-17T01:14:06.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.948+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141a1260 con 0x7fa9141028f0 2023-10-17T01:14:06.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.951+0000 7fa90a7fc700 1 -- 172.21.15.45:0/243551990 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fa90c004ad0 con 0x7fa9141028f0 2023-10-17T01:14:06.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 Processor -- start 2023-10-17T01:14:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 -- start start 2023-10-17T01:14:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db20935f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db20936cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db209376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2088d9c0 con 0x55db20937260 2023-10-17T01:14:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2088b380 con 0x55db209364b0 2023-10-17T01:14:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa90a7fc700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9000a0bb0 0x7fa9000a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.952+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20886c70 con 0x55db20935b50 2023-10-17T01:14:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa90a7fc700 1 -- 172.21.15.45:0/243551990 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa90c0cbdc0 con 0x7fa9141028f0 2023-10-17T01:14:06.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91899a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db20935f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db20936cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91899a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db20935f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57054/0 (socket says 172.21.15.45:57054) 2023-10-17T01:14:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db20936cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46878/0 (socket says 172.21.15.45:46878) 2023-10-17T01:14:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91899a700 1 -- 172.21.15.45:0/3341368448 learned_addr learned my addr 172.21.15.45:0/3341368448 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:06.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91999c700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9000a0bb0 0x7fa9000a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91899a700 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db209376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 -- 172.21.15.45:0/3341368448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 msgr2=0x55db20935f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db20935f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 -- 172.21.15.45:0/3341368448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 msgr2=0x55db209376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db209376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 -- 172.21.15.45:0/3341368448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db206b4d90 con 0x55db209364b0 2023-10-17T01:14:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91919b700 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db20936cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa91000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64bec65abeda1bb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.953+0000 7fa91999c700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9000a0bb0 0x7fa9000a3070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009de0 tx=0x7fa90c0208b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa8ff7fe700 1 -- 172.21.15.45:0/3341368448 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91001c040 con 0x55db209364b0 2023-10-17T01:14:06.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa8ff7fe700 1 -- 172.21.15.45:0/3341368448 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91000f760 con 0x55db209364b0 2023-10-17T01:14:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa8ff7fe700 1 -- 172.21.15.45:0/3341368448 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910003680 con 0x55db209364b0 2023-10-17T01:14:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 -- 172.21.15.45:0/3341368448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 msgr2=0x55db20936cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db20936cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa91000bef0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 -- 172.21.15.45:0/3341368448 shutdown_connections 2023-10-17T01:14:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db209376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db20935f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/3341368448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db20936cd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 -- 172.21.15.45:0/3341368448 >> 172.21.15.45:0/3341368448 conn(0x55db20895080 msgr2=0x55db20930900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 -- 172.21.15.45:0/3341368448 shutdown_connections 2023-10-17T01:14:06.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.954+0000 7fa92eda5b80 1 -- 172.21.15.45:0/3341368448 wait complete. 2023-10-17T01:14:06.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.955+0000 7fa92eda5b80 1 -- start start 2023-10-17T01:14:06.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.955+0000 7fa92eda5b80 1 Processor -- start 2023-10-17T01:14:06.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.955+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db2093f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.955+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db2093fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db20946e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db208234c0 con 0x55db20937260 2023-10-17T01:14:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20823160 con 0x55db209364b0 2023-10-17T01:14:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2070a8d0 con 0x55db20935b50 2023-10-17T01:14:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa92eda5b80 10 client.0 _ll_register_callbacks cb 0x55db20770a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db2093f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db2093fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db2093f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57056/0 (socket says 172.21.15.45:57056) 2023-10-17T01:14:06.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 -- 172.21.15.45:0/835968781 learned_addr learned my addr 172.21.15.45:0/835968781 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:06.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db20946e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 -- 172.21.15.45:0/835968781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 msgr2=0x55db2093fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db2093fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 -- 172.21.15.45:0/835968781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 msgr2=0x55db20946e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db20946e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 -- 172.21.15.45:0/835968781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa914109af0 con 0x55db20935b50 2023-10-17T01:14:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91919b700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db2093fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa91899a700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db2093f8c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa90400f460 tx=0x7fa9040047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e9efb7094a9d2a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.956+0000 7fa8fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:14:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa8fcff9700 1 -- 172.21.15.45:0/835968781 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90400fc90 con 0x55db20935b50 2023-10-17T01:14:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa8fcff9700 1 -- 172.21.15.45:0/835968781 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa90400fdf0 con 0x55db20935b50 2023-10-17T01:14:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db208b1cc0 con 0x55db20935b50 2023-10-17T01:14:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa8fcff9700 1 -- 172.21.15.45:0/835968781 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa904004030 con 0x55db20935b50 2023-10-17T01:14:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa92eda5b80 10 client.24976 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55db2069dc10 con 0x55db20935b50 2023-10-17T01:14:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.957+0000 7fa8fcff9700 1 -- 172.21.15.45:0/835968781 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db208b1cc0 con 0x55db20935b50 2023-10-17T01:14:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa8fcff9700 1 -- 172.21.15.45:0/835968781 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa90402c050 con 0x55db20935b50 2023-10-17T01:14:06.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa92eda5b80 10 client.24976 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:06.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa92eda5b80 10 client.24976 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-17T01:14:06.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa92eda5b80 20 client.24976 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:06.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db2094ab70 0x55db20949fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa92eda5b80 4 client.24976 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 --> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db20904ce0 con 0x55db2094ab70 2023-10-17T01:14:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa91919b700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db2094ab70 0x55db20949fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa91919b700 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db2094ab70 0x55db20949fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee8d9e626cfff6ce server_cookie=ea74aead7b210948 in_seq=0 out_seq=0 2023-10-17T01:14:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.958+0000 7fa8fcff9700 10 client.24976 ms_handle_connect on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.959+0000 7fa8fcff9700 1 -- 172.21.15.45:0/835968781 <== mds.4 v2:172.21.15.181:6834/1888494346 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55db20904ce0 con 0x55db2094ab70 2023-10-17T01:14:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.959+0000 7fa8fcff9700 10 client.24976 handle_command_reply: tid=0 2023-10-17T01:14:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 client.24976 shutdown 2023-10-17T01:14:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 msgr2=0x55db2093f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db2093f8c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa90400f460 tx=0x7fa9040047c0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 shutdown_connections 2023-10-17T01:14:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db2094ab70 0x55db20949fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20937260 0x55db20946e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db2093f8c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/835968781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db209364b0 0x55db2093fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa8fcff9700 0 client.24976 ms_handle_reset on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:06.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 >> 172.21.15.45:0/835968781 conn(0x55db20895080 msgr2=0x55db2092ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 shutdown_connections 2023-10-17T01:14:06.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 1 -- 172.21.15.45:0/835968781 wait complete. 2023-10-17T01:14:06.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.960+0000 7fa92eda5b80 20 client.24976 trim_cache size 0 max 16384 2023-10-17T01:14:06.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 Processor -- start 2023-10-17T01:14:06.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 -- start start 2023-10-17T01:14:06.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20935b50 0x55db20949fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 0x55db2093a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db2093a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2093de30 con 0x55db20935b50 2023-10-17T01:14:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20830830 con 0x55db20937260 2023-10-17T01:14:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.978+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2088bcf0 con 0x55db209364b0 2023-10-17T01:14:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 0x55db2093a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91899a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20935b50 0x55db20949fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 0x55db2093a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57072/0 (socket says 172.21.15.45:57072) 2023-10-17T01:14:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 -- 172.21.15.45:0/2817597919 learned_addr learned my addr 172.21.15.45:0/2817597919 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:06.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91899a700 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db2093a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 -- 172.21.15.45:0/2817597919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 msgr2=0x55db2093a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db2093a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 -- 172.21.15.45:0/2817597919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20935b50 msgr2=0x55db20949fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20935b50 0x55db20949fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 -- 172.21.15.45:0/2817597919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db206db200 con 0x55db209364b0 2023-10-17T01:14:06.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91899a700 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db2093a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:14:06.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa91919b700 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 0x55db2093a250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55db209368b0 tx=0x7fa910009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00ed5947b9882cd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.979+0000 7fa8fffff700 1 -- 172.21.15.45:0/2817597919 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91001c040 con 0x55db209364b0 2023-10-17T01:14:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa8fffff700 1 -- 172.21.15.45:0/2817597919 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa910031dc0 con 0x55db209364b0 2023-10-17T01:14:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa8fffff700 1 -- 172.21.15.45:0/2817597919 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000b9e0 con 0x55db209364b0 2023-10-17T01:14:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2817597919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 msgr2=0x55db2093a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 0x55db2093a250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55db209368b0 tx=0x7fa910009c30 comp rx=0 tx=0).stop 2023-10-17T01:14:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2817597919 shutdown_connections 2023-10-17T01:14:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db2093a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db20935b50 0x55db20949fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2817597919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db209364b0 0x55db2093a250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2817597919 >> 172.21.15.45:0/2817597919 conn(0x55db20895080 msgr2=0x55db2093d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2817597919 shutdown_connections 2023-10-17T01:14:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.980+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2817597919 wait complete. 2023-10-17T01:14:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.981+0000 7fa92eda5b80 1 -- start start 2023-10-17T01:14:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.981+0000 7fa92eda5b80 1 Processor -- start 2023-10-17T01:14:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db209535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db209364b0 0x55db209501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db20950720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20950ca0 con 0x55db209364b0 2023-10-17T01:14:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db20950e10 con 0x55db20937260 2023-10-17T01:14:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2099c510 con 0x55db20935b50 2023-10-17T01:14:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa92eda5b80 10 client.0 _ll_register_callbacks cb 0x7fa904011d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91899a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db209535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91919b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db209364b0 0x55db209501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91899a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db209535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57086/0 (socket says 172.21.15.45:57086) 2023-10-17T01:14:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91899a700 1 -- 172.21.15.45:0/2389747464 learned_addr learned my addr 172.21.15.45:0/2389747464 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91899a700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db20950720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91919b700 1 -- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 msgr2=0x55db209535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91919b700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db209535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91919b700 1 -- 172.21.15.45:0/2389747464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 msgr2=0x55db20950720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91919b700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db20950720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91919b700 1 -- 172.21.15.45:0/2389747464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db206b4d90 con 0x55db209364b0 2023-10-17T01:14:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.982+0000 7fa91899a700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db209535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa91919b700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db209364b0 0x55db209501e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55db2092cf40 tx=0x7fa9100057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba81e0ac5f910ac server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa8fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa8fcff9700 1 -- 172.21.15.45:0/2389747464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91001c040 con 0x55db209364b0 2023-10-17T01:14:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa91000f760 con 0x55db209364b0 2023-10-17T01:14:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa8fcff9700 1 -- 172.21.15.45:0/2389747464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa910031b00 con 0x55db209364b0 2023-10-17T01:14:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa8fcff9700 1 -- 172.21.15.45:0/2389747464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9100052f0 con 0x55db209364b0 2023-10-17T01:14:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa92eda5b80 10 client.14958 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55db206b3260 con 0x55db209364b0 2023-10-17T01:14:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.983+0000 7fa8fcff9700 1 -- 172.21.15.45:0/2389747464 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91000f760 con 0x55db209364b0 2023-10-17T01:14:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa8fcff9700 1 -- 172.21.15.45:0/2389747464 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa910015020 con 0x55db209364b0 2023-10-17T01:14:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa92eda5b80 10 client.14958 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa92eda5b80 10 client.14958 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-17T01:14:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa92eda5b80 20 client.14958 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db209a0d60 0x55db2099d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa92eda5b80 4 client.14958 mds_command: new command op to 24514 tid=0[{"prefix": "status"}] 2023-10-17T01:14:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 --> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db2087fcf0 con 0x55db209a0d60 2023-10-17T01:14:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa91899a700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db209a0d60 0x55db2099d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa91899a700 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db209a0d60 0x55db2099d6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c3c0d99c452cfaa server_cookie=f14eb97a89af1691 in_seq=0 out_seq=0 2023-10-17T01:14:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.984+0000 7fa8fcff9700 10 client.14958 ms_handle_connect on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa8fcff9700 1 -- 172.21.15.45:0/2389747464 <== mds.4 v2:172.21.15.181:6834/1888494346 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55db2087fcf0 con 0x55db209a0d60 2023-10-17T01:14:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa8fcff9700 10 client.14958 handle_command_reply: tid=0 2023-10-17T01:14:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 client.14958 shutdown 2023-10-17T01:14:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db209364b0 msgr2=0x55db209501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db209364b0 0x55db209501e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55db2092cf40 tx=0x7fa9100057a0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 shutdown_connections 2023-10-17T01:14:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55db209a0d60 0x55db2099d6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55db20937260 0x55db20950720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55db20935b50 0x55db209535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 --2- 172.21.15.45:0/2389747464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db209364b0 0x55db209501e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa8fcff9700 0 client.14958 ms_handle_reset on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 >> 172.21.15.45:0/2389747464 conn(0x55db20895080 msgr2=0x55db20949fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 shutdown_connections 2023-10-17T01:14:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 1 -- 172.21.15.45:0/2389747464 wait complete. 2023-10-17T01:14:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.985+0000 7fa92eda5b80 20 client.14958 trim_cache size 0 max 16384 2023-10-17T01:14:07.003 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:07.004 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:07.004 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 4, 2023-10-17T01:14:07.004 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24514, 2023-10-17T01:14:07.004 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:07.004 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:07.005 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:07.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 59.537335896000002, 2023-10-17T01:14:07.005 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:07.005 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:07.005 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:07.006 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 86.900348520999998 2023-10-17T01:14:07.006 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9000a0bb0 msgr2=0x7fa9000a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9000a0bb0 0x7fa9000a3070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009de0 tx=0x7fa90c0208b0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 msgr2=0x7fa9141a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 0x7fa9141a02c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c110 tx=0x7fa90c00c420 comp rx=0 tx=0).stop 2023-10-17T01:14:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 shutdown_connections 2023-10-17T01:14:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9000a0bb0 0x7fa9000a3070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141028f0 0x7fa9141a02c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa914100d60 0x7fa914198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 --2- 172.21.15.45:0/243551990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914101f50 0x7fa914199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 >> 172.21.15.45:0/243551990 conn(0x7fa9140fc2d0 msgr2=0x7fa9141078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 shutdown_connections 2023-10-17T01:14:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:06.986+0000 7fa91b3ff700 1 -- 172.21.15.45:0/243551990 wait complete. 2023-10-17T01:14:07.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-17T01:14:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 -- 172.21.15.45:0/3911437265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398074d90 msgr2=0x7fc3980731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/3911437265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398074d90 0x7fc3980731e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc394009990 tx=0x7fc394009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 -- 172.21.15.45:0/3911437265 shutdown_connections 2023-10-17T01:14:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/3911437265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc39810d310 0x7fc39810f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/3911437265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc398074d90 0x7fc3980731e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/3911437265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3980737b0 0x7fc398073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 -- 172.21.15.45:0/3911437265 >> 172.21.15.45:0/3911437265 conn(0x7fc3980fc290 msgr2=0x7fc3980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 -- 172.21.15.45:0/3911437265 shutdown_connections 2023-10-17T01:14:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.229+0000 7fc39f3ff700 1 -- 172.21.15.45:0/3911437265 wait complete. 2023-10-17T01:14:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 Processor -- start 2023-10-17T01:14:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 -- start start 2023-10-17T01:14:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 0x7fc39819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc398074d90 0x7fc39819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc39810d310 0x7fc3981a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3981a4c00 con 0x7fc39810d310 2023-10-17T01:14:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.230+0000 7fc39f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3981a4d40 con 0x7fc3980737b0 2023-10-17T01:14:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39f3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3981a4ed0 con 0x7fc398074d90 2023-10-17T01:14:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 0x7fc39819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 0x7fc39819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46914/0 (socket says 172.21.15.45:46914) 2023-10-17T01:14:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 -- 172.21.15.45:0/1763388987 learned_addr learned my addr 172.21.15.45:0/1763388987 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d99c700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc39810d310 0x7fc3981a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39c99a700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc398074d90 0x7fc39819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 -- 172.21.15.45:0/1763388987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc398074d90 msgr2=0x7fc39819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc398074d90 0x7fc39819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 -- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc39810d310 msgr2=0x7fc3981a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc39810d310 0x7fc3981a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 -- 172.21.15.45:0/1763388987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940095f0 con 0x7fc3980737b0 2023-10-17T01:14:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39c99a700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc398074d90 0x7fc39819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.231+0000 7fc39d19b700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 0x7fc39819d080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc394005910 tx=0x7fc39400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.232+0000 7fc38e7fc700 1 -- 172.21.15.45:0/1763388987 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39401d070 con 0x7fc3980737b0 2023-10-17T01:14:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.232+0000 7fc38e7fc700 1 -- 172.21.15.45:0/1763388987 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc39400fb80 con 0x7fc3980737b0 2023-10-17T01:14:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.232+0000 7fc38e7fc700 1 -- 172.21.15.45:0/1763388987 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394017650 con 0x7fc3980737b0 2023-10-17T01:14:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.232+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3981a5150 con 0x7fc3980737b0 2023-10-17T01:14:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.232+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3981a5520 con 0x7fc3980737b0 2023-10-17T01:14:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.235+0000 7fc38e7fc700 1 -- 172.21.15.45:0/1763388987 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fc39400fcf0 con 0x7fc3980737b0 2023-10-17T01:14:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 Processor -- start 2023-10-17T01:14:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 -- start start 2023-10-17T01:14:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc38e7fc700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc3840a0bb0 0x7fc3840a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966edb50 0x55a9966edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a9966eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ef260 0x55a9966ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc38e7fc700 1 -- 172.21.15.45:0/1763388987 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc3940cd8c0 con 0x7fc3980737b0 2023-10-17T01:14:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9966459c0 con 0x55a9966ef260 2023-10-17T01:14:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a996643380 con 0x55a9966ee4b0 2023-10-17T01:14:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.237+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99663ec70 con 0x55a9966edb50 2023-10-17T01:14:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a9966eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39d99c700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc3840a0bb0 0x7fc3840a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a9966eecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46924/0 (socket says 172.21.15.45:46924) 2023-10-17T01:14:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 -- 172.21.15.45:0/3417931067 learned_addr learned my addr 172.21.15.45:0/3417931067 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39d99c700 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ef260 0x55a9966ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 -- 172.21.15.45:0/3417931067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966edb50 msgr2=0x55a9966edf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966edb50 0x55a9966edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 -- 172.21.15.45:0/3417931067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ef260 msgr2=0x55a9966ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ef260 0x55a9966ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 -- 172.21.15.45:0/3417931067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a99646cd90 con 0x55a9966ee4b0 2023-10-17T01:14:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a9966eecd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc38800bd40 tx=0x7fc38800f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e104786054fa5d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc39d99c700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc3840a0bb0 0x7fc3840a3070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc390007770 tx=0x7fc390009940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc3837fe700 1 -- 172.21.15.45:0/3417931067 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800fe50 con 0x55a9966ee4b0 2023-10-17T01:14:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc3837fe700 1 -- 172.21.15.45:0/3417931067 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc38800be90 con 0x55a9966ee4b0 2023-10-17T01:14:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.238+0000 7fc3837fe700 1 -- 172.21.15.45:0/3417931067 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38801b770 con 0x55a9966ee4b0 2023-10-17T01:14:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3417931067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 msgr2=0x55a9966eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a9966eecd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc38800bd40 tx=0x7fc38800f640 comp rx=0 tx=0).stop 2023-10-17T01:14:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3417931067 shutdown_connections 2023-10-17T01:14:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ef260 0x55a9966ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966edb50 0x55a9966edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3417931067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a9966eecd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3417931067 >> 172.21.15.45:0/3417931067 conn(0x55a99664d080 msgr2=0x55a9966e8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3417931067 shutdown_connections 2023-10-17T01:14:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.239+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3417931067 wait complete. 2023-10-17T01:14:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 -- start start 2023-10-17T01:14:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 Processor -- start 2023-10-17T01:14:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966edb50 0x55a9966f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 0x55a9966f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a9966fee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9965db4c0 con 0x55a9966ee4b0 2023-10-17T01:14:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9965db160 con 0x55a9966edb50 2023-10-17T01:14:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9964c28d0 con 0x55a9966ef260 2023-10-17T01:14:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.240+0000 7fc3b2c37b80 10 client.0 _ll_register_callbacks cb 0x55a996528a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39c99a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 0x55a9966f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39c99a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 0x55a9966f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39446/0 (socket says 172.21.15.45:39446) 2023-10-17T01:14:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39c99a700 1 -- 172.21.15.45:0/3981719736 learned_addr learned my addr 172.21.15.45:0/3981719736 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d99c700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a9966fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966edb50 0x55a9966f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 -- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 msgr2=0x55a9966fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a9966fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 -- 172.21.15.45:0/3981719736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 msgr2=0x55a9966f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 0x55a9966f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 -- 172.21.15.45:0/3981719736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38800b9a0 con 0x55a9966edb50 2023-10-17T01:14:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 0x55a9966f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966edb50 0x55a9966f78f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3940cb240 tx=0x7fc39409b350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e688a7f19174deea server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc380ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc380ff9700 1 -- 172.21.15.45:0/3981719736 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940a2b60 con 0x55a9966edb50 2023-10-17T01:14:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc380ff9700 1 -- 172.21.15.45:0/3981719736 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc39409ae10 con 0x55a9966edb50 2023-10-17T01:14:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.241+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a996669cc0 con 0x55a9966edb50 2023-10-17T01:14:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc380ff9700 1 -- 172.21.15.45:0/3981719736 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc39409ec40 con 0x55a9966edb50 2023-10-17T01:14:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 10 client.24992 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a996455c10 con 0x55a9966edb50 2023-10-17T01:14:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc380ff9700 1 -- 172.21.15.45:0/3981719736 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a996669cc0 con 0x55a9966edb50 2023-10-17T01:14:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc380ff9700 1 -- 172.21.15.45:0/3981719736 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc394005240 con 0x55a9966edb50 2023-10-17T01:14:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 10 client.24992 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 10 client.24992 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-17T01:14:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 20 client.24992 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996702c80 0x55a996701da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 4 client.24992 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.242+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 --> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9966bcce0 con 0x55a996702c80 2023-10-17T01:14:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.243+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996702c80 0x55a996701da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.243+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996702c80 0x55a996701da0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb6a4b68915419bb server_cookie=3e88707d65018038 in_seq=0 out_seq=0 2023-10-17T01:14:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.243+0000 7fc380ff9700 10 client.24992 ms_handle_connect on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc380ff9700 1 -- 172.21.15.45:0/3981719736 <== mds.3 v2:172.21.15.181:6836/1396793294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55a9966bcce0 con 0x55a996702c80 2023-10-17T01:14:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc380ff9700 10 client.24992 handle_command_reply: tid=0 2023-10-17T01:14:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 client.24992 shutdown 2023-10-17T01:14:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966edb50 msgr2=0x55a9966f78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966edb50 0x55a9966f78f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3940cb240 tx=0x7fc39409b350 comp rx=0 tx=0).stop 2023-10-17T01:14:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 shutdown_connections 2023-10-17T01:14:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a9966fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966edb50 0x55a9966f78f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996702c80 0x55a996701da0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3981719736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966ee4b0 0x55a9966f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc380ff9700 0 client.24992 ms_handle_reset on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 >> 172.21.15.45:0/3981719736 conn(0x55a99664d080 msgr2=0x55a9966e7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 shutdown_connections 2023-10-17T01:14:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.244+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3981719736 wait complete. 2023-10-17T01:14:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.245+0000 7fc3b2c37b80 20 client.24992 trim_cache size 0 max 16384 2023-10-17T01:14:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 Processor -- start 2023-10-17T01:14:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 -- start start 2023-10-17T01:14:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9966ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ee4b0 0x55a9966f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ef260 0x55a9966f6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9966f74c0 con 0x55a9966edb50 2023-10-17T01:14:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9966b3140 con 0x55a9966ef260 2023-10-17T01:14:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9966f87f0 con 0x55a9966ee4b0 2023-10-17T01:14:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc39d19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9966ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc39d19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9966ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39448/0 (socket says 172.21.15.45:39448) 2023-10-17T01:14:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc39d19b700 1 -- 172.21.15.45:0/3648454678 learned_addr learned my addr 172.21.15.45:0/3648454678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc39d99c700 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ef260 0x55a9966f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.263+0000 7fc39c99a700 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ee4b0 0x55a9966f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc39d19b700 1 -- 172.21.15.45:0/3648454678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ee4b0 msgr2=0x55a9966f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ee4b0 0x55a9966f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc39d19b700 1 -- 172.21.15.45:0/3648454678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ef260 msgr2=0x55a9966f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ef260 0x55a9966f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc39d19b700 1 -- 172.21.15.45:0/3648454678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a996493200 con 0x55a9966edb50 2023-10-17T01:14:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc39d19b700 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9966ff6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc394000c00 tx=0x7fc394022b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff1a961e58b9287 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.264+0000 7fc383fff700 1 -- 172.21.15.45:0/3648454678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39409ae10 con 0x55a9966edb50 2023-10-17T01:14:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.267+0000 7fc383fff700 1 -- 172.21.15.45:0/3648454678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc394099d70 con 0x55a9966edb50 2023-10-17T01:14:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.267+0000 7fc383fff700 1 -- 172.21.15.45:0/3648454678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940a2bc0 con 0x55a9966edb50 2023-10-17T01:14:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3648454678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 msgr2=0x55a9966ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9966ff6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc394000c00 tx=0x7fc394022b10 comp rx=0 tx=0).stop 2023-10-17T01:14:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3648454678 shutdown_connections 2023-10-17T01:14:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ef260 0x55a9966f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9966ff6a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/3648454678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ee4b0 0x55a9966f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3648454678 >> 172.21.15.45:0/3648454678 conn(0x55a99664d080 msgr2=0x55a996701da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3648454678 shutdown_connections 2023-10-17T01:14:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/3648454678 wait complete. 2023-10-17T01:14:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 -- start start 2023-10-17T01:14:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.268+0000 7fc3b2c37b80 1 Processor -- start 2023-10-17T01:14:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9967080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a996708620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a996753fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a996708bf0 con 0x55a9966edb50 2023-10-17T01:14:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a996708d60 con 0x55a9966ee4b0 2023-10-17T01:14:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a996754580 con 0x55a9966ef260 2023-10-17T01:14:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc3b2c37b80 10 client.0 _ll_register_callbacks cb 0x7fc39409abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a996708620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39d19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9967080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a996708620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46952/0 (socket says 172.21.15.45:46952) 2023-10-17T01:14:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 -- 172.21.15.45:0/2517941042 learned_addr learned my addr 172.21.15.45:0/2517941042 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39d99c700 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a996753fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 -- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 msgr2=0x55a996753fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a996753fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 -- 172.21.15.45:0/2517941042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 msgr2=0x55a9967080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.269+0000 7fc39c99a700 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9967080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.270+0000 7fc39c99a700 1 -- 172.21.15.45:0/2517941042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a996455c10 con 0x55a9966ee4b0 2023-10-17T01:14:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.270+0000 7fc39c99a700 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a996708620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc38800b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de0318abc2b954 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.270+0000 7fc380ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.270+0000 7fc380ff9700 1 -- 172.21.15.45:0/2517941042 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38801f040 con 0x55a9966ee4b0 2023-10-17T01:14:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.270+0000 7fc380ff9700 1 -- 172.21.15.45:0/2517941042 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc38802a300 con 0x55a9966ee4b0 2023-10-17T01:14:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.270+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc38800be90 con 0x55a9966ee4b0 2023-10-17T01:14:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.272+0000 7fc380ff9700 1 -- 172.21.15.45:0/2517941042 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc388004220 con 0x55a9966ee4b0 2023-10-17T01:14:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 10 client.24998 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a99646b260 con 0x55a9966ee4b0 2023-10-17T01:14:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc380ff9700 1 -- 172.21.15.45:0/2517941042 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc38800be90 con 0x55a9966ee4b0 2023-10-17T01:14:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc380ff9700 1 -- 172.21.15.45:0/2517941042 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc388030020 con 0x55a9966ee4b0 2023-10-17T01:14:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 10 client.24998 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 10 client.24998 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-17T01:14:07.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 20 client.24998 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:07.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996759280 0x55a996755bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 4 client.24998 mds_command: new command op to 24526 tid=0[{"prefix": "status"}] 2023-10-17T01:14:07.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 --> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a996637cf0 con 0x55a996759280 2023-10-17T01:14:07.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.273+0000 7fc39d19b700 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996759280 0x55a996755bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc39d19b700 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996759280 0x55a996755bf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2be4c8d7c2e9051d server_cookie=36e9e91a91e6acc9 in_seq=0 out_seq=0 2023-10-17T01:14:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc380ff9700 10 client.24998 ms_handle_connect on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc380ff9700 1 -- 172.21.15.45:0/2517941042 <== mds.3 v2:172.21.15.181:6836/1396793294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a996637cf0 con 0x55a996759280 2023-10-17T01:14:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc380ff9700 10 client.24998 handle_command_reply: tid=0 2023-10-17T01:14:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 client.24998 shutdown 2023-10-17T01:14:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 msgr2=0x55a996708620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a996708620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc38800b6d0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 shutdown_connections 2023-10-17T01:14:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9966ef260 0x55a996753fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55a996759280 0x55a996755bf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a9966edb50 0x55a9967080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 --2- 172.21.15.45:0/2517941042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a9966ee4b0 0x55a996708620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc380ff9700 0 client.24998 ms_handle_reset on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.274+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 >> 172.21.15.45:0/2517941042 conn(0x55a99664d080 msgr2=0x55a9966f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 shutdown_connections 2023-10-17T01:14:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc3b2c37b80 1 -- 172.21.15.45:0/2517941042 wait complete. 2023-10-17T01:14:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc3b2c37b80 20 client.24998 trim_cache size 0 max 16384 2023-10-17T01:14:07.288 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:07.289 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:07.289 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 3, 2023-10-17T01:14:07.289 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24526, 2023-10-17T01:14:07.289 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:07.289 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:07.289 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:07.290 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 60.835036315000004, 2023-10-17T01:14:07.290 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:07.290 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:07.290 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:07.290 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 84.948884554000003 2023-10-17T01:14:07.291 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc3840a0bb0 msgr2=0x7fc3840a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc3840a0bb0 0x7fc3840a3070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc390007770 tx=0x7fc390009940 comp rx=0 tx=0).stop 2023-10-17T01:14:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 msgr2=0x7fc39819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 0x7fc39819d080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc394005910 tx=0x7fc39400f690 comp rx=0 tx=0).stop 2023-10-17T01:14:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 shutdown_connections 2023-10-17T01:14:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc3840a0bb0 0x7fc3840a3070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc39810d310 0x7fc3981a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3980737b0 0x7fc39819d080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 --2- 172.21.15.45:0/1763388987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc398074d90 0x7fc39819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.275+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 >> 172.21.15.45:0/1763388987 conn(0x7fc3980fc290 msgr2=0x7fc3980fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.276+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 shutdown_connections 2023-10-17T01:14:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.276+0000 7fc39f3ff700 1 -- 172.21.15.45:0/1763388987 wait complete. 2023-10-17T01:14:07.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-17T01:14:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 -- 172.21.15.45:0/2060025239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 msgr2=0x7f3078101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 --2- 172.21.15.45:0/2060025239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078101160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30680098f0 tx=0x7f3068009c00 comp rx=0 tx=0).stop 2023-10-17T01:14:07.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 -- 172.21.15.45:0/2060025239 shutdown_connections 2023-10-17T01:14:07.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 --2- 172.21.15.45:0/2060025239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30781028d0 0x7f3078109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 --2- 172.21.15.45:0/2060025239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078101160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 --2- 172.21.15.45:0/2060025239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3078101f30 0x7f3078102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 -- 172.21.15.45:0/2060025239 >> 172.21.15.45:0/2060025239 conn(0x7f30780fc290 msgr2=0x7f30780fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 -- 172.21.15.45:0/2060025239 shutdown_connections 2023-10-17T01:14:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.509+0000 7f307e5ff700 1 -- 172.21.15.45:0/2060025239 wait complete. 2023-10-17T01:14:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.511+0000 7f307e5ff700 1 Processor -- start 2023-10-17T01:14:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.511+0000 7f307e5ff700 1 -- start start 2023-10-17T01:14:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307e5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3078101f30 0x7f3078073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307e5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30781028d0 0x7f3078073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307e5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3078073e20 con 0x7f30781028d0 2023-10-17T01:14:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3078073f90 con 0x7f3078101f30 2023-10-17T01:14:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307e5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30781a0b00 con 0x7f3078100d40 2023-10-17T01:14:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f307cb9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30781028d0 0x7f3078073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57154/0 (socket says 172.21.15.45:57154) 2023-10-17T01:14:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f30777fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3078101f30 0x7f3078073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 -- 172.21.15.45:0/285264835 learned_addr learned my addr 172.21.15.45:0/285264835 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 -- 172.21.15.45:0/285264835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3078101f30 msgr2=0x7f3078073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3078101f30 0x7f3078073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 -- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30781028d0 msgr2=0x7f3078073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30781028d0 0x7f3078073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.512+0000 7f3077fff700 1 -- 172.21.15.45:0/285264835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070009d00 con 0x7f3078100d40 2023-10-17T01:14:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.513+0000 7f3077fff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078074b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3068000c00 tx=0x7f306800b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.513+0000 7f30757fa700 1 -- 172.21.15.45:0/285264835 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306801c070 con 0x7f3078100d40 2023-10-17T01:14:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.513+0000 7f30757fa700 1 -- 172.21.15.45:0/285264835 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f306800bb90 con 0x7f3078100d40 2023-10-17T01:14:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.513+0000 7f30757fa700 1 -- 172.21.15.45:0/285264835 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30680039f0 con 0x7f3078100d40 2023-10-17T01:14:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.513+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068009520 con 0x7f3078100d40 2023-10-17T01:14:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.513+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30781a1010 con 0x7f3078100d40 2023-10-17T01:14:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.516+0000 7f30757fa700 1 -- 172.21.15.45:0/285264835 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f3068020030 con 0x7f3078100d40 2023-10-17T01:14:07.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.517+0000 7f30757fa700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f30600a0c30 0x7f30600a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.518+0000 7f30757fa700 1 -- 172.21.15.45:0/285264835 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f30680cd300 con 0x7f3078100d40 2023-10-17T01:14:07.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.518+0000 7f30777fe700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f30600a0c30 0x7f30600a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.518+0000 7f30777fe700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f30600a0c30 0x7f30600a30f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f306c009c60 tx=0x7f306c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.520+0000 7f3091fdeb80 1 Processor -- start 2023-10-17T01:14:07.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.520+0000 7f3091fdeb80 1 -- start start 2023-10-17T01:14:07.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d93fb50 0x55ef5d93ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 0x55ef5d940cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d9416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d8979c0 con 0x55ef5d93fb50 2023-10-17T01:14:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d895380 con 0x55ef5d941260 2023-10-17T01:14:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d890c70 con 0x55ef5d9404b0 2023-10-17T01:14:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 0x55ef5d940cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3077fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d93fb50 0x55ef5d93ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 0x55ef5d940cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57156/0 (socket says 172.21.15.45:57156) 2023-10-17T01:14:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 -- 172.21.15.45:0/1479673393 learned_addr learned my addr 172.21.15.45:0/1479673393 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f307cb9c700 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d9416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 -- 172.21.15.45:0/1479673393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 msgr2=0x55ef5d9416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d9416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 -- 172.21.15.45:0/1479673393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d93fb50 msgr2=0x55ef5d93ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d93fb50 0x55ef5d93ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f30777fe700 1 -- 172.21.15.45:0/1479673393 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef5d6bed90 con 0x55ef5d9404b0 2023-10-17T01:14:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.521+0000 7f3077fff700 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d93fb50 0x55ef5d93ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.522+0000 7f30777fe700 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 0x55ef5d940cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e4e0 tx=0x7f306c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f4b7e8eadccc34 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.522+0000 7f305e7fc700 1 -- 172.21.15.45:0/1479673393 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c00ee00 con 0x55ef5d9404b0 2023-10-17T01:14:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.522+0000 7f305e7fc700 1 -- 172.21.15.45:0/1479673393 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f306c021070 con 0x55ef5d9404b0 2023-10-17T01:14:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.522+0000 7f305e7fc700 1 -- 172.21.15.45:0/1479673393 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c01d7f0 con 0x55ef5d9404b0 2023-10-17T01:14:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1479673393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 msgr2=0x55ef5d940cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 0x55ef5d940cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e4e0 tx=0x7f306c014000 comp rx=0 tx=0).stop 2023-10-17T01:14:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1479673393 shutdown_connections 2023-10-17T01:14:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d9416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d93fb50 0x55ef5d93ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1479673393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d9404b0 0x55ef5d940cd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1479673393 >> 172.21.15.45:0/1479673393 conn(0x55ef5d89f080 msgr2=0x55ef5d93a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1479673393 shutdown_connections 2023-10-17T01:14:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.523+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1479673393 wait complete. 2023-10-17T01:14:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.524+0000 7f3091fdeb80 1 -- start start 2023-10-17T01:14:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.524+0000 7f3091fdeb80 1 Processor -- start 2023-10-17T01:14:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d949810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d949d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d950d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d82d4c0 con 0x55ef5d9404b0 2023-10-17T01:14:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d82d160 con 0x55ef5d941260 2023-10-17T01:14:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d7148d0 con 0x55ef5d93fb50 2023-10-17T01:14:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3091fdeb80 10 client.0 _ll_register_callbacks cb 0x55ef5d77aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d949810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d949810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57170/0 (socket says 172.21.15.45:57170) 2023-10-17T01:14:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f3077fff700 1 -- 172.21.15.45:0/751559966 learned_addr learned my addr 172.21.15.45:0/751559966 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f30777fe700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d949d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.525+0000 7f307cb9c700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d950d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3077fff700 1 -- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 msgr2=0x55ef5d950d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3077fff700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d950d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3077fff700 1 -- 172.21.15.45:0/751559966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 msgr2=0x55ef5d949d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3077fff700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d949d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3077fff700 1 -- 172.21.15.45:0/751559966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30781a1010 con 0x55ef5d93fb50 2023-10-17T01:14:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3077fff700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d949810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3068005bf0 tx=0x7f30680ba0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c0e381aa30d419 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f30577fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:14:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f30577fe700 1 -- 172.21.15.45:0/751559966 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306809af50 con 0x55ef5d93fb50 2023-10-17T01:14:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f30577fe700 1 -- 172.21.15.45:0/751559966 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f306809a850 con 0x55ef5d93fb50 2023-10-17T01:14:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.526+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef5d8bbcc0 con 0x55ef5d93fb50 2023-10-17T01:14:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f30577fe700 1 -- 172.21.15.45:0/751559966 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306809e680 con 0x55ef5d93fb50 2023-10-17T01:14:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 10 client.25009 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f306c017070 con 0x55ef5d93fb50 2023-10-17T01:14:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f30577fe700 1 -- 172.21.15.45:0/751559966 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef5d8bbcc0 con 0x55ef5d93fb50 2023-10-17T01:14:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f30577fe700 1 -- 172.21.15.45:0/751559966 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f306809aab0 con 0x55ef5d93fb50 2023-10-17T01:14:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 10 client.25009 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 10 client.25009 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-17T01:14:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 20 client.25009 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d954690 0x55ef5d953cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 4 client.25009 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.530+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 --> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef5d90ece0 con 0x55ef5d954690 2023-10-17T01:14:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.531+0000 7f30777fe700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d954690 0x55ef5d953cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.531+0000 7f30577fe700 10 client.25009 ms_handle_connect on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.531+0000 7f30777fe700 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d954690 0x55ef5d953cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f15d247a6e9b04a0 server_cookie=970c8420913d3960 in_seq=0 out_seq=0 2023-10-17T01:14:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f30577fe700 1 -- 172.21.15.45:0/751559966 <== mds.2 v2:172.21.15.60:6836/1246290891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ef5d90ece0 con 0x55ef5d954690 2023-10-17T01:14:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f30577fe700 10 client.25009 handle_command_reply: tid=0 2023-10-17T01:14:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f3091fdeb80 1 client.25009 shutdown 2023-10-17T01:14:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 msgr2=0x55ef5d949810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d949810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3068005bf0 tx=0x7f30680ba0c0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 shutdown_connections 2023-10-17T01:14:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.532+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d954690 0x55ef5d953cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d950d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d949810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/751559966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d949d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f30577fe700 0 client.25009 ms_handle_reset on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 >> 172.21.15.45:0/751559966 conn(0x55ef5d89f080 msgr2=0x55ef5d939fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 shutdown_connections 2023-10-17T01:14:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 1 -- 172.21.15.45:0/751559966 wait complete. 2023-10-17T01:14:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.533+0000 7f3091fdeb80 20 client.25009 trim_cache size 0 max 16384 2023-10-17T01:14:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.551+0000 7f3091fdeb80 1 Processor -- start 2023-10-17T01:14:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.551+0000 7f3091fdeb80 1 -- start start 2023-10-17T01:14:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d93b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d953cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d939fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d94a710 con 0x55ef5d9404b0 2023-10-17T01:14:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d947fb0 con 0x55ef5d941260 2023-10-17T01:14:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d90ed10 con 0x55ef5d93fb50 2023-10-17T01:14:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f30777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d953cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d93b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d93b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57186/0 (socket says 172.21.15.45:57186) 2023-10-17T01:14:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f30777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d953cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39508/0 (socket says 172.21.15.45:39508) 2023-10-17T01:14:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 -- 172.21.15.45:0/1556251959 learned_addr learned my addr 172.21.15.45:0/1556251959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f307cb9c700 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d939fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 -- 172.21.15.45:0/1556251959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 msgr2=0x55ef5d939fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d939fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 -- 172.21.15.45:0/1556251959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 msgr2=0x55ef5d953cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d953cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f3077fff700 1 -- 172.21.15.45:0/1556251959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef5d6a7c10 con 0x55ef5d93fb50 2023-10-17T01:14:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f307cb9c700 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d939fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.552+0000 7f30777fe700 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d953cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.553+0000 7f3077fff700 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d93b7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3068098ba0 tx=0x7f3068096cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7efbf6e6242f1ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.553+0000 7f305effd700 1 -- 172.21.15.45:0/1556251959 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068080070 con 0x55ef5d93fb50 2023-10-17T01:14:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.553+0000 7f305effd700 1 -- 172.21.15.45:0/1556251959 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3068099de0 con 0x55ef5d93fb50 2023-10-17T01:14:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.553+0000 7f305effd700 1 -- 172.21.15.45:0/1556251959 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30680a2830 con 0x55ef5d93fb50 2023-10-17T01:14:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1556251959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 msgr2=0x55ef5d93b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d93b7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3068098ba0 tx=0x7f3068096cc0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1556251959 shutdown_connections 2023-10-17T01:14:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d941260 0x55ef5d939fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d93b7b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/1556251959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d9404b0 0x55ef5d953cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1556251959 >> 172.21.15.45:0/1556251959 conn(0x55ef5d89f080 msgr2=0x55ef5d93a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1556251959 shutdown_connections 2023-10-17T01:14:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 -- 172.21.15.45:0/1556251959 wait complete. 2023-10-17T01:14:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.554+0000 7f3091fdeb80 1 -- start start 2023-10-17T01:14:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 Processor -- start 2023-10-17T01:14:07.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d958670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d9404b0 0x55ef5d958bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d941260 0x55ef5d9a6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d9592c0 con 0x55ef5d941260 2023-10-17T01:14:07.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d9a6540 con 0x55ef5d9404b0 2023-10-17T01:14:07.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5d9a6680 con 0x55ef5d93fb50 2023-10-17T01:14:07.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3091fdeb80 10 client.0 _ll_register_callbacks cb 0x7f306809a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:07.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d958670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3077fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d958670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:57188/0 (socket says 172.21.15.45:57188) 2023-10-17T01:14:07.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.555+0000 7f3077fff700 1 -- 172.21.15.45:0/683698944 learned_addr learned my addr 172.21.15.45:0/683698944 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f30777fe700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d9404b0 0x55ef5d958bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f307cb9c700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d941260 0x55ef5d9a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3077fff700 1 -- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d9404b0 msgr2=0x55ef5d958bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3077fff700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d9404b0 0x55ef5d958bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3077fff700 1 -- 172.21.15.45:0/683698944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d941260 msgr2=0x55ef5d9a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3077fff700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d941260 0x55ef5d9a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3077fff700 1 -- 172.21.15.45:0/683698944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c02a040 con 0x55ef5d93fb50 2023-10-17T01:14:07.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f30777fe700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d9404b0 0x55ef5d958bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:07.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3077fff700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d958670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f306800fa70 tx=0x7f30680054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265f3ae552aadfda server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f30577fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:14:07.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f30577fe700 1 -- 172.21.15.45:0/683698944 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068080030 con 0x55ef5d93fb50 2023-10-17T01:14:07.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f30577fe700 1 -- 172.21.15.45:0/683698944 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3068081070 con 0x55ef5d93fb50 2023-10-17T01:14:07.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.556+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f306c021070 con 0x55ef5d93fb50 2023-10-17T01:14:07.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f30577fe700 1 -- 172.21.15.45:0/683698944 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306809acc0 con 0x55ef5d93fb50 2023-10-17T01:14:07.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 10 client.25015 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:07.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f3070009350 con 0x55ef5d93fb50 2023-10-17T01:14:07.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f30577fe700 1 -- 172.21.15.45:0/683698944 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3068089080 con 0x55ef5d93fb50 2023-10-17T01:14:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f30577fe700 1 -- 172.21.15.45:0/683698944 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f30680b8bb0 con 0x55ef5d93fb50 2023-10-17T01:14:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 10 client.25015 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 10 client.25015 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-17T01:14:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 20 client.25015 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d9aaef0 0x55ef5d9a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 4 client.25015 mds_command: new command op to 24539 tid=0[{"prefix": "status"}] 2023-10-17T01:14:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.557+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 --> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ef5d889cf0 con 0x55ef5d9aaef0 2023-10-17T01:14:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f30777fe700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d9aaef0 0x55ef5d9a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f30777fe700 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d9aaef0 0x55ef5d9a78d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=969198dc87516b3a server_cookie=5a42557b08ec09bc in_seq=0 out_seq=0 2023-10-17T01:14:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f30577fe700 10 client.25015 ms_handle_connect on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:07.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f30577fe700 1 -- 172.21.15.45:0/683698944 <== mds.2 v2:172.21.15.60:6836/1246290891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ef5d889cf0 con 0x55ef5d9aaef0 2023-10-17T01:14:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f30577fe700 10 client.25015 handle_command_reply: tid=0 2023-10-17T01:14:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 client.25015 shutdown 2023-10-17T01:14:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 msgr2=0x55ef5d958670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d958670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f306800fa70 tx=0x7f30680054d0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 shutdown_connections 2023-10-17T01:14:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55ef5d9aaef0 0x55ef5d9a78d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef5d941260 0x55ef5d9a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef5d93fb50 0x55ef5d958670 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 --2- 172.21.15.45:0/683698944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef5d9404b0 0x55ef5d958bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f30577fe700 0 client.25015 ms_handle_reset on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.558+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 >> 172.21.15.45:0/683698944 conn(0x55ef5d89f080 msgr2=0x55ef5d953cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.559+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 shutdown_connections 2023-10-17T01:14:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.559+0000 7f3091fdeb80 1 -- 172.21.15.45:0/683698944 wait complete. 2023-10-17T01:14:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.559+0000 7f3091fdeb80 20 client.25015 trim_cache size 0 max 16384 2023-10-17T01:14:07.576 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:07.576 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:07.576 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2023-10-17T01:14:07.576 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24539, 2023-10-17T01:14:07.576 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:07.577 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:07.577 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:07.577 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 62.132604184000002, 2023-10-17T01:14:07.577 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:07.577 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:07.578 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:07.578 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 83.064967852999999 2023-10-17T01:14:07.578 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f30600a0c30 msgr2=0x7f30600a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f30600a0c30 0x7f30600a30f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f306c009c60 tx=0x7f306c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 msgr2=0x7f3078074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078074b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3068000c00 tx=0x7f306800b560 comp rx=0 tx=0).stop 2023-10-17T01:14:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 shutdown_connections 2023-10-17T01:14:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f30600a0c30 0x7f30600a30f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30781028d0 0x7f3078073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3078100d40 0x7f3078074b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 --2- 172.21.15.45:0/285264835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3078101f30 0x7f3078073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 >> 172.21.15.45:0/285264835 conn(0x7f30780fc290 msgr2=0x7f30780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 shutdown_connections 2023-10-17T01:14:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.560+0000 7f307e5ff700 1 -- 172.21.15.45:0/285264835 wait complete. 2023-10-17T01:14:07.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-17T01:14:07.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:07 smithi045 ceph-mon[98474]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:07.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:07 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3539982202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:07.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:07.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:07 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 -- 172.21.15.45:0/2137284472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0728100d40 msgr2=0x7f0728101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 --2- 172.21.15.45:0/2137284472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0728100d40 0x7f0728101160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f071c0099c0 tx=0x7f071c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 -- 172.21.15.45:0/2137284472 shutdown_connections 2023-10-17T01:14:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 --2- 172.21.15.45:0/2137284472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07281028d0 0x7f0728109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 --2- 172.21.15.45:0/2137284472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0728100d40 0x7f0728101160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 --2- 172.21.15.45:0/2137284472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 0x7f0728102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 -- 172.21.15.45:0/2137284472 >> 172.21.15.45:0/2137284472 conn(0x7f07280fc290 msgr2=0x7f07280fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 -- 172.21.15.45:0/2137284472 shutdown_connections 2023-10-17T01:14:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.792+0000 7f072ebff700 1 -- 172.21.15.45:0/2137284472 wait complete. 2023-10-17T01:14:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.793+0000 7f072ebff700 1 Processor -- start 2023-10-17T01:14:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.793+0000 7f072ebff700 1 -- start start 2023-10-17T01:14:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072ebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0728100d40 0x7f0728198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072ebff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 0x7f07281991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072ebff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07281028d0 0x7f07281a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072ebff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07281a0800 con 0x7f07281028d0 2023-10-17T01:14:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072ebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07281a0970 con 0x7f0728100d40 2023-10-17T01:14:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072ebff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07281a0b30 con 0x7f0728101f30 2023-10-17T01:14:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072c99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0728100d40 0x7f0728198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072d19c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07281028d0 0x7f07281a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072c99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0728100d40 0x7f0728198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47008/0 (socket says 172.21.15.45:47008) 2023-10-17T01:14:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072c99b700 1 -- 172.21.15.45:0/1142350182 learned_addr learned my addr 172.21.15.45:0/1142350182 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f0727fff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 0x7f07281991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072d19c700 1 -- 172.21.15.45:0/1142350182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 msgr2=0x7f07281991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072d19c700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 0x7f07281991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072d19c700 1 -- 172.21.15.45:0/1142350182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0728100d40 msgr2=0x7f0728198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072d19c700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0728100d40 0x7f0728198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f072d19c700 1 -- 172.21.15.45:0/1142350182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0095f0 con 0x7f07281028d0 2023-10-17T01:14:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.794+0000 7f0727fff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 0x7f07281991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.795+0000 7f072d19c700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07281028d0 0x7f07281a0230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f072000efc0 tx=0x7f072000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.795+0000 7f0725ffb700 1 -- 172.21.15.45:0/1142350182 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720018070 con 0x7f07281028d0 2023-10-17T01:14:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.795+0000 7f0725ffb700 1 -- 172.21.15.45:0/1142350182 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f072000f040 con 0x7f07281028d0 2023-10-17T01:14:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.795+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07281a0db0 con 0x7f07281028d0 2023-10-17T01:14:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.795+0000 7f0725ffb700 1 -- 172.21.15.45:0/1142350182 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072000cdf0 con 0x7f07281028d0 2023-10-17T01:14:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.795+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07281a1260 con 0x7f07281028d0 2023-10-17T01:14:07.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.798+0000 7f0725ffb700 1 -- 172.21.15.45:0/1142350182 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f0720003a00 con 0x7f07281028d0 2023-10-17T01:14:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.799+0000 7f0742590b80 1 Processor -- start 2023-10-17T01:14:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.799+0000 7f0742590b80 1 -- start start 2023-10-17T01:14:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.799+0000 7f0742590b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50514b50 0x560a50514f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.799+0000 7f0742590b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a505154b0 0x560a50515cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0742590b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a505166e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0725ffb700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07180a0bb0 0x7f07180a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0742590b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5046c9c0 con 0x560a505154b0 2023-10-17T01:14:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0742590b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5046a380 con 0x560a50514b50 2023-10-17T01:14:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0742590b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a50465c70 con 0x560a50516260 2023-10-17T01:14:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0725ffb700 1 -- 172.21.15.45:0/1142350182 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07200cbf50 con 0x7f07281028d0 2023-10-17T01:14:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f072c99b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a505154b0 0x560a50515cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a505166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a505166e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50540/0 (socket says 172.21.15.45:50540) 2023-10-17T01:14:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 -- 172.21.15.45:0/235623681 learned_addr learned my addr 172.21.15.45:0/235623681 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50514b50 0x560a50514f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f072d19c700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07180a0bb0 0x7f07180a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 -- 172.21.15.45:0/235623681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50514b50 msgr2=0x560a50514f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50514b50 0x560a50514f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 -- 172.21.15.45:0/235623681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a505154b0 msgr2=0x560a50515cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a505154b0 0x560a50515cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.800+0000 7f0727fff700 1 -- 172.21.15.45:0/235623681 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a50293d90 con 0x560a50516260 2023-10-17T01:14:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.801+0000 7f0727fff700 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a505166e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07140095f0 tx=0x7f07140097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2656a6976efc60 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.801+0000 7f0712ffd700 1 -- 172.21.15.45:0/235623681 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071400f450 con 0x560a50516260 2023-10-17T01:14:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.801+0000 7f0712ffd700 1 -- 172.21.15.45:0/235623681 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071400fa70 con 0x560a50516260 2023-10-17T01:14:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.801+0000 7f072d19c700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07180a0bb0 0x7f07180a3070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f072000c7a0 tx=0x7f072000cf50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.801+0000 7f0712ffd700 1 -- 172.21.15.45:0/235623681 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071401e7e0 con 0x560a50516260 2023-10-17T01:14:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 -- 172.21.15.45:0/235623681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 msgr2=0x560a505166e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a505166e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07140095f0 tx=0x7f07140097f0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 -- 172.21.15.45:0/235623681 shutdown_connections 2023-10-17T01:14:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a505166e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50514b50 0x560a50514f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 --2- 172.21.15.45:0/235623681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a505154b0 0x560a50515cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 -- 172.21.15.45:0/235623681 >> 172.21.15.45:0/235623681 conn(0x560a50474080 msgr2=0x560a5050f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 -- 172.21.15.45:0/235623681 shutdown_connections 2023-10-17T01:14:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.802+0000 7f0742590b80 1 -- 172.21.15.45:0/235623681 wait complete. 2023-10-17T01:14:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 -- start start 2023-10-17T01:14:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 Processor -- start 2023-10-17T01:14:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5051e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a505154b0 0x560a5051ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a50525d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a504024c0 con 0x560a50514b50 2023-10-17T01:14:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a50402160 con 0x560a505154b0 2023-10-17T01:14:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a502e98d0 con 0x560a50516260 2023-10-17T01:14:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.803+0000 7f0742590b80 10 client.0 _ll_register_callbacks cb 0x560a5034fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5051e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5051e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49242/0 (socket says 172.21.15.45:49242) 2023-10-17T01:14:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 -- 172.21.15.45:0/1857901483 learned_addr learned my addr 172.21.15.45:0/1857901483 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f072c99b700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a505154b0 0x560a5051ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a50525d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 -- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 msgr2=0x560a50525d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a50525d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 -- 172.21.15.45:0/1857901483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a505154b0 msgr2=0x560a5051ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a505154b0 0x560a5051ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 -- 172.21.15.45:0/1857901483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071401a040 con 0x560a50514b50 2023-10-17T01:14:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0727fff700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5051e7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f071400ffa0 tx=0x7f0714009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaca7dd47ceed94e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0707fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.804+0000 7f0707fff700 1 -- 172.21.15.45:0/1857901483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714005020 con 0x560a50514b50 2023-10-17T01:14:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0707fff700 1 -- 172.21.15.45:0/1857901483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0714000e40 con 0x560a50514b50 2023-10-17T01:14:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a50490cc0 con 0x560a50514b50 2023-10-17T01:14:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0707fff700 1 -- 172.21.15.45:0/1857901483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0714005180 con 0x560a50514b50 2023-10-17T01:14:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 10 client.15006 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:07.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560a5027cc10 con 0x560a50514b50 2023-10-17T01:14:07.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0707fff700 1 -- 172.21.15.45:0/1857901483 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a50490cc0 con 0x560a50514b50 2023-10-17T01:14:07.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0707fff700 1 -- 172.21.15.45:0/1857901483 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f071401e5e0 con 0x560a50514b50 2023-10-17T01:14:07.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:07.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 10 client.15006 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-17T01:14:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 20 client.15006 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505298f0 0x560a50528d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 4 client.15006 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.805+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 --> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a504e3ce0 con 0x560a505298f0 2023-10-17T01:14:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.806+0000 7f072c99b700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505298f0 0x560a50528d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.806+0000 7f072c99b700 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505298f0 0x560a50528d50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79bb010ce199b9de server_cookie=82eb15a024077215 in_seq=0 out_seq=0 2023-10-17T01:14:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.806+0000 7f0707fff700 10 client.15006 ms_handle_connect on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0707fff700 1 -- 172.21.15.45:0/1857901483 <== mds.1 v2:172.21.15.181:6838/1273587217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x560a504e3ce0 con 0x560a505298f0 2023-10-17T01:14:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0707fff700 10 client.15006 handle_command_reply: tid=0 2023-10-17T01:14:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 client.15006 shutdown 2023-10-17T01:14:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 msgr2=0x560a5051e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5051e7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f071400ffa0 tx=0x7f0714009e60 comp rx=0 tx=0).stop 2023-10-17T01:14:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 shutdown_connections 2023-10-17T01:14:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a50516260 0x560a50525d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505298f0 0x560a50528d50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5051e7e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 --2- 172.21.15.45:0/1857901483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a505154b0 0x560a5051ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 >> 172.21.15.45:0/1857901483 conn(0x560a50474080 msgr2=0x560a5050efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0707fff700 0 client.15006 ms_handle_reset on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.807+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 shutdown_connections 2023-10-17T01:14:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.808+0000 7f0742590b80 1 -- 172.21.15.45:0/1857901483 wait complete. 2023-10-17T01:14:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.808+0000 7f0742590b80 20 client.15006 trim_cache size 0 max 16384 2023-10-17T01:14:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 Processor -- start 2023-10-17T01:14:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 -- start start 2023-10-17T01:14:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a50526590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a50528d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5051c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a504e3ce0 con 0x560a50514b50 2023-10-17T01:14:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a50490cc0 con 0x560a50516260 2023-10-17T01:14:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.826+0000 7f0742590b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5040f830 con 0x560a505154b0 2023-10-17T01:14:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:07 smithi060 ceph-mon[134136]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:07.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:07 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3539982202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:07.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:07.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:07 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a50526590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a50526590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49258/0 (socket says 172.21.15.45:49258) 2023-10-17T01:14:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 -- 172.21.15.45:0/338517306 learned_addr learned my addr 172.21.15.45:0/338517306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f072c99b700 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a50528d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5051c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 -- 172.21.15.45:0/338517306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 msgr2=0x560a50528d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a50528d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 -- 172.21.15.45:0/338517306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 msgr2=0x560a5051c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5051c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 -- 172.21.15.45:0/338517306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a502ba200 con 0x560a50514b50 2023-10-17T01:14:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.827+0000 7f0727fff700 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a50526590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0714009640 tx=0x7f0714000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab4564c204a8512 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.828+0000 7f07137fe700 1 -- 172.21.15.45:0/338517306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140292f0 con 0x560a50514b50 2023-10-17T01:14:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.828+0000 7f07137fe700 1 -- 172.21.15.45:0/338517306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0714009730 con 0x560a50514b50 2023-10-17T01:14:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.828+0000 7f07137fe700 1 -- 172.21.15.45:0/338517306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714029900 con 0x560a50514b50 2023-10-17T01:14:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.828+0000 7f0742590b80 1 -- 172.21.15.45:0/338517306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 msgr2=0x560a50526590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.828+0000 7f0742590b80 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a50526590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0714009640 tx=0x7f0714000bc0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 -- 172.21.15.45:0/338517306 shutdown_connections 2023-10-17T01:14:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5051c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a50526590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 --2- 172.21.15.45:0/338517306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a50528d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 -- 172.21.15.45:0/338517306 >> 172.21.15.45:0/338517306 conn(0x560a50474080 msgr2=0x560a50510600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 -- 172.21.15.45:0/338517306 shutdown_connections 2023-10-17T01:14:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 -- 172.21.15.45:0/338517306 wait complete. 2023-10-17T01:14:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.829+0000 7f0742590b80 1 -- start start 2023-10-17T01:14:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 Processor -- start 2023-10-17T01:14:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5052d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a5052ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5057b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a505327a0 con 0x560a50514b50 2023-10-17T01:14:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a50532910 con 0x560a50516260 2023-10-17T01:14:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5057b760 con 0x560a505154b0 2023-10-17T01:14:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0742590b80 10 client.0 _ll_register_callbacks cb 0x7f07200097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0727fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5052d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f072c99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a5052ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0727fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5052d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49266/0 (socket says 172.21.15.45:49266) 2023-10-17T01:14:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f0727fff700 1 -- 172.21.15.45:0/3344735883 learned_addr learned my addr 172.21.15.45:0/3344735883 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.830+0000 7f072c99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a5052ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50556/0 (socket says 172.21.15.45:50556) 2023-10-17T01:14:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f0727fff700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5057b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f072c99b700 1 -- 172.21.15.45:0/3344735883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 msgr2=0x560a5057b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f072c99b700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5057b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f072c99b700 1 -- 172.21.15.45:0/3344735883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 msgr2=0x560a5052d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f072c99b700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5052d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f072c99b700 1 -- 172.21.15.45:0/3344735883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a5027cc10 con 0x560a505154b0 2023-10-17T01:14:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f0727fff700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5052d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f072c99b700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a5052ddd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f071c00b580 tx=0x7f071c00bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8a8a3850c8a3c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f0707fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:14:07.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f0707fff700 1 -- 172.21.15.45:0/3344735883 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c02b8c0 con 0x560a505154b0 2023-10-17T01:14:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f071400fa70 con 0x560a505154b0 2023-10-17T01:14:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.831+0000 7f0707fff700 1 -- 172.21.15.45:0/3344735883 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f071c02ba20 con 0x560a505154b0 2023-10-17T01:14:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0707fff700 1 -- 172.21.15.45:0/3344735883 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f071c02bbe0 con 0x560a505154b0 2023-10-17T01:14:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 10 client.25033 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560a50292260 con 0x560a505154b0 2023-10-17T01:14:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0707fff700 1 -- 172.21.15.45:0/3344735883 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071400fa70 con 0x560a505154b0 2023-10-17T01:14:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0707fff700 1 -- 172.21.15.45:0/3344735883 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f071c0099f0 con 0x560a505154b0 2023-10-17T01:14:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 10 client.25033 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 10 client.25033 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-17T01:14:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 20 client.25033 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:07.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505804d0 0x560a5057ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 4 client.25033 mds_command: new command op to 24550 tid=0[{"prefix": "status"}] 2023-10-17T01:14:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.832+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 --> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a5045ecf0 con 0x560a505804d0 2023-10-17T01:14:07.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.833+0000 7f0727fff700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505804d0 0x560a5057ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:07.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.833+0000 7f0707fff700 10 client.25033 ms_handle_connect on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.833+0000 7f0727fff700 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505804d0 0x560a5057ce20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af24053e0f1afec3 server_cookie=969bdeb0cc055832 in_seq=0 out_seq=0 2023-10-17T01:14:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.833+0000 7f0707fff700 1 -- 172.21.15.45:0/3344735883 <== mds.1 v2:172.21.15.181:6838/1273587217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560a5045ecf0 con 0x560a505804d0 2023-10-17T01:14:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.833+0000 7f0707fff700 10 client.25033 handle_command_reply: tid=0 2023-10-17T01:14:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.833+0000 7f0742590b80 1 client.25033 shutdown 2023-10-17T01:14:07.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 msgr2=0x560a5052ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a5052ddd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f071c00b580 tx=0x7f071c00bb10 comp rx=0 tx=0).stop 2023-10-17T01:14:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 shutdown_connections 2023-10-17T01:14:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a50516260 0x560a5057b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x560a505804d0 0x560a5057ce20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a50514b50 0x560a5052d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 --2- 172.21.15.45:0/3344735883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a505154b0 0x560a5052ddd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 >> 172.21.15.45:0/3344735883 conn(0x560a50474080 msgr2=0x560a50526590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0707fff700 0 client.25033 ms_handle_reset on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 shutdown_connections 2023-10-17T01:14:07.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 1 -- 172.21.15.45:0/3344735883 wait complete. 2023-10-17T01:14:07.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.834+0000 7f0742590b80 20 client.25033 trim_cache size 0 max 16384 2023-10-17T01:14:07.850 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:07.850 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:07.850 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2023-10-17T01:14:07.851 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24550, 2023-10-17T01:14:07.851 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:07.851 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:07.851 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:07.851 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 63.421737356000001, 2023-10-17T01:14:07.852 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:07.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:07.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:07.852 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 81.192194916999995 2023-10-17T01:14:07.852 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:07.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07180a0bb0 msgr2=0x7f07180a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07180a0bb0 0x7f07180a3070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f072000c7a0 tx=0x7f072000cf50 comp rx=0 tx=0).stop 2023-10-17T01:14:07.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07281028d0 msgr2=0x7f07281a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:07.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07281028d0 0x7f07281a0230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f072000efc0 tx=0x7f072000c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 shutdown_connections 2023-10-17T01:14:07.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07180a0bb0 0x7f07180a3070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07281028d0 0x7f07281a0230 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0728100d40 0x7f0728198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 --2- 172.21.15.45:0/1142350182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0728101f30 0x7f07281991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:07.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 >> 172.21.15.45:0/1142350182 conn(0x7f07280fc290 msgr2=0x7f07280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:07.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 shutdown_connections 2023-10-17T01:14:07.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:07.835+0000 7f072ebff700 1 -- 172.21.15.45:0/1142350182 wait complete. 2023-10-17T01:14:07.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-17T01:14:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:07 smithi181 ceph-mon[134520]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:07 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3539982202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:07 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 -- 172.21.15.45:0/2145050286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc074d90 msgr2=0x7f11bc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 --2- 172.21.15.45:0/2145050286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc074d90 0x7f11bc0731e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009990 tx=0x7f11ac009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 -- 172.21.15.45:0/2145050286 shutdown_connections 2023-10-17T01:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 --2- 172.21.15.45:0/2145050286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 0x7f11bc10b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 --2- 172.21.15.45:0/2145050286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc074d90 0x7f11bc0731e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 --2- 172.21.15.45:0/2145050286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11bc0737b0 0x7f11bc073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 -- 172.21.15.45:0/2145050286 >> 172.21.15.45:0/2145050286 conn(0x7f11bc0fc2d0 msgr2=0x7f11bc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 -- 172.21.15.45:0/2145050286 shutdown_connections 2023-10-17T01:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.064+0000 7f11c21ff700 1 -- 172.21.15.45:0/2145050286 wait complete. 2023-10-17T01:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.065+0000 7f11c21ff700 1 Processor -- start 2023-10-17T01:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.065+0000 7f11c21ff700 1 -- start start 2023-10-17T01:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11c21ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 0x7f11bc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11c21ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11bc074d90 0x7f11bc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11c21ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 0x7f11bc1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11c21ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc1a0800 con 0x7f11bc0737b0 2023-10-17T01:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11c21ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc1a0970 con 0x7f11bc108c20 2023-10-17T01:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11c21ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc1a0b30 con 0x7f11bc074d90 2023-10-17T01:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 0x7f11bc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 0x7f11bc198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49296/0 (socket says 172.21.15.45:49296) 2023-10-17T01:14:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 -- 172.21.15.45:0/4284654092 learned_addr learned my addr 172.21.15.45:0/4284654092 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11baffd700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11bc074d90 0x7f11bc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bbfff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 0x7f11bc1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 -- 172.21.15.45:0/4284654092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11bc074d90 msgr2=0x7f11bc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11bc074d90 0x7f11bc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 -- 172.21.15.45:0/4284654092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 msgr2=0x7f11bc1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 0x7f11bc1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bb7fe700 1 -- 172.21.15.45:0/4284654092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0095f0 con 0x7f11bc0737b0 2023-10-17T01:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.066+0000 7f11bbfff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 0x7f11bc1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.067+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 0x7f11bc198c90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00b680 tx=0x7f11ac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.067+0000 7f11b8ff9700 1 -- 172.21.15.45:0/4284654092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac01d070 con 0x7f11bc0737b0 2023-10-17T01:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.067+0000 7f11b8ff9700 1 -- 172.21.15.45:0/4284654092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ac022470 con 0x7f11bc0737b0 2023-10-17T01:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.067+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc1a0db0 con 0x7f11bc0737b0 2023-10-17T01:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.067+0000 7f11b8ff9700 1 -- 172.21.15.45:0/4284654092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac017650 con 0x7f11bc0737b0 2023-10-17T01:14:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.067+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc1a1180 con 0x7f11bc0737b0 2023-10-17T01:14:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.070+0000 7f11b8ff9700 1 -- 172.21.15.45:0/4284654092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f11ac0225e0 con 0x7f11bc0737b0 2023-10-17T01:14:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.071+0000 7f11d5a38b80 1 Processor -- start 2023-10-17T01:14:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11b8ff9700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f11a40a0c30 0x7f11a40a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 -- start start 2023-10-17T01:14:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11b8ff9700 1 -- 172.21.15.45:0/4284654092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f11ac0cd7e0 con 0x7f11bc0737b0 2023-10-17T01:14:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11bbfff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f11a40a0c30 0x7f11a40a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161bf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161bf9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161bfa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161b509c0 con 0x556161bf8b50 2023-10-17T01:14:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161b4e380 con 0x556161bfa260 2023-10-17T01:14:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161b49c70 con 0x556161bf94b0 2023-10-17T01:14:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161bf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.072+0000 7f11baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161bf9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161bf9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50594/0 (socket says 172.21.15.45:50594) 2023-10-17T01:14:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11baffd700 1 -- 172.21.15.45:0/791697736 learned_addr learned my addr 172.21.15.45:0/791697736 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161bf8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49304/0 (socket says 172.21.15.45:49304) 2023-10-17T01:14:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bbfff700 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161bfa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bbfff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f11a40a0c30 0x7f11a40a30f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009400 tx=0x7f11b4009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 -- 172.21.15.45:0/791697736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 msgr2=0x556161bf9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161bf9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 -- 172.21.15.45:0/791697736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 msgr2=0x556161bfa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161bfa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 -- 172.21.15.45:0/791697736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556161977d90 con 0x556161bf8b50 2023-10-17T01:14:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161bf8f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cb160 tx=0x7f11ac09a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff8fd9267bd0f5c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11a1ffb700 1 -- 172.21.15.45:0/791697736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac09ad30 con 0x556161bf8b50 2023-10-17T01:14:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11a1ffb700 1 -- 172.21.15.45:0/791697736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ac09eb60 con 0x556161bf8b50 2023-10-17T01:14:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.073+0000 7f11a1ffb700 1 -- 172.21.15.45:0/791697736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac09a220 con 0x556161bf8b50 2023-10-17T01:14:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 -- 172.21.15.45:0/791697736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 msgr2=0x556161bf8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161bf8f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cb160 tx=0x7f11ac09a940 comp rx=0 tx=0).stop 2023-10-17T01:14:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 -- 172.21.15.45:0/791697736 shutdown_connections 2023-10-17T01:14:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161bfa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161bf8f70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/791697736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161bf9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 -- 172.21.15.45:0/791697736 >> 172.21.15.45:0/791697736 conn(0x556161b58080 msgr2=0x556161bf3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:08.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 -- 172.21.15.45:0/791697736 shutdown_connections 2023-10-17T01:14:08.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.074+0000 7f11d5a38b80 1 -- 172.21.15.45:0/791697736 wait complete. 2023-10-17T01:14:08.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 -- start start 2023-10-17T01:14:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 Processor -- start 2023-10-17T01:14:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf8b50 0x556161c02500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 0x556161c02a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c09a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161ae64c0 con 0x556161bfa260 2023-10-17T01:14:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161ae6160 con 0x556161bf94b0 2023-10-17T01:14:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.075+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561619cd8d0 con 0x556161bf8b50 2023-10-17T01:14:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11d5a38b80 10 client.0 _ll_register_callbacks cb 0x556161a33a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 0x556161c02a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 0x556161c02a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:57572/0 (socket says 172.21.15.45:57572) 2023-10-17T01:14:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 -- 172.21.15.45:0/2915695306 learned_addr learned my addr 172.21.15.45:0/2915695306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11bbfff700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c09a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf8b50 0x556161c02500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 -- 172.21.15.45:0/2915695306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf8b50 msgr2=0x556161c02500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf8b50 0x556161c02500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 -- 172.21.15.45:0/2915695306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 msgr2=0x556161c09a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c09a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 -- 172.21.15.45:0/2915695306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc1a1180 con 0x556161bf94b0 2023-10-17T01:14:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f11baffd700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 0x556161c02a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f11b00126d0 tx=0x7f11b00129e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ebd6d9f91db53e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.076+0000 7f119affd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f119affd700 1 -- 172.21.15.45:0/2915695306 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0017070 con 0x556161bf94b0 2023-10-17T01:14:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556161b74cc0 con 0x556161bf94b0 2023-10-17T01:14:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f119affd700 1 -- 172.21.15.45:0/2915695306 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f11b0004050 con 0x556161bf94b0 2023-10-17T01:14:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f119affd700 1 -- 172.21.15.45:0/2915695306 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b0004210 con 0x556161bf94b0 2023-10-17T01:14:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f11d5a38b80 10 client.25043 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556161960c10 con 0x556161bf94b0 2023-10-17T01:14:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f119affd700 1 -- 172.21.15.45:0/2915695306 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556161b74cc0 con 0x556161bf94b0 2023-10-17T01:14:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.077+0000 7f119affd700 1 -- 172.21.15.45:0/2915695306 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f11b00104c0 con 0x556161bf94b0 2023-10-17T01:14:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11d5a38b80 10 client.25043 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11d5a38b80 10 client.25043 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-17T01:14:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11d5a38b80 20 client.25043 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c0d7e0 0x556161c0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11d5a38b80 4 client.25043 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556161bc7ce0 con 0x556161c0d7e0 2023-10-17T01:14:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c0d7e0 0x556161c0cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c0d7e0 0x556161c0cbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751f1ea647afd003 server_cookie=b36bd9e08dada1a0 in_seq=0 out_seq=0 2023-10-17T01:14:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.078+0000 7f119affd700 10 client.25043 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.079+0000 7f119affd700 1 -- 172.21.15.45:0/2915695306 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x556161bc7ce0 con 0x556161c0d7e0 2023-10-17T01:14:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.079+0000 7f119affd700 10 client.25043 handle_command_reply: tid=0 2023-10-17T01:14:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.079+0000 7f11d5a38b80 1 client.25043 shutdown 2023-10-17T01:14:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 msgr2=0x556161c02a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 0x556161c02a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f11b00126d0 tx=0x7f11b00129e0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 shutdown_connections 2023-10-17T01:14:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c09a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf8b50 0x556161c02500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c0d7e0 0x556161c0cbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/2915695306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf94b0 0x556161c02a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 >> 172.21.15.45:0/2915695306 conn(0x556161b58080 msgr2=0x556161bf2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f119affd700 0 client.25043 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 shutdown_connections 2023-10-17T01:14:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 1 -- 172.21.15.45:0/2915695306 wait complete. 2023-10-17T01:14:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.080+0000 7f11d5a38b80 20 client.25043 trim_cache size 0 max 16384 2023-10-17T01:14:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 Processor -- start 2023-10-17T01:14:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 -- start start 2023-10-17T01:14:08.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 0x556161bf2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c020d0 con 0x556161bfa260 2023-10-17T01:14:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161bc7d10 con 0x556161bf8b50 2023-10-17T01:14:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161bf4850 con 0x556161bf94b0 2023-10-17T01:14:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.098+0000 7f11bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c0d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 0x556161bf2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c0d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49316/0 (socket says 172.21.15.45:49316) 2023-10-17T01:14:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 0x556161bf2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:57576/0 (socket says 172.21.15.45:57576) 2023-10-17T01:14:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bbfff700 1 -- 172.21.15.45:0/1672022532 learned_addr learned my addr 172.21.15.45:0/1672022532 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11baffd700 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c0cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 -- 172.21.15.45:0/1672022532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 msgr2=0x556161c0cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c0cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 -- 172.21.15.45:0/1672022532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 msgr2=0x556161c0d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c0d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 -- 172.21.15.45:0/1672022532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55616199e200 con 0x556161bf8b50 2023-10-17T01:14:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11baffd700 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:08.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 0x556161bf2fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11ac006010 tx=0x7f11ac009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa3d812dcf193f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:08.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11a27fc700 1 -- 172.21.15.45:0/1672022532 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0b3240 con 0x556161bf8b50 2023-10-17T01:14:08.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11a27fc700 1 -- 172.21.15.45:0/1672022532 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ac0a2d10 con 0x556161bf8b50 2023-10-17T01:14:08.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.099+0000 7f11a27fc700 1 -- 172.21.15.45:0/1672022532 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac09b260 con 0x556161bf8b50 2023-10-17T01:14:08.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1672022532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 msgr2=0x556161bf2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 0x556161bf2fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11ac006010 tx=0x7f11ac009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1672022532 shutdown_connections 2023-10-17T01:14:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bfa260 0x556161c0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bf8b50 0x556161bf2fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1672022532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1672022532 >> 172.21.15.45:0/1672022532 conn(0x556161b58080 msgr2=0x556161bf3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:08.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1672022532 shutdown_connections 2023-10-17T01:14:08.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.100+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1672022532 wait complete. 2023-10-17T01:14:08.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.101+0000 7f11d5a38b80 1 -- start start 2023-10-17T01:14:08.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.101+0000 7f11d5a38b80 1 Processor -- start 2023-10-17T01:14:08.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161c10d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161c16110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c16650 con 0x556161bf8b50 2023-10-17T01:14:08.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c167c0 con 0x556161bfa260 2023-10-17T01:14:08.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556161c5f130 con 0x556161bf94b0 2023-10-17T01:14:08.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11d5a38b80 10 client.0 _ll_register_callbacks cb 0x7f11b0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:08.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c11270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50626/0 (socket says 172.21.15.45:50626) 2023-10-17T01:14:08.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 -- 172.21.15.45:0/1069456766 learned_addr learned my addr 172.21.15.45:0/1069456766 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:08.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161c10d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11bbfff700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161c16110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 -- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 msgr2=0x556161c16110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161c16110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 -- 172.21.15.45:0/1069456766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 msgr2=0x556161c10d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161c10d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.102+0000 7f11baffd700 1 -- 172.21.15.45:0/1069456766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556161960c10 con 0x556161bf94b0 2023-10-17T01:14:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f11baffd700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c11270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009e60 tx=0x7f11b0009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7646e5b47a4c83 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f119affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:14:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f119affd700 1 -- 172.21.15.45:0/1069456766 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0017070 con 0x556161bf94b0 2023-10-17T01:14:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f119affd700 1 -- 172.21.15.45:0/1069456766 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f11b0010430 con 0x556161bf94b0 2023-10-17T01:14:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11ac09eb60 con 0x556161bf94b0 2023-10-17T01:14:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f119affd700 1 -- 172.21.15.45:0/1069456766 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b002d070 con 0x556161bf94b0 2023-10-17T01:14:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f11d5a38b80 10 client.25051 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f11ac0b2040 con 0x556161bf94b0 2023-10-17T01:14:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.103+0000 7f119affd700 1 -- 172.21.15.45:0/1069456766 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ac09eb60 con 0x556161bf94b0 2023-10-17T01:14:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f119affd700 1 -- 172.21.15.45:0/1069456766 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f11b0003840 con 0x556161bf94b0 2023-10-17T01:14:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11d5a38b80 10 client.25051 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11d5a38b80 10 client.25051 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-17T01:14:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11d5a38b80 20 client.25051 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c63dd0 0x556161c60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11d5a38b80 4 client.25051 mds_command: new command op to 24587 tid=0[{"prefix": "status"}] 2023-10-17T01:14:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556161b42cf0 con 0x556161c63dd0 2023-10-17T01:14:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.104+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c63dd0 0x556161c60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11bb7fe700 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c63dd0 0x556161c60760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b196e0520a69910e server_cookie=86bb12932c96ed3c in_seq=0 out_seq=0 2023-10-17T01:14:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f119affd700 10 client.25051 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f119affd700 1 -- 172.21.15.45:0/1069456766 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556161b42cf0 con 0x556161c63dd0 2023-10-17T01:14:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f119affd700 10 client.25051 handle_command_reply: tid=0 2023-10-17T01:14:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 client.25051 shutdown 2023-10-17T01:14:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 msgr2=0x556161c11270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c11270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009e60 tx=0x7f11b0009470 comp rx=0 tx=0).stop 2023-10-17T01:14:08.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 shutdown_connections 2023-10-17T01:14:08.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556161c63dd0 0x556161c60760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556161bfa260 0x556161c16110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556161bf8b50 0x556161c10d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 --2- 172.21.15.45:0/1069456766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556161bf94b0 0x556161c11270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 >> 172.21.15.45:0/1069456766 conn(0x556161b58080 msgr2=0x556161c00600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:08.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f119affd700 0 client.25051 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 shutdown_connections 2023-10-17T01:14:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 1 -- 172.21.15.45:0/1069456766 wait complete. 2023-10-17T01:14:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.105+0000 7f11d5a38b80 20 client.25051 trim_cache size 0 max 16384 2023-10-17T01:14:08.132 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:08.133 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:08.133 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-17T01:14:08.133 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24587, 2023-10-17T01:14:08.133 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:08.134 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:08.134 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:08.134 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 67.064593185000007, 2023-10-17T01:14:08.134 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:08.135 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:08.135 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:08.135 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 72.780670634000003 2023-10-17T01:14:08.135 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f11a40a0c30 msgr2=0x7f11a40a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f11a40a0c30 0x7f11a40a30f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009400 tx=0x7f11b4009920 comp rx=0 tx=0).stop 2023-10-17T01:14:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 msgr2=0x7f11bc198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 0x7f11bc198c90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00b680 tx=0x7f11ac005740 comp rx=0 tx=0).stop 2023-10-17T01:14:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 shutdown_connections 2023-10-17T01:14:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f11a40a0c30 0x7f11a40a30f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11bc108c20 0x7f11bc1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11bc0737b0 0x7f11bc198c90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 --2- 172.21.15.45:0/4284654092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11bc074d90 0x7f11bc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:08.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 >> 172.21.15.45:0/4284654092 conn(0x7f11bc0fc2d0 msgr2=0x7f11bc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:08.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 shutdown_connections 2023-10-17T01:14:08.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:08.106+0000 7f11c21ff700 1 -- 172.21.15.45:0/4284654092 wait complete. 2023-10-17T01:14:08.140 INFO:tasks.cephadm:Setup complete, yielding 2023-10-17T01:14:08.140 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-17T01:14:08.150 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:14:08.151 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-17T01:14:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:14:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:14:08.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:14:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:14:08.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:14:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:14:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:14:09.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:09 smithi045 ceph-mon[98474]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:09 smithi060 ceph-mon[134136]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:09 smithi181 ceph-mon[134520]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:10.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 -- 172.21.15.45:0/4183904822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 msgr2=0x7f60d8073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:10.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4183904822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8073c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009990 tx=0x7f60c8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 -- 172.21.15.45:0/4183904822 shutdown_connections 2023-10-17T01:14:10.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4183904822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8108cb0 0x7f60d810b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4183904822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60d8074d80 0x7f60d80731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4183904822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8073c30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 -- 172.21.15.45:0/4183904822 >> 172.21.15.45:0/4183904822 conn(0x7f60d80fc340 msgr2=0x7f60d80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:10.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 -- 172.21.15.45:0/4183904822 shutdown_connections 2023-10-17T01:14:10.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.312+0000 7f60e07b2700 1 -- 172.21.15.45:0/4183904822 wait complete. 2023-10-17T01:14:10.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.313+0000 7f60e07b2700 1 Processor -- start 2023-10-17T01:14:10.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 -- start start 2023-10-17T01:14:10.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:10.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8074d80 0x7f60d8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:10.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60d8108cb0 0x7f60d81a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:10.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a0890 con 0x7f60d80737b0 2023-10-17T01:14:10.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a09d0 con 0x7f60d8108cb0 2023-10-17T01:14:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60e07b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81a0b90 con 0x7f60d8074d80 2023-10-17T01:14:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60de54e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60de54e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49346/0 (socket says 172.21.15.45:49346) 2023-10-17T01:14:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60de54e700 1 -- 172.21.15.45:0/4280792354 learned_addr learned my addr 172.21.15.45:0/4280792354 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60ddd4d700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8074d80 0x7f60d8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.314+0000 7f60ded4f700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60d8108cb0 0x7f60d81a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60de54e700 1 -- 172.21.15.45:0/4280792354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8074d80 msgr2=0x7f60d8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60de54e700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8074d80 0x7f60d8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60de54e700 1 -- 172.21.15.45:0/4280792354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60d8108cb0 msgr2=0x7f60d81a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60de54e700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60d8108cb0 0x7f60d81a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60de54e700 1 -- 172.21.15.45:0/4280792354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80095f0 con 0x7f60d80737b0 2023-10-17T01:14:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60de54e700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8198d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60d400e970 tx=0x7f60d400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4009790 con 0x7f60d80737b0 2023-10-17T01:14:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d400c7e0 con 0x7f60d80737b0 2023-10-17T01:14:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d81a0e70 con 0x7f60d80737b0 2023-10-17T01:14:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40054e0 con 0x7f60d80737b0 2023-10-17T01:14:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.315+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81a1290 con 0x7f60d80737b0 2023-10-17T01:14:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.316+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8107360 con 0x7f60d80737b0 2023-10-17T01:14:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.318+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f60d400c950 con 0x7f60d80737b0 2023-10-17T01:14:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.320+0000 7f60cf7fe700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60c40a0c30 0x7f60c40a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.320+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f60d4014070 con 0x7f60d80737b0 2023-10-17T01:14:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.320+0000 7f60ddd4d700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60c40a0c30 0x7f60c40a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.321+0000 7f60ddd4d700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60c40a0c30 0x7f60c40a30f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f60c800b2e0 tx=0x7f60c8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.324+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f60d4095ce0 con 0x7f60d80737b0 2023-10-17T01:14:10.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.551+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d8061190 con 0x7f60c40a0c30 2023-10-17T01:14:10.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.552+0000 7f60cf7fe700 1 -- 172.21.15.45:0/4280792354 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f60d8061190 con 0x7f60c40a0c30 2023-10-17T01:14:10.553 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2023-10-17T01:14:10.554 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2023-10-17T01:14:10.554 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2023-10-17T01:14:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.554+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60c40a0c30 msgr2=0x7f60c40a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.554+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60c40a0c30 0x7f60c40a30f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f60c800b2e0 tx=0x7f60c8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.554+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 msgr2=0x7f60d8198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.554+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8198d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60d400e970 tx=0x7f60d400ec80 comp rx=0 tx=0).stop 2023-10-17T01:14:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 shutdown_connections 2023-10-17T01:14:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f60c40a0c30 0x7f60c40a30f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60d8108cb0 0x7f60d81a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60d80737b0 0x7f60d8198d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 --2- 172.21.15.45:0/4280792354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d8074d80 0x7f60d8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:10.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 >> 172.21.15.45:0/4280792354 conn(0x7f60d80fc340 msgr2=0x7f60d80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:10.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 shutdown_connections 2023-10-17T01:14:10.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:10.555+0000 7f60e07b2700 1 -- 172.21.15.45:0/4280792354 wait complete. 2023-10-17T01:14:11.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-17T01:14:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:11 smithi045 ceph-mon[98474]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:11 smithi060 ceph-mon[134136]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:11 smithi181 ceph-mon[134520]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:12.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:12 smithi045 ceph-mon[98474]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:12 smithi060 ceph-mon[134136]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:12.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:12 smithi181 ceph-mon[134520]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:13.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 -- 172.21.15.45:0/3588587060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 msgr2=0x7fca20102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:13.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 --2- 172.21.15.45:0/3588587060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20102440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca100099c0 tx=0x7fca10009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 -- 172.21.15.45:0/3588587060 shutdown_connections 2023-10-17T01:14:13.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 --2- 172.21.15.45:0/3588587060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20102980 0x7fca20109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 --2- 172.21.15.45:0/3588587060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20100de0 0x7fca20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 --2- 172.21.15.45:0/3588587060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20102440 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 -- 172.21.15.45:0/3588587060 >> 172.21.15.45:0/3588587060 conn(0x7fca200fc340 msgr2=0x7fca200fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 -- 172.21.15.45:0/3588587060 shutdown_connections 2023-10-17T01:14:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.353+0000 7fca28142700 1 -- 172.21.15.45:0/3588587060 wait complete. 2023-10-17T01:14:13.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.354+0000 7fca28142700 1 Processor -- start 2023-10-17T01:14:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.354+0000 7fca28142700 1 -- start start 2023-10-17T01:14:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca28142700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20100de0 0x7fca20074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca28142700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca28142700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20102980 0x7fca20073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca28142700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20073e00 con 0x7fca20101fe0 2023-10-17T01:14:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca28142700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20073f70 con 0x7fca20102980 2023-10-17T01:14:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca28142700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca201a0bb0 con 0x7fca20100de0 2023-10-17T01:14:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:13.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca25ede700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20100de0 0x7fca20074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:13.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49370/0 (socket says 172.21.15.45:49370) 2023-10-17T01:14:13.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 -- 172.21.15.45:0/3546321121 learned_addr learned my addr 172.21.15.45:0/3546321121 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:13.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca266df700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20102980 0x7fca20073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:13.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 -- 172.21.15.45:0/3546321121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20100de0 msgr2=0x7fca20074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20100de0 0x7fca20074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 -- 172.21.15.45:0/3546321121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20102980 msgr2=0x7fca20073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20102980 0x7fca20073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.355+0000 7fca256dd700 1 -- 172.21.15.45:0/3546321121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100095f0 con 0x7fca20101fe0 2023-10-17T01:14:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.356+0000 7fca256dd700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20073140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca1000bf40 tx=0x7fca1000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.356+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1001d070 con 0x7fca20101fe0 2023-10-17T01:14:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.356+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca10005050 con 0x7fca20101fe0 2023-10-17T01:14:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.356+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10004b40 con 0x7fca20101fe0 2023-10-17T01:14:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.356+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca201a0e30 con 0x7fca20101fe0 2023-10-17T01:14:13.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.356+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca201a1190 con 0x7fca20101fe0 2023-10-17T01:14:13.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.357+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2010c630 con 0x7fca20101fe0 2023-10-17T01:14:13.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.359+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fca10004ce0 con 0x7fca20101fe0 2023-10-17T01:14:13.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.360+0000 7fca16ffd700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fca0c0a0bb0 0x7fca0c0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:13.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.361+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fca100cd1a0 con 0x7fca20101fe0 2023-10-17T01:14:13.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.361+0000 7fca25ede700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fca0c0a0bb0 0x7fca0c0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:13.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.361+0000 7fca25ede700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fca0c0a0bb0 0x7fca0c0a3070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca20101e40 tx=0x7fca1c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:13.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.364+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fca10096b50 con 0x7fca20101fe0 2023-10-17T01:14:13.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.585+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fca20061190 con 0x7fca0c0a0bb0 2023-10-17T01:14:13.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.599+0000 7fca16ffd700 1 -- 172.21.15.45:0/3546321121 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7fca20061190 con 0x7fca0c0a0bb0 2023-10-17T01:14:13.600 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-17T01:14:13.601 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (80s) 6s ago 80s 16.7M - 18.2.0-668-gab9fe435 be87ae50630c 3a024fcced10 2023-10-17T01:14:13.601 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi060 running (78s) 7s ago 78s 30.4M - 18.2.0-668-gab9fe435 be87ae50630c c04ff3c6305f 2023-10-17T01:14:13.601 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi181 running (91s) 7s ago 91s 18.2M - 18.2.0-668-gab9fe435 be87ae50630c 87760045607c 2023-10-17T01:14:13.601 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (98s) 6s ago 97s 13.8M - 18.2.0-668-gab9fe435 be87ae50630c a9a817201820 2023-10-17T01:14:13.602 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi060 running (96s) 7s ago 95s 21.6M - 18.2.0-668-gab9fe435 be87ae50630c b3fe1e674b61 2023-10-17T01:14:13.602 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi181 running (93s) 7s ago 93s 20.4M - 18.2.0-668-gab9fe435 be87ae50630c 61e06affa190 2023-10-17T01:14:13.602 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (84s) 6s ago 83s 16.8M - 18.2.0-668-gab9fe435 be87ae50630c a0215e2ff39f 2023-10-17T01:14:13.602 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi060 running (89s) 7s ago 89s 21.5M - 18.2.0-668-gab9fe435 be87ae50630c 88f89fc3afee 2023-10-17T01:14:13.602 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi181 running (87s) 7s ago 86s 19.9M - 18.2.0-668-gab9fe435 be87ae50630c edd3cab65ebb 2023-10-17T01:14:13.603 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (101s) 6s ago 101s 13.7M - 18.2.0-668-gab9fe435 be87ae50630c b9d0043a2370 2023-10-17T01:14:13.603 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi060 running (104s) 7s ago 104s 20.4M - 18.2.0-668-gab9fe435 be87ae50630c 603319b9b792 2023-10-17T01:14:13.603 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi181 running (106s) 7s ago 106s 18.1M - 18.2.0-668-gab9fe435 be87ae50630c b027706a655c 2023-10-17T01:14:13.603 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (8m) 6s ago 8m 526M - 18.2.0-668-gab9fe435 be87ae50630c 62c2e090a593 2023-10-17T01:14:13.603 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi060 *:8443,8765 running (6m) 7s ago 6m 452M - 18.2.0-668-gab9fe435 be87ae50630c fb7dfbe4e42a 2023-10-17T01:14:13.604 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi181 *:8443,8765 running (6m) 7s ago 6m 452M - 18.2.0-668-gab9fe435 be87ae50630c e48df65a8591 2023-10-17T01:14:13.604 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (8m) 6s ago 8m 92.1M 2048M 18.2.0-668-gab9fe435 be87ae50630c a6f83c0b712c 2023-10-17T01:14:13.604 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi060 running (6m) 7s ago 6m 67.2M 2048M 18.2.0-668-gab9fe435 be87ae50630c 86826790a891 2023-10-17T01:14:13.604 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi181 running (7m) 7s ago 7m 67.3M 2048M 18.2.0-668-gab9fe435 be87ae50630c 19fd7011081b 2023-10-17T01:14:13.604 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (6m) 6s ago 6m 68.0M 4319M 18.2.0-668-gab9fe435 be87ae50630c e9281704a23d 2023-10-17T01:14:13.604 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi060 running (5m) 7s ago 5m 74.1M 4293M 18.2.0-668-gab9fe435 be87ae50630c 55fa8ea8357a 2023-10-17T01:14:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi181 running (5m) 7s ago 5m 74.1M 4292M 18.2.0-668-gab9fe435 be87ae50630c 0729a2336d88 2023-10-17T01:14:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (5m) 6s ago 5m 62.8M 4319M 18.2.0-668-gab9fe435 be87ae50630c 7a7fa5d625af 2023-10-17T01:14:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi060 running (4m) 7s ago 4m 59.6M 4293M 18.2.0-668-gab9fe435 be87ae50630c cb4e735081ac 2023-10-17T01:14:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi181 running (4m) 7s ago 4m 64.9M 4292M 18.2.0-668-gab9fe435 be87ae50630c 107b5a5d2f22 2023-10-17T01:14:13.606 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (3m) 6s ago 3m 59.1M 4319M 18.2.0-668-gab9fe435 be87ae50630c f52cc58b92d8 2023-10-17T01:14:13.606 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi060 running (3m) 7s ago 3m 62.1M 4293M 18.2.0-668-gab9fe435 be87ae50630c 1fe897f1c60f 2023-10-17T01:14:13.606 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi181 running (3m) 7s ago 3m 61.8M 4292M 18.2.0-668-gab9fe435 be87ae50630c bce5813389bf 2023-10-17T01:14:13.606 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (2m) 6s ago 2m 56.3M 4319M 18.2.0-668-gab9fe435 be87ae50630c 077a2cae3d5c 2023-10-17T01:14:13.606 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi060 running (2m) 7s ago 2m 64.2M 4293M 18.2.0-668-gab9fe435 be87ae50630c 394d7cb93795 2023-10-17T01:14:13.607 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi181 running (2m) 7s ago 2m 54.6M 4292M 18.2.0-668-gab9fe435 be87ae50630c 78cb76df44a8 2023-10-17T01:14:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fca0c0a0bb0 msgr2=0x7fca0c0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:13.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fca0c0a0bb0 0x7fca0c0a3070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca20101e40 tx=0x7fca1c007400 comp rx=0 tx=0).stop 2023-10-17T01:14:13.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 msgr2=0x7fca20073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:13.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20073140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fca1000bf40 tx=0x7fca1000f690 comp rx=0 tx=0).stop 2023-10-17T01:14:13.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 shutdown_connections 2023-10-17T01:14:13.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fca0c0a0bb0 0x7fca0c0a3070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20102980 0x7fca20073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20100de0 0x7fca20074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.601+0000 7fca28142700 1 --2- 172.21.15.45:0/3546321121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca20101fe0 0x7fca20073140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.602+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 >> 172.21.15.45:0/3546321121 conn(0x7fca200fc340 msgr2=0x7fca200fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.602+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 shutdown_connections 2023-10-17T01:14:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:13.602+0000 7fca28142700 1 -- 172.21.15.45:0/3546321121 wait complete. 2023-10-17T01:14:13.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:13 smithi045 ceph-mon[98474]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:13 smithi060 ceph-mon[134136]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:13 smithi181 ceph-mon[134520]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:14.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-17T01:14:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:15.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:15 smithi045 ceph-mon[98474]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:15.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:15 smithi045 ceph-mon[98474]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:15 smithi060 ceph-mon[134136]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:15 smithi060 ceph-mon[134136]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:15 smithi181 ceph-mon[134520]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:15 smithi181 ceph-mon[134520]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 -- 172.21.15.45:0/3365607125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24101fe0 msgr2=0x7fed24102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 --2- 172.21.15.45:0/3365607125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24101fe0 0x7fed24102440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fed180099c0 tx=0x7fed18009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 -- 172.21.15.45:0/3365607125 shutdown_connections 2023-10-17T01:14:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 --2- 172.21.15.45:0/3365607125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed24102980 0x7fed24109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 --2- 172.21.15.45:0/3365607125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24100de0 0x7fed24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 --2- 172.21.15.45:0/3365607125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24101fe0 0x7fed24102440 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 -- 172.21.15.45:0/3365607125 >> 172.21.15.45:0/3365607125 conn(0x7fed240fc340 msgr2=0x7fed240fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 -- 172.21.15.45:0/3365607125 shutdown_connections 2023-10-17T01:14:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.378+0000 7fed295cc700 1 -- 172.21.15.45:0/3365607125 wait complete. 2023-10-17T01:14:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.379+0000 7fed295cc700 1 Processor -- start 2023-10-17T01:14:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.379+0000 7fed295cc700 1 -- start start 2023-10-17T01:14:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed295cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 0x7fed24074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed295cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed24101fe0 0x7fed24073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed295cc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24102980 0x7fed24073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed295cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24073e00 con 0x7fed24100de0 2023-10-17T01:14:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed295cc700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24073f70 con 0x7fed24102980 2023-10-17T01:14:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed295cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed241a0bb0 con 0x7fed24101fe0 2023-10-17T01:14:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 0x7fed24074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 0x7fed24074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49400/0 (socket says 172.21.15.45:49400) 2023-10-17T01:14:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 -- 172.21.15.45:0/3160221119 learned_addr learned my addr 172.21.15.45:0/3160221119 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed227fc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed24101fe0 0x7fed24073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed237fe700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24102980 0x7fed24073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 -- 172.21.15.45:0/3160221119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed24101fe0 msgr2=0x7fed24073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed24101fe0 0x7fed24073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 -- 172.21.15.45:0/3160221119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24102980 msgr2=0x7fed24073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24102980 0x7fed24073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.380+0000 7fed22ffd700 1 -- 172.21.15.45:0/3160221119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180095f0 con 0x7fed24100de0 2023-10-17T01:14:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.381+0000 7fed22ffd700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 0x7fed24074af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e970 tx=0x7fed1400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.381+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14009790 con 0x7fed24100de0 2023-10-17T01:14:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.381+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed1400c7e0 con 0x7fed24100de0 2023-10-17T01:14:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.381+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed241a0e90 con 0x7fed24100de0 2023-10-17T01:14:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.381+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14003c70 con 0x7fed24100de0 2023-10-17T01:14:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.381+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed241a12b0 con 0x7fed24100de0 2023-10-17T01:14:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.382+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2410c630 con 0x7fed24100de0 2023-10-17T01:14:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.384+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fed1400cc50 con 0x7fed24100de0 2023-10-17T01:14:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.385+0000 7fed0bfff700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fed0c0a0c30 0x7fed0c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.385+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fed14014070 con 0x7fed24100de0 2023-10-17T01:14:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.386+0000 7fed227fc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fed0c0a0c30 0x7fed0c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.386+0000 7fed227fc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fed0c0a0c30 0x7fed0c0a30f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b580 tx=0x7fed18005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.390+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fed14095cd0 con 0x7fed24100de0 2023-10-17T01:14:16.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.610+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fed24061190 con 0x7fed0c0a0c30 2023-10-17T01:14:16.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.616+0000 7fed0bfff700 1 -- 172.21.15.45:0/3160221119 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7fed24061190 con 0x7fed0c0a0c30 2023-10-17T01:14:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-17T01:14:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 10s ago 112s smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi060=b;smithi060=e;smithi060=h;smithi060=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-10-17T01:14:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 10s ago 6m smithi045=x;smithi060=y;smithi181=z;count:3 2023-10-17T01:14:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 10s ago 7m smithi045:172.21.15.45=a;smithi060:172.21.15.60=b;smithi181:172.21.15.181=c;count:3 2023-10-17T01:14:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 10s ago - 2023-10-17T01:14:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fed0c0a0c30 msgr2=0x7fed0c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fed0c0a0c30 0x7fed0c0a30f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b580 tx=0x7fed18005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 msgr2=0x7fed24074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:16.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 0x7fed24074af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e970 tx=0x7fed1400ec80 comp rx=0 tx=0).stop 2023-10-17T01:14:16.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 shutdown_connections 2023-10-17T01:14:16.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fed0c0a0c30 0x7fed0c0a30f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24102980 0x7fed24073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed24100de0 0x7fed24074af0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 --2- 172.21.15.45:0/3160221119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed24101fe0 0x7fed24073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:16.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 >> 172.21.15.45:0/3160221119 conn(0x7fed240fc340 msgr2=0x7fed240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:16.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 shutdown_connections 2023-10-17T01:14:16.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:16.618+0000 7fed295cc700 1 -- 172.21.15.45:0/3160221119 wait complete. 2023-10-17T01:14:17.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-17T01:14:17.769 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:17 smithi045 ceph-mon[98474]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:17.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:17 smithi060 ceph-mon[134136]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:17 smithi181 ceph-mon[134520]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:18.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:18 smithi045 ceph-mon[98474]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:18.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:18 smithi060 ceph-mon[134136]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:18 smithi181 ceph-mon[134520]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:19.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 -- 172.21.15.45:0/71955494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 msgr2=0x7fd2bc073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:19.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 --2- 172.21.15.45:0/71955494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc073c30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009990 tx=0x7fd2b0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 -- 172.21.15.45:0/71955494 shutdown_connections 2023-10-17T01:14:19.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 --2- 172.21.15.45:0/71955494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc108cb0 0x7fd2bc10b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 --2- 172.21.15.45:0/71955494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc074d80 0x7fd2bc0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 --2- 172.21.15.45:0/71955494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc073c30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 -- 172.21.15.45:0/71955494 >> 172.21.15.45:0/71955494 conn(0x7fd2bc0fc340 msgr2=0x7fd2bc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:19.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.278+0000 7fd2c2705700 1 -- 172.21.15.45:0/71955494 shutdown_connections 2023-10-17T01:14:19.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.279+0000 7fd2c2705700 1 -- 172.21.15.45:0/71955494 wait complete. 2023-10-17T01:14:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.280+0000 7fd2c2705700 1 Processor -- start 2023-10-17T01:14:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.280+0000 7fd2c2705700 1 -- start start 2023-10-17T01:14:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c2705700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c2705700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc074d80 0x7fd2bc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c2705700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc108cb0 0x7fd2bc1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c2705700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc1a08c0 con 0x7fd2bc0737b0 2023-10-17T01:14:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c2705700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc1a0a00 con 0x7fd2bc108cb0 2023-10-17T01:14:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c2705700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc1a0b90 con 0x7fd2bc074d80 2023-10-17T01:14:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37188/0 (socket says 172.21.15.45:37188) 2023-10-17T01:14:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 -- 172.21.15.45:0/1013542408 learned_addr learned my addr 172.21.15.45:0/1013542408 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bb7fe700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc074d80 0x7fd2bc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c0ca2700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc108cb0 0x7fd2bc1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 -- 172.21.15.45:0/1013542408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc074d80 msgr2=0x7fd2bc199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc074d80 0x7fd2bc199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 -- 172.21.15.45:0/1013542408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc108cb0 msgr2=0x7fd2bc1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc108cb0 0x7fd2bc1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bbfff700 1 -- 172.21.15.45:0/1013542408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00095f0 con 0x7fd2bc0737b0 2023-10-17T01:14:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2bb7fe700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc074d80 0x7fd2bc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.281+0000 7fd2c0ca2700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc108cb0 0x7fd2bc1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:14:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.282+0000 7fd2bbfff700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc198d30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00e9a0 tx=0x7fd2ac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.282+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac009790 con 0x7fd2bc0737b0 2023-10-17T01:14:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.282+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc1a0e70 con 0x7fd2bc0737b0 2023-10-17T01:14:19.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.282+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ac004cf0 con 0x7fd2bc0737b0 2023-10-17T01:14:19.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.282+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac005620 con 0x7fd2bc0737b0 2023-10-17T01:14:19.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.282+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc1a1280 con 0x7fd2bc0737b0 2023-10-17T01:14:19.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.283+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2bc107360 con 0x7fd2bc0737b0 2023-10-17T01:14:19.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.285+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fd2ac004750 con 0x7fd2bc0737b0 2023-10-17T01:14:19.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.286+0000 7fd2b97fa700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2a40a0bb0 0x7fd2a40a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:19.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.286+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2ac014070 con 0x7fd2bc0737b0 2023-10-17T01:14:19.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.287+0000 7fd2bb7fe700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2a40a0bb0 0x7fd2a40a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:19.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.287+0000 7fd2bb7fe700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2a40a0bb0 0x7fd2a40a3070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000c00 tx=0x7fd2b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:19.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.290+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd2ac095e30 con 0x7fd2bc0737b0 2023-10-17T01:14:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.508+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2bc061190 con 0x7fd2a40a0bb0 2023-10-17T01:14:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.510+0000 7fd2b97fa700 1 -- 172.21.15.45:0/1013542408 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fd2bc061190 con 0x7fd2a40a0bb0 2023-10-17T01:14:19.511 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2023-10-17T01:14:19.511 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2023-10-17T01:14:19.511 INFO:teuthology.orchestra.run.smithi045.stdout:smithi060 172.21.15.60 2023-10-17T01:14:19.511 INFO:teuthology.orchestra.run.smithi045.stdout:smithi181 172.21.15.181 2023-10-17T01:14:19.512 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2023-10-17T01:14:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2a40a0bb0 msgr2=0x7fd2a40a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2a40a0bb0 0x7fd2a40a3070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000c00 tx=0x7fd2b0005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 msgr2=0x7fd2bc198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc198d30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00e9a0 tx=0x7fd2ac00ed60 comp rx=0 tx=0).stop 2023-10-17T01:14:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 shutdown_connections 2023-10-17T01:14:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd2a40a0bb0 0x7fd2a40a3070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2bc108cb0 0x7fd2bc1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2bc0737b0 0x7fd2bc198d30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 --2- 172.21.15.45:0/1013542408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2bc074d80 0x7fd2bc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 >> 172.21.15.45:0/1013542408 conn(0x7fd2bc0fc340 msgr2=0x7fd2bc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 shutdown_connections 2023-10-17T01:14:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:19.512+0000 7fd2c2705700 1 -- 172.21.15.45:0/1013542408 wait complete. 2023-10-17T01:14:19.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:19 smithi045 ceph-mon[98474]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:19 smithi060 ceph-mon[134136]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:19 smithi181 ceph-mon[134520]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:19.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-17T01:14:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:20.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:20 smithi045 ceph-mon[98474]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:20.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:20 smithi060 ceph-mon[134136]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:20.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:20 smithi181 ceph-mon[134520]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:21.764 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:21 smithi045 ceph-mon[98474]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:21.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:21 smithi060 ceph-mon[134136]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:21 smithi181 ceph-mon[134520]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 -- 172.21.15.45:0/807581454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 msgr2=0x7f5994109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 --2- 172.21.15.45:0/807581454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f5994109230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f598c00b0d0 tx=0x7f598c00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 -- 172.21.15.45:0/807581454 shutdown_connections 2023-10-17T01:14:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 --2- 172.21.15.45:0/807581454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f5994109230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 --2- 172.21.15.45:0/807581454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994100de0 0x7f5994101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 --2- 172.21.15.45:0/807581454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5994101fe0 0x7f5994102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 -- 172.21.15.45:0/807581454 >> 172.21.15.45:0/807581454 conn(0x7f59940fc340 msgr2=0x7f59940fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 -- 172.21.15.45:0/807581454 shutdown_connections 2023-10-17T01:14:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.444+0000 7f5998caf700 1 -- 172.21.15.45:0/807581454 wait complete. 2023-10-17T01:14:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.445+0000 7f5998caf700 1 Processor -- start 2023-10-17T01:14:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 -- start start 2023-10-17T01:14:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5994100de0 0x7f5994198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 0x7f59941992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f59941a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59941a08f0 con 0x7f5994102980 2023-10-17T01:14:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59941a0a30 con 0x7f5994100de0 2023-10-17T01:14:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5998caf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59941a0bc0 con 0x7f5994101fe0 2023-10-17T01:14:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5992d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f59941a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5991d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 0x7f59941992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5991d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 0x7f59941992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:49574/0 (socket says 172.21.15.45:49574) 2023-10-17T01:14:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f599259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5994100de0 0x7f5994198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5992d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f59941a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37198/0 (socket says 172.21.15.45:37198) 2023-10-17T01:14:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.446+0000 7f5991d9b700 1 -- 172.21.15.45:0/2506861490 learned_addr learned my addr 172.21.15.45:0/2506861490 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5991d9b700 1 -- 172.21.15.45:0/2506861490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5994100de0 msgr2=0x7f5994198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5991d9b700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5994100de0 0x7f5994198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5991d9b700 1 -- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 msgr2=0x7f59941a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5991d9b700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f59941a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5991d9b700 1 -- 172.21.15.45:0/2506861490 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c009d00 con 0x7f5994101fe0 2023-10-17T01:14:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5992d9d700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f59941a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5991d9b700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 0x7f59941992a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f598800e9d0 tx=0x7f598800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988009790 con 0x7f5994101fe0 2023-10-17T01:14:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598800ee70 con 0x7f5994101fe0 2023-10-17T01:14:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59941a0ea0 con 0x7f5994101fe0 2023-10-17T01:14:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.447+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988010ea0 con 0x7f5994101fe0 2023-10-17T01:14:22.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.448+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59941a1310 con 0x7f5994101fe0 2023-10-17T01:14:22.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.449+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5994193430 con 0x7f5994101fe0 2023-10-17T01:14:22.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.451+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f598800cbd0 con 0x7f5994101fe0 2023-10-17T01:14:22.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.452+0000 7f59837fe700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f597c0a0bb0 0x7f597c0a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:22.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.452+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5988014070 con 0x7f5994101fe0 2023-10-17T01:14:22.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.453+0000 7f599259c700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f597c0a0bb0 0x7f597c0a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:22.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.453+0000 7f599259c700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f597c0a0bb0 0x7f597c0a3070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5994101e40 tx=0x7f5984009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:22.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.454+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f5988096130 con 0x7f5994101fe0 2023-10-17T01:14:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.673+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5994061190 con 0x7f597c0a0bb0 2023-10-17T01:14:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.675+0000 7f59837fe700 1 -- 172.21.15.45:0/2506861490 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f5994061190 con 0x7f597c0a0bb0 2023-10-17T01:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-17T01:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-10-17T01:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout:smithi060 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476008A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-10-17T01:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-17T01:14:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f597c0a0bb0 msgr2=0x7f597c0a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f597c0a0bb0 0x7f597c0a3070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5994101e40 tx=0x7f5984009260 comp rx=0 tx=0).stop 2023-10-17T01:14:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 msgr2=0x7f59941992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 0x7f59941992a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f598800e9d0 tx=0x7f598800ece0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 shutdown_connections 2023-10-17T01:14:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f597c0a0bb0 0x7f597c0a3070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5994102980 0x7f59941a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5994100de0 0x7f5994198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 --2- 172.21.15.45:0/2506861490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101fe0 0x7f59941992a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 >> 172.21.15.45:0/2506861490 conn(0x7f59940fc340 msgr2=0x7f59941078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 shutdown_connections 2023-10-17T01:14:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:22.677+0000 7f5998caf700 1 -- 172.21.15.45:0/2506861490 wait complete. 2023-10-17T01:14:23.187 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-17T01:14:23.200 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:14:23.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-17T01:14:23.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:23 smithi045 ceph-mon[98474]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:23 smithi060 ceph-mon[134136]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:23 smithi181 ceph-mon[134520]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:24.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:24 smithi045 ceph-mon[98474]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:24.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:24 smithi060 ceph-mon[134136]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:24.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:24 smithi181 ceph-mon[134520]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-17T01:14:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 -- 172.21.15.45:0/1230295833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 msgr2=0x7f3e50101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 --2- 172.21.15.45:0/1230295833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50101200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3e400099c0 tx=0x7f3e40009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 -- 172.21.15.45:0/1230295833 shutdown_connections 2023-10-17T01:14:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 --2- 172.21.15.45:0/1230295833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 0x7f3e50109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 --2- 172.21.15.45:0/1230295833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50101200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 --2- 172.21.15.45:0/1230295833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e50101fe0 0x7f3e50102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 -- 172.21.15.45:0/1230295833 >> 172.21.15.45:0/1230295833 conn(0x7f3e500fc340 msgr2=0x7f3e500fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 -- 172.21.15.45:0/1230295833 shutdown_connections 2023-10-17T01:14:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.360+0000 7f3e55c92700 1 -- 172.21.15.45:0/1230295833 wait complete. 2023-10-17T01:14:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 Processor -- start 2023-10-17T01:14:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 -- start start 2023-10-17T01:14:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e50101fe0 0x7f3e50197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 0x7f3e5019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5019e660 con 0x7f3e50100de0 2023-10-17T01:14:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5019e7a0 con 0x7f3e50102980 2023-10-17T01:14:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.361+0000 7f3e55c92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5019e930 con 0x7f3e50101fe0 2023-10-17T01:14:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37212/0 (socket says 172.21.15.45:37212) 2023-10-17T01:14:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 -- 172.21.15.45:0/4198528016 learned_addr learned my addr 172.21.15.45:0/4198528016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4effd700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e50101fe0 0x7f3e50197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4ffff700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 0x7f3e5019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 -- 172.21.15.45:0/4198528016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e50101fe0 msgr2=0x7f3e50197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e50101fe0 0x7f3e50197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 -- 172.21.15.45:0/4198528016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 msgr2=0x7f3e5019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 0x7f3e5019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 -- 172.21.15.45:0/4198528016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400095f0 con 0x7f3e50100de0 2023-10-17T01:14:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4ffff700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 0x7f3e5019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.362+0000 7f3e4f7fe700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50196ad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000bf40 tx=0x7f3e4000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.363+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4001d070 con 0x7f3e50100de0 2023-10-17T01:14:25.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.363+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5019ebb0 con 0x7f3e50100de0 2023-10-17T01:14:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.363+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e40005050 con 0x7f3e50100de0 2023-10-17T01:14:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.363+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004bd0 con 0x7f3e50100de0 2023-10-17T01:14:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.363+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5019efc0 con 0x7f3e50100de0 2023-10-17T01:14:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.364+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50108400 con 0x7f3e50100de0 2023-10-17T01:14:25.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.366+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f3e40004d50 con 0x7f3e50100de0 2023-10-17T01:14:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.367+0000 7f3e4cff9700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3e380a0bb0 0x7f3e380a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.367+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3e400cd240 con 0x7f3e50100de0 2023-10-17T01:14:25.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.368+0000 7f3e4effd700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3e380a0bb0 0x7f3e380a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:25.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.368+0000 7f3e4effd700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3e380a0bb0 0x7f3e380a3070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e44007c60 tx=0x7f3e440073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.371+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3e40096bf0 con 0x7f3e50100de0 2023-10-17T01:14:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.629+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e5002ce80 con 0x7f3e50100de0 2023-10-17T01:14:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.630+0000 7f3e4cff9700 1 -- 172.21.15.45:0/4198528016 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2900 (secure 0 0 0) 0x7f3e40026b70 con 0x7f3e50100de0 2023-10-17T01:14:25.632 INFO:teuthology.orchestra.run.smithi045.stdout:e26 2023-10-17T01:14:25.632 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-17T01:14:25.632 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-17T01:14:25.633 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2023-10-17T01:14:25.633 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:25.633 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2023-10-17T01:14:25.633 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2023-10-17T01:14:25.633 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 26 2023-10-17T01:14:25.634 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-17T01:14:25.634 INFO:teuthology.orchestra.run.smithi045.stdout:created 2023-10-17T01:13:01.038089+0000 2023-10-17T01:14:25.634 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2023-10-17T01:13:08.449854+0000 2023-10-17T01:14:25.634 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2023-10-17T01:14:25.634 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2023-10-17T01:14:25.635 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2023-10-17T01:14:25.635 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2023-10-17T01:14:25.635 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2023-10-17T01:14:25.635 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2023-10-17T01:14:25.635 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2023-10-17T01:14:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2023-10-17T01:14:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-17T01:14:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 5 2023-10-17T01:14:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2,3,4 2023-10-17T01:14:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24587,1=24550,2=24539,3=24526,4=24514} 2023-10-17T01:14:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2023-10-17T01:14:25.637 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2023-10-17T01:14:25.637 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2023-10-17T01:14:25.637 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3] 2023-10-17T01:14:25.637 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2023-10-17T01:14:25.637 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2023-10-17T01:14:25.638 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2023-10-17T01:14:25.638 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2023-10-17T01:14:25.638 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2023-10-17T01:14:25.638 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24587} state up:active seq 3 addr [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.638 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{0:24509} state up:standby-replay seq 1 addr [v2:172.21.15.60:6834/1725566254,v1:172.21.15.60:6835/1725566254] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.639 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{1:24550} state up:active seq 6 addr [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.639 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{1:24473} state up:standby-replay seq 1 addr [v2:172.21.15.60:6832/3345979209,v1:172.21.15.60:6833/3345979209] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.639 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{2:24539} state up:active seq 7 addr [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.639 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{2:24463} state up:standby-replay seq 1 addr [v2:172.21.15.181:6832/2648148860,v1:172.21.15.181:6833/2648148860] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.639 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{3:24526} state up:active seq 8 addr [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.640 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{3:14556} state up:standby-replay seq 1 addr [v2:172.21.15.45:6840/1667366220,v1:172.21.15.45:6841/1667366220] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.640 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{4:24514} state up:active seq 8 addr [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.640 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{4:14544} state up:standby-replay seq 1 addr [v2:172.21.15.45:6838/931419854,v1:172.21.15.45:6839/931419854] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:25.640 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2023-10-17T01:14:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:25.641 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14475} state up:standby seq 1 addr [v2:172.21.15.45:6834/1708283883,v1:172.21.15.45:6835/1708283883] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.641 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14487} state up:standby seq 1 addr [v2:172.21.15.45:6836/1507187876,v1:172.21.15.45:6837/1507187876] compat {c=[1],r=[1],i=[7ff]}] 2023-10-17T01:14:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.632+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3e380a0bb0 msgr2=0x7f3e380a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.632+0000 7f3e55c92700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3e380a0bb0 0x7f3e380a3070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e44007c60 tx=0x7f3e440073d0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.632+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 msgr2=0x7f3e50196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:25.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.632+0000 7f3e55c92700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50196ad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000bf40 tx=0x7f3e4000f690 comp rx=0 tx=0).stop 2023-10-17T01:14:25.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 shutdown_connections 2023-10-17T01:14:25.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3e380a0bb0 0x7f3e380a3070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e50102980 0x7f3e5019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e50100de0 0x7f3e50196ad0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 --2- 172.21.15.45:0/4198528016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e50101fe0 0x7f3e50197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:25.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 >> 172.21.15.45:0/4198528016 conn(0x7f3e500fc340 msgr2=0x7f3e50105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:25.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 shutdown_connections 2023-10-17T01:14:25.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:25.633+0000 7f3e55c92700 1 -- 172.21.15.45:0/4198528016 wait complete. 2023-10-17T01:14:25.644 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:25.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:25 smithi045 ceph-mon[98474]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:25 smithi060 ceph-mon[134136]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:25 smithi181 ceph-mon[134520]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:26.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-17T01:14:26.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:26 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4198528016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-17T01:14:26.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:26 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/4198528016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-17T01:14:26.864 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:26.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:26 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/4198528016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-17T01:14:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:27 smithi045 ceph-mon[98474]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:27 smithi060 ceph-mon[134136]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:27 smithi181 ceph-mon[134520]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-17T01:14:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 -- 172.21.15.45:0/2277887690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 msgr2=0x7fc590102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 --2- 172.21.15.45:0/2277887690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc590102440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5840099c0 tx=0x7fc584009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 -- 172.21.15.45:0/2277887690 shutdown_connections 2023-10-17T01:14:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 --2- 172.21.15.45:0/2277887690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 0x7fc590109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 --2- 172.21.15.45:0/2277887690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc590100de0 0x7fc590101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 --2- 172.21.15.45:0/2277887690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc590102440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 -- 172.21.15.45:0/2277887690 >> 172.21.15.45:0/2277887690 conn(0x7fc5900fc340 msgr2=0x7fc5900fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 -- 172.21.15.45:0/2277887690 shutdown_connections 2023-10-17T01:14:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.401+0000 7fc596382700 1 -- 172.21.15.45:0/2277887690 wait complete. 2023-10-17T01:14:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.402+0000 7fc596382700 1 Processor -- start 2023-10-17T01:14:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.402+0000 7fc596382700 1 -- start start 2023-10-17T01:14:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc596382700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc590100de0 0x7fc590198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc596382700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc5901992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc596382700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 0x7fc5901a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc596382700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5901a08f0 con 0x7fc590101fe0 2023-10-17T01:14:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc596382700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5901a0a30 con 0x7fc590100de0 2023-10-17T01:14:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc596382700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5901a0bc0 con 0x7fc590102980 2023-10-17T01:14:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc5901992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc5901992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41546/0 (socket says 172.21.15.45:41546) 2023-10-17T01:14:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 -- 172.21.15.45:0/1416767992 learned_addr learned my addr 172.21.15.45:0/1416767992 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58ffff700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc590100de0 0x7fc590198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc59491f700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 0x7fc5901a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 -- 172.21.15.45:0/1416767992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 msgr2=0x7fc5901a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 0x7fc5901a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 -- 172.21.15.45:0/1416767992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc590100de0 msgr2=0x7fc590198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc590100de0 0x7fc590198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc58f7fe700 1 -- 172.21.15.45:0/1416767992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840095f0 con 0x7fc590101fe0 2023-10-17T01:14:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.403+0000 7fc59491f700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 0x7fc5901a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.404+0000 7fc58f7fe700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc5901992a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5840095c0 tx=0x7fc58400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.404+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58401d070 con 0x7fc590101fe0 2023-10-17T01:14:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.404+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5901a0e40 con 0x7fc590101fe0 2023-10-17T01:14:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.404+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc584005510 con 0x7fc590101fe0 2023-10-17T01:14:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.404+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5901a1200 con 0x7fc590101fe0 2023-10-17T01:14:28.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.404+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584004af0 con 0x7fc590101fe0 2023-10-17T01:14:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.405+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59010c630 con 0x7fc590101fe0 2023-10-17T01:14:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.407+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fc58400fb70 con 0x7fc590101fe0 2023-10-17T01:14:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.408+0000 7fc58d7fa700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc5780a0bb0 0x7fc5780a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.408+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc5840ccff0 con 0x7fc590101fe0 2023-10-17T01:14:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.409+0000 7fc58ffff700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc5780a0bb0 0x7fc5780a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.409+0000 7fc58ffff700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc5780a0bb0 0x7fc5780a3070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc590101e40 tx=0x7fc580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.412+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fc58405c0b0 con 0x7fc590101fe0 2023-10-17T01:14:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.628+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc59010c820 con 0x7fc590101fe0 2023-10-17T01:14:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.629+0000 7fc58d7fa700 1 -- 172.21.15.45:0/1416767992 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5237 (secure 0 0 0) 0x7fc5840968e0 con 0x7fc590101fe0 2023-10-17T01:14:28.631 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 85 2023-10-17T01:14:28.631 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 2023-10-17T01:14:28.631 INFO:teuthology.orchestra.run.smithi045.stdout:created 2023-10-17T01:05:32.108363+0000 2023-10-17T01:14:28.632 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2023-10-17T01:13:33.505285+0000 2023-10-17T01:14:28.632 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-17T01:14:28.632 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 26 2023-10-17T01:14:28.632 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2023-10-17T01:14:28.632 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2023-10-17T01:14:28.633 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2023-10-17T01:14:28.633 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2023-10-17T01:14:28.633 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2023-10-17T01:14:28.634 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release reef 2023-10-17T01:14:28.634 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2023-10-17T01:14:28.634 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-17T01:14:28.634 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-10-17T01:14:28.634 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-10-17T01:14:28.635 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2023-10-17T01:14:28.635 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/563780419,v1:172.21.15.45:6803/563780419] [v2:172.21.15.45:6804/563780419,v1:172.21.15.45:6805/563780419] exists,up 84831425-d402-4501-8e75-de13106f9cf9 2023-10-17T01:14:28.635 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6800/1446433682,v1:172.21.15.60:6801/1446433682] [v2:172.21.15.60:6802/1446433682,v1:172.21.15.60:6803/1446433682] exists,up a91b91e7-247a-46b5-b229-bdadf8126588 2023-10-17T01:14:28.635 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/2332429544,v1:172.21.15.181:6801/2332429544] [v2:172.21.15.181:6802/2332429544,v1:172.21.15.181:6803/2332429544] exists,up e2b9dd7f-cf88-49c9-bd44-36a439272919 2023-10-17T01:14:28.635 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/570572613,v1:172.21.15.45:6811/570572613] [v2:172.21.15.45:6812/570572613,v1:172.21.15.45:6813/570572613] exists,up 806e62ab-4fa6-4c3f-80b7-eb25f498baab 2023-10-17T01:14:28.636 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6808/2865514976,v1:172.21.15.60:6809/2865514976] [v2:172.21.15.60:6810/2865514976,v1:172.21.15.60:6811/2865514976] exists,up 3232fc8e-ae84-448d-9665-adc7f64f6c99 2023-10-17T01:14:28.636 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/547299005,v1:172.21.15.181:6809/547299005] [v2:172.21.15.181:6810/547299005,v1:172.21.15.181:6811/547299005] exists,up 28b10aa9-ecab-4539-9fa5-e753609c3bb2 2023-10-17T01:14:28.636 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 46 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/2740142104,v1:172.21.15.45:6819/2740142104] [v2:172.21.15.45:6820/2740142104,v1:172.21.15.45:6821/2740142104] exists,up fb616844-fabd-44f0-9347-9868616fa738 2023-10-17T01:14:28.636 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 51 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6816/4084805853,v1:172.21.15.60:6817/4084805853] [v2:172.21.15.60:6818/4084805853,v1:172.21.15.60:6819/4084805853] exists,up d58f8b41-8d53-4e6c-b2d2-7997cb77e063 2023-10-17T01:14:28.636 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/3559316201,v1:172.21.15.181:6817/3559316201] [v2:172.21.15.181:6818/3559316201,v1:172.21.15.181:6819/3559316201] exists,up 58034e5c-8cae-4694-8a22-bfec322b43b1 2023-10-17T01:14:28.637 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/1619950386,v1:172.21.15.45:6827/1619950386] [v2:172.21.15.45:6828/1619950386,v1:172.21.15.45:6829/1619950386] exists,up f6c8b883-1f5b-4e01-8453-7ecae38051e6 2023-10-17T01:14:28.637 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 71 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6824/1158474064,v1:172.21.15.60:6825/1158474064] [v2:172.21.15.60:6826/1158474064,v1:172.21.15.60:6827/1158474064] exists,up 50b41c1b-61b5-4b1b-9c52-93d9adb7e080 2023-10-17T01:14:28.637 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/429412155,v1:172.21.15.181:6825/429412155] [v2:172.21.15.181:6826/429412155,v1:172.21.15.181:6827/429412155] exists,up 6a70d229-f101-42fe-8231-5e3cc447d528 2023-10-17T01:14:28.637 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.0 [7,11] 2023-10-17T01:14:28.637 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.4 [2,11] 2023-10-17T01:14:28.638 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.10 [2,1] 2023-10-17T01:14:28.638 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1a [8,4] 2023-10-17T01:14:28.638 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1b [5,9] 2023-10-17T01:14:28.638 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.2c [4,9] 2023-10-17T01:14:28.638 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.3c [4,9] 2023-10-17T01:14:28.639 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/518390758 expires 2023-10-18T01:06:32.348953+0000 2023-10-17T01:14:28.639 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1775908551 expires 2023-10-18T01:06:32.348953+0000 2023-10-17T01:14:28.639 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1775908551 expires 2023-10-18T01:06:32.348953+0000 2023-10-17T01:14:28.639 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/3224370568 expires 2023-10-18T01:06:05.897832+0000 2023-10-17T01:14:28.639 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3128471816 expires 2023-10-18T01:06:05.897832+0000 2023-10-17T01:14:28.640 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2130990792 expires 2023-10-18T01:06:05.897832+0000 2023-10-17T01:14:28.640 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/3224370568 expires 2023-10-18T01:06:05.897832+0000 2023-10-17T01:14:28.640 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/887015676 expires 2023-10-18T01:06:32.348953+0000 2023-10-17T01:14:28.640 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/4289245093 expires 2023-10-18T01:06:05.897832+0000 2023-10-17T01:14:28.640 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2694295985 expires 2023-10-18T01:06:32.348953+0000 2023-10-17T01:14:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc5780a0bb0 msgr2=0x7fc5780a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc5780a0bb0 0x7fc5780a3070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc590101e40 tx=0x7fc580009260 comp rx=0 tx=0).stop 2023-10-17T01:14:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 msgr2=0x7fc5901992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc5901992a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5840095c0 tx=0x7fc58400f690 comp rx=0 tx=0).stop 2023-10-17T01:14:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 shutdown_connections 2023-10-17T01:14:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc5780a0bb0 0x7fc5780a3070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc590102980 0x7fc5901a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc590100de0 0x7fc590198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 --2- 172.21.15.45:0/1416767992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc590101fe0 0x7fc5901992a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 >> 172.21.15.45:0/1416767992 conn(0x7fc5900fc340 msgr2=0x7fc5900fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 shutdown_connections 2023-10-17T01:14:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:28.632+0000 7fc596382700 1 -- 172.21.15.45:0/1416767992 wait complete. 2023-10-17T01:14:29.152 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-17T01:14:29.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 -- 172.21.15.45:0/2456421390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 msgr2=0x7f08a41023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/2456421390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a41023b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08980099c0 tx=0x7f0898009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 -- 172.21.15.45:0/2456421390 shutdown_connections 2023-10-17T01:14:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/2456421390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a41028f0 0x7f08a41091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/2456421390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08a4100d60 0x7f08a4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/2456421390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a41023b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 -- 172.21.15.45:0/2456421390 >> 172.21.15.45:0/2456421390 conn(0x7f08a40fc290 msgr2=0x7f08a40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 -- 172.21.15.45:0/2456421390 shutdown_connections 2023-10-17T01:14:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.374+0000 7f08aa3ff700 1 -- 172.21.15.45:0/2456421390 wait complete. 2023-10-17T01:14:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.375+0000 7f08aa3ff700 1 Processor -- start 2023-10-17T01:14:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.375+0000 7f08aa3ff700 1 -- start start 2023-10-17T01:14:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08aa3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4100d60 0x7f08a4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08aa3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08aa3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08a41028f0 0x7f08a41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08aa3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41a0790 con 0x7f08a4101f50 2023-10-17T01:14:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08aa3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41a0900 con 0x7f08a41028f0 2023-10-17T01:14:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08aa3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41a0ac0 con 0x7f08a4100d60 2023-10-17T01:14:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a4199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41572/0 (socket says 172.21.15.45:41572) 2023-10-17T01:14:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 -- 172.21.15.45:0/3325344324 learned_addr learned my addr 172.21.15.45:0/3325344324 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a3fff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4100d60 0x7f08a4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a899c700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08a41028f0 0x7f08a41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 -- 172.21.15.45:0/3325344324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4100d60 msgr2=0x7f08a4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4100d60 0x7f08a4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 -- 172.21.15.45:0/3325344324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08a41028f0 msgr2=0x7f08a41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08a41028f0 0x7f08a41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.376+0000 7f08a37fe700 1 -- 172.21.15.45:0/3325344324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08980095f0 con 0x7f08a4101f50 2023-10-17T01:14:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.377+0000 7f08a37fe700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a4199190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f089800bf40 tx=0x7f089800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.377+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089801d070 con 0x7f08a4101f50 2023-10-17T01:14:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.377+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0898005050 con 0x7f08a4101f50 2023-10-17T01:14:29.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.377+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a41a0d40 con 0x7f08a4101f50 2023-10-17T01:14:29.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.377+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898004b40 con 0x7f08a4101f50 2023-10-17T01:14:29.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.377+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a41a1160 con 0x7f08a4101f50 2023-10-17T01:14:29.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.378+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a41933b0 con 0x7f08a4101f50 2023-10-17T01:14:29.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.380+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f0898004ce0 con 0x7f08a4101f50 2023-10-17T01:14:29.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.381+0000 7f08a17fa700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f088c0a0bf0 0x7f088c0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.381+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f08980cd1a0 con 0x7f08a4101f50 2023-10-17T01:14:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.382+0000 7f08a3fff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f088c0a0bf0 0x7f088c0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.382+0000 7f08a3fff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f088c0a0bf0 0x7f088c0a30b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f08a419a120 tx=0x7f0894009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.385+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0898096b50 con 0x7f08a4101f50 2023-10-17T01:14:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.604+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08a402ce10 con 0x7f08a4101f50 2023-10-17T01:14:29.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.606+0000 7f08a17fa700 1 -- 172.21.15.45:0/3325344324 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f089809a170 con 0x7f08a4101f50 2023-10-17T01:14:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:29.608 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:29.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.608+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f088c0a0bf0 msgr2=0x7f088c0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.608+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f088c0a0bf0 0x7f088c0a30b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f08a419a120 tx=0x7f0894009260 comp rx=0 tx=0).stop 2023-10-17T01:14:29.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 msgr2=0x7f08a4199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a4199190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f089800bf40 tx=0x7f089800f690 comp rx=0 tx=0).stop 2023-10-17T01:14:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 shutdown_connections 2023-10-17T01:14:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f088c0a0bf0 0x7f088c0a30b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08a41028f0 0x7f08a41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a4100d60 0x7f08a4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 --2- 172.21.15.45:0/3325344324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a4101f50 0x7f08a4199190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 >> 172.21.15.45:0/3325344324 conn(0x7f08a40fc290 msgr2=0x7f08a4107830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:29.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 shutdown_connections 2023-10-17T01:14:29.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.609+0000 7f08aa3ff700 1 -- 172.21.15.45:0/3325344324 wait complete. 2023-10-17T01:14:29.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:29.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:29 smithi045 ceph-mon[98474]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:29 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1416767992' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-17T01:14:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:29 smithi060 ceph-mon[134136]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:29 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1416767992' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-17T01:14:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 -- 172.21.15.45:0/2434388842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c101f50 msgr2=0x7f7d8c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 --2- 172.21.15.45:0/2434388842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1023b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7d800099c0 tx=0x7f7d80009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 -- 172.21.15.45:0/2434388842 shutdown_connections 2023-10-17T01:14:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 --2- 172.21.15.45:0/2434388842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c1028f0 0x7f7d8c1091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 --2- 172.21.15.45:0/2434388842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8c100d60 0x7f7d8c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 --2- 172.21.15.45:0/2434388842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1023b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 -- 172.21.15.45:0/2434388842 >> 172.21.15.45:0/2434388842 conn(0x7f7d8c0fc2d0 msgr2=0x7f7d8c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 -- 172.21.15.45:0/2434388842 shutdown_connections 2023-10-17T01:14:29.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.837+0000 7f7d915ff700 1 -- 172.21.15.45:0/2434388842 wait complete. 2023-10-17T01:14:29.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 Processor -- start 2023-10-17T01:14:29.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 -- start start 2023-10-17T01:14:29.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8c100d60 0x7f7d8c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c1028f0 0x7f7d8c1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c1a07b0 con 0x7f7d8c1028f0 2023-10-17T01:14:29.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c1a0920 con 0x7f7d8c101f50 2023-10-17T01:14:29.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.838+0000 7f7d915ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c1a0ae0 con 0x7f7d8c100d60 2023-10-17T01:14:29.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c1028f0 0x7f7d8c1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53652/0 (socket says 172.21.15.45:53652) 2023-10-17T01:14:29.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8a7fc700 1 -- 172.21.15.45:0/880423294 learned_addr learned my addr 172.21.15.45:0/880423294 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8affd700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8c100d60 0x7f7d8c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 -- 172.21.15.45:0/880423294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8c100d60 msgr2=0x7f7d8c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8c100d60 0x7f7d8c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 -- 172.21.15.45:0/880423294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c101f50 msgr2=0x7f7d8c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:29.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:29.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 -- 172.21.15.45:0/880423294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d800095f0 con 0x7f7d8c1028f0 2023-10-17T01:14:29.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.839+0000 7f7d8b7fe700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c1028f0 0x7f7d8c1a01e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400efc0 tx=0x7f7d8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:29.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.840+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84018070 con 0x7f7d8c1028f0 2023-10-17T01:14:29.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.840+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8c1a0d60 con 0x7f7d8c1028f0 2023-10-17T01:14:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.840+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d8400f040 con 0x7f7d8c1028f0 2023-10-17T01:14:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.840+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400cd40 con 0x7f7d8c1028f0 2023-10-17T01:14:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.840+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8c1a11c0 con 0x7f7d8c1028f0 2023-10-17T01:14:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.841+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8c10c5c0 con 0x7f7d8c1028f0 2023-10-17T01:14:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.843+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f7d84003a00 con 0x7f7d8c1028f0 2023-10-17T01:14:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.844+0000 7f7d73fff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d740a0bb0 0x7f7d740a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.844+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7d840cbee0 con 0x7f7d8c1028f0 2023-10-17T01:14:29.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.844+0000 7f7d8affd700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d740a0bb0 0x7f7d740a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:29.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.845+0000 7f7d8affd700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d740a0bb0 0x7f7d740a3070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c19a140 tx=0x7f7d7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:29.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:29.848+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f7d84095770 con 0x7f7d8c1028f0 2023-10-17T01:14:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:29 smithi181 ceph-mon[134520]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-17T01:14:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:29 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1416767992' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-17T01:14:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.061+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d8c1a1800 con 0x7f7d8c1028f0 2023-10-17T01:14:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.061+0000 7f7d73fff700 1 -- 172.21.15.45:0/880423294 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d84098d90 con 0x7f7d8c1028f0 2023-10-17T01:14:30.063 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d740a0bb0 msgr2=0x7f7d740a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d740a0bb0 0x7f7d740a3070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c19a140 tx=0x7f7d7c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c1028f0 msgr2=0x7f7d8c1a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c1028f0 0x7f7d8c1a01e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400efc0 tx=0x7f7d8400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 shutdown_connections 2023-10-17T01:14:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7d740a0bb0 0x7f7d740a3070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d8c1028f0 0x7f7d8c1a01e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8c100d60 0x7f7d8c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 --2- 172.21.15.45:0/880423294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d8c101f50 0x7f7d8c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 >> 172.21.15.45:0/880423294 conn(0x7f7d8c0fc2d0 msgr2=0x7f7d8c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:30.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 shutdown_connections 2023-10-17T01:14:30.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.064+0000 7f7d915ff700 1 -- 172.21.15.45:0/880423294 wait complete. 2023-10-17T01:14:30.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 -- 172.21.15.45:0/560016121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684101f50 msgr2=0x7f06841023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 --2- 172.21.15.45:0/560016121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684101f50 0x7f06841023b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06780099c0 tx=0x7f0678009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 -- 172.21.15.45:0/560016121 shutdown_connections 2023-10-17T01:14:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 --2- 172.21.15.45:0/560016121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06841028f0 0x7f06841091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 --2- 172.21.15.45:0/560016121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0684100d60 0x7f0684101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 --2- 172.21.15.45:0/560016121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684101f50 0x7f06841023b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 -- 172.21.15.45:0/560016121 >> 172.21.15.45:0/560016121 conn(0x7f06840fc2d0 msgr2=0x7f06840fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 -- 172.21.15.45:0/560016121 shutdown_connections 2023-10-17T01:14:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.292+0000 7f0689dff700 1 -- 172.21.15.45:0/560016121 wait complete. 2023-10-17T01:14:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.293+0000 7f0689dff700 1 Processor -- start 2023-10-17T01:14:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.293+0000 7f0689dff700 1 -- start start 2023-10-17T01:14:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.293+0000 7f0689dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 0x7f0684198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.293+0000 7f0689dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0684101f50 0x7f0684199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f0689dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06841028f0 0x7f06841a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f0689dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a0790 con 0x7f0684100d60 2023-10-17T01:14:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f0689dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a0900 con 0x7f06841028f0 2023-10-17T01:14:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f0689dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841a0ac0 con 0x7f0684101f50 2023-10-17T01:14:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 0x7f0684198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 0x7f0684198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41618/0 (socket says 172.21.15.45:41618) 2023-10-17T01:14:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 -- 172.21.15.45:0/2070481771 learned_addr learned my addr 172.21.15.45:0/2070481771 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f0683fff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06841028f0 0x7f06841a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f0682ffd700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0684101f50 0x7f0684199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 -- 172.21.15.45:0/2070481771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0684101f50 msgr2=0x7f0684199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0684101f50 0x7f0684199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 -- 172.21.15.45:0/2070481771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06841028f0 msgr2=0x7f06841a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06841028f0 0x7f06841a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 -- 172.21.15.45:0/2070481771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780095f0 con 0x7f0684100d60 2023-10-17T01:14:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.294+0000 7f06837fe700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 0x7f0684198c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f067400e970 tx=0x7f067400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.295+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674009790 con 0x7f0684100d60 2023-10-17T01:14:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.295+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067400eec0 con 0x7f0684100d60 2023-10-17T01:14:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.295+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06841a0da0 con 0x7f0684100d60 2023-10-17T01:14:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.295+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674003cf0 con 0x7f0684100d60 2023-10-17T01:14:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.295+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06841a1210 con 0x7f0684100d60 2023-10-17T01:14:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.296+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0684193360 con 0x7f0684100d60 2023-10-17T01:14:30.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.298+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f067400ccd0 con 0x7f0684100d60 2023-10-17T01:14:30.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.299+0000 7f0680ff9700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f066c0a0c30 0x7f066c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.299+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0674014070 con 0x7f0684100d60 2023-10-17T01:14:30.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.300+0000 7f0682ffd700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f066c0a0c30 0x7f066c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.300+0000 7f0682ffd700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f066c0a0c30 0x7f066c0a30f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f067800b580 tx=0x7f067801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:30.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.303+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0674095d40 con 0x7f0684100d60 2023-10-17T01:14:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.515+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f068402cea0 con 0x7f0684100d60 2023-10-17T01:14:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.516+0000 7f0680ff9700 1 -- 172.21.15.45:0/2070481771 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0674099360 con 0x7f0684100d60 2023-10-17T01:14:30.517 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:30.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f066c0a0c30 msgr2=0x7f066c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f066c0a0c30 0x7f066c0a30f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f067800b580 tx=0x7f067801a040 comp rx=0 tx=0).stop 2023-10-17T01:14:30.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 msgr2=0x7f0684198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 0x7f0684198c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f067400e970 tx=0x7f067400ec80 comp rx=0 tx=0).stop 2023-10-17T01:14:30.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 shutdown_connections 2023-10-17T01:14:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f066c0a0c30 0x7f066c0a30f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06841028f0 0x7f06841a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0684100d60 0x7f0684198c50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 --2- 172.21.15.45:0/2070481771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0684101f50 0x7f0684199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 >> 172.21.15.45:0/2070481771 conn(0x7f06840fc2d0 msgr2=0x7f0684107880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 shutdown_connections 2023-10-17T01:14:30.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.518+0000 7f0689dff700 1 -- 172.21.15.45:0/2070481771 wait complete. 2023-10-17T01:14:30.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:30.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 -- 172.21.15.45:0/3466742168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 msgr2=0x7faa60101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 --2- 172.21.15.45:0/3466742168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60101160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faa500099c0 tx=0x7faa50009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 -- 172.21.15.45:0/3466742168 shutdown_connections 2023-10-17T01:14:30.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 --2- 172.21.15.45:0/3466742168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa601028d0 0x7faa60109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 --2- 172.21.15.45:0/3466742168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60101160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 --2- 172.21.15.45:0/3466742168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa60101f30 0x7faa60102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 -- 172.21.15.45:0/3466742168 >> 172.21.15.45:0/3466742168 conn(0x7faa600fc290 msgr2=0x7faa600fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:30.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 -- 172.21.15.45:0/3466742168 shutdown_connections 2023-10-17T01:14:30.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.747+0000 7faa655ff700 1 -- 172.21.15.45:0/3466742168 wait complete. 2023-10-17T01:14:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.749+0000 7faa655ff700 1 Processor -- start 2023-10-17T01:14:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.749+0000 7faa655ff700 1 -- start start 2023-10-17T01:14:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa655ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa655ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101f30 0x7faa60073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa655ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa601028d0 0x7faa60073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa655ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa60073e20 con 0x7faa60100d40 2023-10-17T01:14:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa655ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa60073f90 con 0x7faa60101f30 2023-10-17T01:14:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa655ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa601a0b10 con 0x7faa601028d0 2023-10-17T01:14:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41636/0 (socket says 172.21.15.45:41636) 2023-10-17T01:14:30.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 -- 172.21.15.45:0/612282116 learned_addr learned my addr 172.21.15.45:0/612282116 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:30.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5e7fc700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101f30 0x7faa60073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5f7fe700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa601028d0 0x7faa60073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 -- 172.21.15.45:0/612282116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa601028d0 msgr2=0x7faa60073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa601028d0 0x7faa60073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 -- 172.21.15.45:0/612282116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101f30 msgr2=0x7faa60073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101f30 0x7faa60073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5effd700 1 -- 172.21.15.45:0/612282116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500095f0 con 0x7faa60100d40 2023-10-17T01:14:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.750+0000 7faa5f7fe700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa601028d0 0x7faa60073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.751+0000 7faa5effd700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60074b00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faa500098e0 tx=0x7faa5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.751+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5001d070 con 0x7faa60100d40 2023-10-17T01:14:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.751+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa50005510 con 0x7faa60100d40 2023-10-17T01:14:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.751+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa601a0d90 con 0x7faa60100d40 2023-10-17T01:14:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.751+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50004c00 con 0x7faa60100d40 2023-10-17T01:14:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.751+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa601a11b0 con 0x7faa60100d40 2023-10-17T01:14:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.752+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa60108320 con 0x7faa60100d40 2023-10-17T01:14:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.754+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7faa5000fb80 con 0x7faa60100d40 2023-10-17T01:14:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.755+0000 7faa47fff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faa480a0bb0 0x7faa480a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.755+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faa500cd200 con 0x7faa60100d40 2023-10-17T01:14:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.756+0000 7faa5e7fc700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faa480a0bb0 0x7faa480a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.756+0000 7faa5e7fc700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faa480a0bb0 0x7faa480a3070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa54007c60 tx=0x7faa540073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:30.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.759+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7faa50096a90 con 0x7faa60100d40 2023-10-17T01:14:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:30 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3325344324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:30 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/880423294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:30 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2070481771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:30 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3325344324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:30 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/880423294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:30 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2070481771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:30 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3325344324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:30.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:30 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/880423294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:30.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:30 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2070481771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.973+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa6002cea0 con 0x7faa60100d40 2023-10-17T01:14:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.973+0000 7faa47fff700 1 -- 172.21.15.45:0/612282116 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa5009a0b0 con 0x7faa60100d40 2023-10-17T01:14:30.975 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faa480a0bb0 msgr2=0x7faa480a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faa480a0bb0 0x7faa480a3070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa54007c60 tx=0x7faa540073d0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 msgr2=0x7faa60074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:30.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60074b00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faa500098e0 tx=0x7faa5000f690 comp rx=0 tx=0).stop 2023-10-17T01:14:30.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 shutdown_connections 2023-10-17T01:14:30.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faa480a0bb0 0x7faa480a3070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa601028d0 0x7faa60073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa60100d40 0x7faa60074b00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 --2- 172.21.15.45:0/612282116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101f30 0x7faa60073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 >> 172.21.15.45:0/612282116 conn(0x7faa600fc290 msgr2=0x7faa60105460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 shutdown_connections 2023-10-17T01:14:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:30.976+0000 7faa655ff700 1 -- 172.21.15.45:0/612282116 wait complete. 2023-10-17T01:14:30.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.199+0000 7fcb42dff700 1 -- 172.21.15.45:0/720207270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c100d50 msgr2=0x7fcb3c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.199+0000 7fcb42dff700 1 --2- 172.21.15.45:0/720207270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c100d50 0x7fcb3c101170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb300099c0 tx=0x7fcb30009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 -- 172.21.15.45:0/720207270 shutdown_connections 2023-10-17T01:14:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 --2- 172.21.15.45:0/720207270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c1028e0 0x7fcb3c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 --2- 172.21.15.45:0/720207270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c100d50 0x7fcb3c101170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 --2- 172.21.15.45:0/720207270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c101f40 0x7fcb3c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 -- 172.21.15.45:0/720207270 >> 172.21.15.45:0/720207270 conn(0x7fcb3c0fc280 msgr2=0x7fcb3c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 -- 172.21.15.45:0/720207270 shutdown_connections 2023-10-17T01:14:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.200+0000 7fcb42dff700 1 -- 172.21.15.45:0/720207270 wait complete. 2023-10-17T01:14:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 Processor -- start 2023-10-17T01:14:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 -- start start 2023-10-17T01:14:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 0x7fcb3c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101f40 0x7fcb3c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c1028e0 0x7fcb3c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19e570 con 0x7fcb3c100d50 2023-10-17T01:14:31.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19e6b0 con 0x7fcb3c1028e0 2023-10-17T01:14:31.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb42dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19e840 con 0x7fcb3c101f40 2023-10-17T01:14:31.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.201+0000 7fcb40b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 0x7fcb3c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 0x7fcb3c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41658/0 (socket says 172.21.15.45:41658) 2023-10-17T01:14:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 -- 172.21.15.45:0/4260385822 learned_addr learned my addr 172.21.15.45:0/4260385822 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb4139c700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c1028e0 0x7fcb3c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb3bfff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101f40 0x7fcb3c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 -- 172.21.15.45:0/4260385822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101f40 msgr2=0x7fcb3c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101f40 0x7fcb3c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 -- 172.21.15.45:0/4260385822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c1028e0 msgr2=0x7fcb3c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c1028e0 0x7fcb3c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 -- 172.21.15.45:0/4260385822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb300095f0 con 0x7fcb3c100d50 2023-10-17T01:14:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb3bfff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101f40 0x7fcb3c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb40b9b700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 0x7fcb3c1969f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009fd0 tx=0x7fcb30009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:31.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3001d070 con 0x7fcb3c100d50 2023-10-17T01:14:31.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb30005030 con 0x7fcb3c100d50 2023-10-17T01:14:31.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30004b30 con 0x7fcb3c100d50 2023-10-17T01:14:31.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.202+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c19eac0 con 0x7fcb3c100d50 2023-10-17T01:14:31.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.203+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c19ee90 con 0x7fcb3c100d50 2023-10-17T01:14:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.204+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c1082e0 con 0x7fcb3c100d50 2023-10-17T01:14:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.205+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fcb300051a0 con 0x7fcb3c100d50 2023-10-17T01:14:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.207+0000 7fcb39ffb700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb2c0a0bf0 0x7fcb2c0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.207+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcb300cd140 con 0x7fcb3c100d50 2023-10-17T01:14:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.207+0000 7fcb3bfff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb2c0a0bf0 0x7fcb2c0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.208+0000 7fcb3bfff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb2c0a0bf0 0x7fcb2c0a30b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009cb0 tx=0x7fcb28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.211+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fcb30096af0 con 0x7fcb3c100d50 2023-10-17T01:14:31.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.470+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb3c02cc30 con 0x7fcb3c100d50 2023-10-17T01:14:31.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.470+0000 7fcb39ffb700 1 -- 172.21.15.45:0/4260385822 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7fcb30026090 con 0x7fcb3c100d50 2023-10-17T01:14:31.472 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:31.472 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:31.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb2c0a0bf0 msgr2=0x7fcb2c0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb2c0a0bf0 0x7fcb2c0a30b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009cb0 tx=0x7fcb28009260 comp rx=0 tx=0).stop 2023-10-17T01:14:31.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 msgr2=0x7fcb3c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 0x7fcb3c1969f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009fd0 tx=0x7fcb30009580 comp rx=0 tx=0).stop 2023-10-17T01:14:31.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 shutdown_connections 2023-10-17T01:14:31.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb2c0a0bf0 0x7fcb2c0a30b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb3c1028e0 0x7fcb3c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c100d50 0x7fcb3c1969f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 --2- 172.21.15.45:0/4260385822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101f40 0x7fcb3c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 >> 172.21.15.45:0/4260385822 conn(0x7fcb3c0fc280 msgr2=0x7fcb3c105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 shutdown_connections 2023-10-17T01:14:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.473+0000 7fcb42dff700 1 -- 172.21.15.45:0/4260385822 wait complete. 2023-10-17T01:14:31.477 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:31.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:31.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 -- 172.21.15.45:0/2852053962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504100d50 msgr2=0x7fa504101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 --2- 172.21.15.45:0/2852053962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504100d50 0x7fa504101170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40099c0 tx=0x7fa4f4009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 -- 172.21.15.45:0/2852053962 shutdown_connections 2023-10-17T01:14:31.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 --2- 172.21.15.45:0/2852053962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5041028e0 0x7fa504109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 --2- 172.21.15.45:0/2852053962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504100d50 0x7fa504101170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 --2- 172.21.15.45:0/2852053962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504101f40 0x7fa5041023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 -- 172.21.15.45:0/2852053962 >> 172.21.15.45:0/2852053962 conn(0x7fa5040fc280 msgr2=0x7fa5040fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:31.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 -- 172.21.15.45:0/2852053962 shutdown_connections 2023-10-17T01:14:31.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.698+0000 7fa509fff700 1 -- 172.21.15.45:0/2852053962 wait complete. 2023-10-17T01:14:31.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.699+0000 7fa509fff700 1 Processor -- start 2023-10-17T01:14:31.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.699+0000 7fa509fff700 1 -- start start 2023-10-17T01:14:31.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa509fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504100d50 0x7fa504074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa509fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 0x7fa504073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa509fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5041028e0 0x7fa504073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa509fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa504073e20 con 0x7fa504101f40 2023-10-17T01:14:31.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa509fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa504073f90 con 0x7fa5041028e0 2023-10-17T01:14:31.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa509fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041a0aa0 con 0x7fa504100d50 2023-10-17T01:14:31.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa502ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 0x7fa504073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa5037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504100d50 0x7fa504074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa502ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 0x7fa504073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41686/0 (socket says 172.21.15.45:41686) 2023-10-17T01:14:31.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa502ffd700 1 -- 172.21.15.45:0/3929440834 learned_addr learned my addr 172.21.15.45:0/3929440834 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:31.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.700+0000 7fa503fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5041028e0 0x7fa504073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa5037fe700 1 -- 172.21.15.45:0/3929440834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5041028e0 msgr2=0x7fa504073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa5037fe700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5041028e0 0x7fa504073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa5037fe700 1 -- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 msgr2=0x7fa504073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa5037fe700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 0x7fa504073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa5037fe700 1 -- 172.21.15.45:0/3929440834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40095f0 con 0x7fa504100d50 2023-10-17T01:14:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa502ffd700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 0x7fa504073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa5037fe700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504100d50 0x7fa504074b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40058e0 tx=0x7fa4f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f401d070 con 0x7fa504100d50 2023-10-17T01:14:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4f4005020 con 0x7fa504100d50 2023-10-17T01:14:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5041a0d20 con 0x7fa504100d50 2023-10-17T01:14:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4004a90 con 0x7fa504100d50 2023-10-17T01:14:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.701+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5041a1140 con 0x7fa504100d50 2023-10-17T01:14:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.702+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50410c5b0 con 0x7fa504100d50 2023-10-17T01:14:31.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.704+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7fa4f4005400 con 0x7fa504100d50 2023-10-17T01:14:31.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.705+0000 7fa500ff9700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa4ec0a0c70 0x7fa4ec0a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:31.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.706+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa4f40cd4f0 con 0x7fa504100d50 2023-10-17T01:14:31.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.706+0000 7fa502ffd700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa4ec0a0c70 0x7fa4ec0a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:31.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.706+0000 7fa502ffd700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa4ec0a0c70 0x7fa4ec0a3130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8011550 tx=0x7fa4f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:31.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.708+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fa4f4096d80 con 0x7fa504100d50 2023-10-17T01:14:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:31 smithi045 ceph-mon[98474]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:31 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/612282116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:31 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/4260385822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:31 smithi060 ceph-mon[134136]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:31 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/612282116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:31 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/4260385822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:31 smithi181 ceph-mon[134520]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-17T01:14:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:31 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/612282116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:31 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/4260385822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:31.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.922+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa504107c70 con 0x7fa504100d50 2023-10-17T01:14:31.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.924+0000 7fa500ff9700 1 -- 172.21.15.45:0/3929440834 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7fa4f4005190 con 0x7fa504100d50 2023-10-17T01:14:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:31.925 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa4ec0a0c70 msgr2=0x7fa4ec0a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa4ec0a0c70 0x7fa4ec0a3130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8011550 tx=0x7fa4f8009260 comp rx=0 tx=0).stop 2023-10-17T01:14:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504100d50 msgr2=0x7fa504074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504100d50 0x7fa504074b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40058e0 tx=0x7fa4f400f690 comp rx=0 tx=0).stop 2023-10-17T01:14:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 shutdown_connections 2023-10-17T01:14:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa4ec0a0c70 0x7fa4ec0a3130 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5041028e0 0x7fa504073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa504100d50 0x7fa504074b00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 --2- 172.21.15.45:0/3929440834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa504101f40 0x7fa504073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 >> 172.21.15.45:0/3929440834 conn(0x7fa5040fc280 msgr2=0x7fa5040fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 shutdown_connections 2023-10-17T01:14:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:31.926+0000 7fa509fff700 1 -- 172.21.15.45:0/3929440834 wait complete. 2023-10-17T01:14:31.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:31.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 -- 172.21.15.45:0/2150956055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 msgr2=0x7f83c8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2150956055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 0x7f83c8102390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0098f0 tx=0x7f83bc009c00 comp rx=0 tx=0).stop 2023-10-17T01:14:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 -- 172.21.15.45:0/2150956055 shutdown_connections 2023-10-17T01:14:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2150956055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c81028d0 0x7f83c8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2150956055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2150956055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 0x7f83c8102390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 -- 172.21.15.45:0/2150956055 >> 172.21.15.45:0/2150956055 conn(0x7f83c80fc290 msgr2=0x7f83c80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:32.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 -- 172.21.15.45:0/2150956055 shutdown_connections 2023-10-17T01:14:32.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.153+0000 7f83ccdff700 1 -- 172.21.15.45:0/2150956055 wait complete. 2023-10-17T01:14:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.154+0000 7f83ccdff700 1 Processor -- start 2023-10-17T01:14:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.154+0000 7f83ccdff700 1 -- start start 2023-10-17T01:14:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83ccdff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83ccdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 0x7f83c81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83ccdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c81028d0 0x7f83c81a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83ccdff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a0820 con 0x7f83c8100d40 2023-10-17T01:14:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83ccdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a0990 con 0x7f83c81028d0 2023-10-17T01:14:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83ccdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a0b50 con 0x7f83c8101f30 2023-10-17T01:14:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41710/0 (socket says 172.21.15.45:41710) 2023-10-17T01:14:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 -- 172.21.15.45:0/2978899343 learned_addr learned my addr 172.21.15.45:0/2978899343 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c5d9b700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 0x7f83c81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c6d9d700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c81028d0 0x7f83c81a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 -- 172.21.15.45:0/2978899343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 msgr2=0x7f83c81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 0x7f83c81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 -- 172.21.15.45:0/2978899343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c81028d0 msgr2=0x7f83c81a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c81028d0 0x7f83c81a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.155+0000 7f83c659c700 1 -- 172.21.15.45:0/2978899343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc009520 con 0x7f83c8100d40 2023-10-17T01:14:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.156+0000 7f83c659c700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8198cb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e9f0 tx=0x7f83b800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.156+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004220 con 0x7f83c8100d40 2023-10-17T01:14:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.156+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b8004380 con 0x7f83c8100d40 2023-10-17T01:14:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.156+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c81a0e30 con 0x7f83c8100d40 2023-10-17T01:14:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.156+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8003be0 con 0x7f83c8100d40 2023-10-17T01:14:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.156+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c81a1250 con 0x7f83c8100d40 2023-10-17T01:14:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.157+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c810c5a0 con 0x7f83c8100d40 2023-10-17T01:14:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.159+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 135583+0+0 (secure 0 0 0) 0x7f83b8003d40 con 0x7f83c8100d40 2023-10-17T01:14:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.160+0000 7f83b77fe700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f83b00a0bb0 0x7f83b00a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.161+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f83b8014070 con 0x7f83c8100d40 2023-10-17T01:14:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.161+0000 7f83c5d9b700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f83b00a0bb0 0x7f83b00a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.161+0000 7f83c5d9b700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f83b00a0bb0 0x7f83b00a3070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005e90 tx=0x7f83bc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.164+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f83b8095bc0 con 0x7f83c8100d40 2023-10-17T01:14:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.381+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83c81a18c0 con 0x7f83c8100d40 2023-10-17T01:14:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.381+0000 7f83b77fe700 1 -- 172.21.15.45:0/2978899343 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83b80991e0 con 0x7f83c8100d40 2023-10-17T01:14:32.383 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.383+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f83b00a0bb0 msgr2=0x7f83b00a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f83b00a0bb0 0x7f83b00a3070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005e90 tx=0x7f83bc019040 comp rx=0 tx=0).stop 2023-10-17T01:14:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 msgr2=0x7f83c8198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8198cb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e9f0 tx=0x7f83b800ed00 comp rx=0 tx=0).stop 2023-10-17T01:14:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 shutdown_connections 2023-10-17T01:14:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f83b00a0bb0 0x7f83b00a3070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c81028d0 0x7f83c81a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8100d40 0x7f83c8198cb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 --2- 172.21.15.45:0/2978899343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83c8101f30 0x7f83c81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 >> 172.21.15.45:0/2978899343 conn(0x7f83c80fc290 msgr2=0x7f83c80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 shutdown_connections 2023-10-17T01:14:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.384+0000 7f83ccdff700 1 -- 172.21.15.45:0/2978899343 wait complete. 2023-10-17T01:14:32.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 -- 172.21.15.45:0/2252675003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0100d40 msgr2=0x7ff6f0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/2252675003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0100d40 0x7ff6f0101160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0099c0 tx=0x7ff6ec009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 -- 172.21.15.45:0/2252675003 shutdown_connections 2023-10-17T01:14:32.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/2252675003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6f01028d0 0x7ff6f0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/2252675003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0100d40 0x7ff6f0101160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/2252675003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f0101f30 0x7ff6f0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 -- 172.21.15.45:0/2252675003 >> 172.21.15.45:0/2252675003 conn(0x7ff6f00fc290 msgr2=0x7ff6f00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:32.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 -- 172.21.15.45:0/2252675003 shutdown_connections 2023-10-17T01:14:32.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.647+0000 7ff6f83ff700 1 -- 172.21.15.45:0/2252675003 wait complete. 2023-10-17T01:14:32.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.648+0000 7ff6f83ff700 1 Processor -- start 2023-10-17T01:14:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 -- start start 2023-10-17T01:14:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f0100d40 0x7ff6f0196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 0x7ff6f0196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6f01028d0 0x7ff6f019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f019e630 con 0x7ff6f0101f30 2023-10-17T01:14:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f019e770 con 0x7ff6f01028d0 2023-10-17T01:14:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f83ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f019e900 con 0x7ff6f0100d40 2023-10-17T01:14:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f599a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 0x7ff6f0196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f599a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 0x7ff6f0196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41734/0 (socket says 172.21.15.45:41734) 2023-10-17T01:14:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f599a700 1 -- 172.21.15.45:0/1903656692 learned_addr learned my addr 172.21.15.45:0/1903656692 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f619b700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f0100d40 0x7ff6f0196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.649+0000 7ff6f699c700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6f01028d0 0x7ff6f019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f599a700 1 -- 172.21.15.45:0/1903656692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f0100d40 msgr2=0x7ff6f0196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f599a700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f0100d40 0x7ff6f0196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f599a700 1 -- 172.21.15.45:0/1903656692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6f01028d0 msgr2=0x7ff6f019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f599a700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6f01028d0 0x7ff6f019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f599a700 1 -- 172.21.15.45:0/1903656692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0095f0 con 0x7ff6f0101f30 2023-10-17T01:14:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f599a700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 0x7ff6f0196ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000e9a0 tx=0x7ff6e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0009790 con 0x7ff6f0101f30 2023-10-17T01:14:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6e000cbd0 con 0x7ff6f0101f30 2023-10-17T01:14:32.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f019ebe0 con 0x7ff6f0101f30 2023-10-17T01:14:32.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0018680 con 0x7ff6f0101f30 2023-10-17T01:14:32.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.650+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f019f140 con 0x7ff6f0101f30 2023-10-17T01:14:32.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.651+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f0108320 con 0x7ff6f0101f30 2023-10-17T01:14:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.653+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff6e000cd40 con 0x7ff6f0101f30 2023-10-17T01:14:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.656+0000 7ff6e77fe700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff6dc0a0ba0 0x7ff6dc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.656+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff6e0014070 con 0x7ff6f0101f30 2023-10-17T01:14:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.656+0000 7ff6f619b700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff6dc0a0ba0 0x7ff6dc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.656+0000 7ff6f619b700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff6dc0a0ba0 0x7ff6dc0a3060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009990 tx=0x7ff6ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:32.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.660+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7ff6e00520b0 con 0x7ff6f0101f30 2023-10-17T01:14:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:32 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3929440834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:32 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2978899343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:32 smithi045 ceph-mon[98474]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:32 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3929440834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:32.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:32 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2978899343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:32.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:32 smithi060 ceph-mon[134136]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.872+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6f0061a00 con 0x7ff6f0101f30 2023-10-17T01:14:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.873+0000 7ff6e77fe700 1 -- 172.21.15.45:0/1903656692 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6e0095d10 con 0x7ff6f0101f30 2023-10-17T01:14:32.874 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff6dc0a0ba0 msgr2=0x7ff6dc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff6dc0a0ba0 0x7ff6dc0a3060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009990 tx=0x7ff6ec005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 msgr2=0x7ff6f0196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 0x7ff6f0196ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000e9a0 tx=0x7ff6e000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 shutdown_connections 2023-10-17T01:14:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff6dc0a0ba0 0x7ff6dc0a3060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6f01028d0 0x7ff6f019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6f0100d40 0x7ff6f0196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 --2- 172.21.15.45:0/1903656692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f0101f30 0x7ff6f0196ff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.875+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 >> 172.21.15.45:0/1903656692 conn(0x7ff6f00fc290 msgr2=0x7ff6f0105460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.876+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 shutdown_connections 2023-10-17T01:14:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:32.876+0000 7ff6f83ff700 1 -- 172.21.15.45:0/1903656692 wait complete. 2023-10-17T01:14:32.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:32 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3929440834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:32 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2978899343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:32 smithi181 ceph-mon[134520]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-17T01:14:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 -- 172.21.15.45:0/3516227743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a8100d60 msgr2=0x7f86a8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 --2- 172.21.15.45:0/3516227743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a8100d60 0x7f86a8101180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f86a40099c0 tx=0x7f86a4009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 -- 172.21.15.45:0/3516227743 shutdown_connections 2023-10-17T01:14:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 --2- 172.21.15.45:0/3516227743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a81028f0 0x7f86a81091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 --2- 172.21.15.45:0/3516227743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a8100d60 0x7f86a8101180 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 --2- 172.21.15.45:0/3516227743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a8101f50 0x7f86a81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 -- 172.21.15.45:0/3516227743 >> 172.21.15.45:0/3516227743 conn(0x7f86a80fc2d0 msgr2=0x7f86a80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:33.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 -- 172.21.15.45:0/3516227743 shutdown_connections 2023-10-17T01:14:33.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.102+0000 7f86afbff700 1 -- 172.21.15.45:0/3516227743 wait complete. 2023-10-17T01:14:33.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.103+0000 7f86afbff700 1 Processor -- start 2023-10-17T01:14:33.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.103+0000 7f86afbff700 1 -- start start 2023-10-17T01:14:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86afbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a8100d60 0x7f86a8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86afbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a8101f50 0x7f86a81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86afbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a81028f0 0x7f86a81a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86afbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81a0820 con 0x7f86a81028f0 2023-10-17T01:14:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86afbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81a0990 con 0x7f86a8101f50 2023-10-17T01:14:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86afbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81a0b50 con 0x7f86a8100d60 2023-10-17T01:14:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ad99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a8100d60 0x7f86a8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ad99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a8100d60 0x7f86a8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35976/0 (socket says 172.21.15.45:35976) 2023-10-17T01:14:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ad99b700 1 -- 172.21.15.45:0/3421726631 learned_addr learned my addr 172.21.15.45:0/3421726631 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ae19c700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a81028f0 0x7f86a81a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ad19a700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a8101f50 0x7f86a81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ae19c700 1 -- 172.21.15.45:0/3421726631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a8100d60 msgr2=0x7f86a8198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ae19c700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a8100d60 0x7f86a8198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ae19c700 1 -- 172.21.15.45:0/3421726631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a8101f50 msgr2=0x7f86a81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ae19c700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a8101f50 0x7f86a81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.104+0000 7f86ae19c700 1 -- 172.21.15.45:0/3421726631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40095f0 con 0x7f86a81028f0 2023-10-17T01:14:33.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.105+0000 7f86ae19c700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a81028f0 0x7f86a81a0250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86a0007f10 tx=0x7f86a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:33.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.105+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0009ea0 con 0x7f86a81028f0 2023-10-17T01:14:33.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.105+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86a000f040 con 0x7f86a81028f0 2023-10-17T01:14:33.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.105+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a81a0e30 con 0x7f86a81028f0 2023-10-17T01:14:33.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.105+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0004350 con 0x7f86a81028f0 2023-10-17T01:14:33.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.105+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a81a1300 con 0x7f86a81028f0 2023-10-17T01:14:33.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.106+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a81933b0 con 0x7f86a81028f0 2023-10-17T01:14:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.108+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f86a000db20 con 0x7f86a81028f0 2023-10-17T01:14:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.109+0000 7f869effd700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f86940a0ba0 0x7f86940a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.109+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f86a00cb970 con 0x7f86a81028f0 2023-10-17T01:14:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.109+0000 7f86ad99b700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f86940a0ba0 0x7f86940a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.110+0000 7f86ad99b700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f86940a0ba0 0x7f86940a3060 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000c00 tx=0x7f86a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.113+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f86a00952d0 con 0x7f86a81028f0 2023-10-17T01:14:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.325+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86a8061a00 con 0x7f86a81028f0 2023-10-17T01:14:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.325+0000 7f869effd700 1 -- 172.21.15.45:0/3421726631 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86a0019020 con 0x7f86a81028f0 2023-10-17T01:14:33.327 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f86940a0ba0 msgr2=0x7f86940a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f86940a0ba0 0x7f86940a3060 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000c00 tx=0x7f86a4005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a81028f0 msgr2=0x7f86a81a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a81028f0 0x7f86a81a0250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86a0007f10 tx=0x7f86a000d3b0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 shutdown_connections 2023-10-17T01:14:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f86940a0ba0 0x7f86940a3060 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86a81028f0 0x7f86a81a0250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86a8100d60 0x7f86a8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 --2- 172.21.15.45:0/3421726631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a8101f50 0x7f86a81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 >> 172.21.15.45:0/3421726631 conn(0x7f86a80fc2d0 msgr2=0x7f86a8107870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 shutdown_connections 2023-10-17T01:14:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.328+0000 7f86afbff700 1 -- 172.21.15.45:0/3421726631 wait complete. 2023-10-17T01:14:33.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 -- 172.21.15.45:0/3349405562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 msgr2=0x7f4988101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 --2- 172.21.15.45:0/3349405562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 0x7f4988101180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49780099c0 tx=0x7f4978009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 -- 172.21.15.45:0/3349405562 shutdown_connections 2023-10-17T01:14:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 --2- 172.21.15.45:0/3349405562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 --2- 172.21.15.45:0/3349405562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 0x7f4988101180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 --2- 172.21.15.45:0/3349405562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4988101f50 0x7f49881023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 -- 172.21.15.45:0/3349405562 >> 172.21.15.45:0/3349405562 conn(0x7f49880fc290 msgr2=0x7f49880fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.553+0000 7f498dbff700 1 -- 172.21.15.45:0/3349405562 shutdown_connections 2023-10-17T01:14:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.554+0000 7f498dbff700 1 -- 172.21.15.45:0/3349405562 wait complete. 2023-10-17T01:14:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.554+0000 7f498dbff700 1 Processor -- start 2023-10-17T01:14:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 -- start start 2023-10-17T01:14:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 0x7f4988198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4988101f50 0x7f49881991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49881a07a0 con 0x7f49881028f0 2023-10-17T01:14:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49881a0910 con 0x7f4988100d60 2023-10-17T01:14:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f498dbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49881a0ad0 con 0x7f4988101f50 2023-10-17T01:14:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f4987fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f4987fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41776/0 (socket says 172.21.15.45:41776) 2023-10-17T01:14:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f4987fff700 1 -- 172.21.15.45:0/3800149440 learned_addr learned my addr 172.21.15.45:0/3800149440 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:33.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f49877fe700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 0x7f4988198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.555+0000 7f4986ffd700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4988101f50 0x7f49881991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4987fff700 1 -- 172.21.15.45:0/3800149440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4988101f50 msgr2=0x7f49881991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4987fff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4988101f50 0x7f49881991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4987fff700 1 -- 172.21.15.45:0/3800149440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 msgr2=0x7f4988198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4987fff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 0x7f4988198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4987fff700 1 -- 172.21.15.45:0/3800149440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980009c40 con 0x7f49881028f0 2023-10-17T01:14:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4987fff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881a01d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f498000c370 tx=0x7f498000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980017070 con 0x7f49881028f0 2023-10-17T01:14:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f498000f040 con 0x7f49881028f0 2023-10-17T01:14:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49780095f0 con 0x7f49881028f0 2023-10-17T01:14:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000e050 con 0x7f49881028f0 2023-10-17T01:14:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.556+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49881a10d0 con 0x7f49881028f0 2023-10-17T01:14:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.557+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49881933b0 con 0x7f49881028f0 2023-10-17T01:14:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.559+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f498000cb60 con 0x7f49881028f0 2023-10-17T01:14:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.561+0000 7f4984ff9700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f49700a0b70 0x7f49700a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.561+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f49800cbb00 con 0x7f49881028f0 2023-10-17T01:14:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.561+0000 7f49877fe700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f49700a0b70 0x7f49700a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.561+0000 7f49877fe700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f49700a0b70 0x7f49700a3030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f498819a130 tx=0x7f4978005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.564+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f49800cde60 con 0x7f49881028f0 2023-10-17T01:14:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:33 smithi045 ceph-mon[98474]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-17T01:14:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:33 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1903656692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:33 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3421726631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:33 smithi060 ceph-mon[134136]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-17T01:14:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:33 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1903656692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:33 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3421726631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:33.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.827+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f498802cc30 con 0x7f49881028f0 2023-10-17T01:14:33.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.828+0000 7f4984ff9700 1 -- 172.21.15.45:0/3800149440 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f4980095460 con 0x7f49881028f0 2023-10-17T01:14:33.830 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:33.830 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f49700a0b70 msgr2=0x7f49700a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f49700a0b70 0x7f49700a3030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f498819a130 tx=0x7f4978005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 msgr2=0x7f49881a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881a01d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f498000c370 tx=0x7f498000c680 comp rx=0 tx=0).stop 2023-10-17T01:14:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 shutdown_connections 2023-10-17T01:14:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f49700a0b70 0x7f49700a3030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49881028f0 0x7f49881a01d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988100d60 0x7f4988198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 --2- 172.21.15.45:0/3800149440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4988101f50 0x7f49881991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 >> 172.21.15.45:0/3800149440 conn(0x7f49880fc290 msgr2=0x7f4988107840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 shutdown_connections 2023-10-17T01:14:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:33.831+0000 7f498dbff700 1 -- 172.21.15.45:0/3800149440 wait complete. 2023-10-17T01:14:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:33.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:33 smithi181 ceph-mon[134520]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-17T01:14:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:33 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1903656692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:33 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3421726631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.054+0000 7fb7e4dff700 1 -- 172.21.15.45:0/1442682631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e0100d50 msgr2=0x7fb7e0101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.054+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/1442682631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e0100d50 0x7fb7e0101170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00099c0 tx=0x7fb7d0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 -- 172.21.15.45:0/1442682631 shutdown_connections 2023-10-17T01:14:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/1442682631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e01028e0 0x7fb7e0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/1442682631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e0100d50 0x7fb7e0101170 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/1442682631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0101f40 0x7fb7e01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 -- 172.21.15.45:0/1442682631 >> 172.21.15.45:0/1442682631 conn(0x7fb7e00fc280 msgr2=0x7fb7e00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 -- 172.21.15.45:0/1442682631 shutdown_connections 2023-10-17T01:14:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.055+0000 7fb7e4dff700 1 -- 172.21.15.45:0/1442682631 wait complete. 2023-10-17T01:14:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 Processor -- start 2023-10-17T01:14:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 -- start start 2023-10-17T01:14:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0100d50 0x7fb7e0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e0101f40 0x7fb7e01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 0x7fb7e01a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e01a07e0 con 0x7fb7e01028e0 2023-10-17T01:14:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e01a0950 con 0x7fb7e0100d50 2023-10-17T01:14:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.056+0000 7fb7e4dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e01a0b10 con 0x7fb7e0101f40 2023-10-17T01:14:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 0x7fb7e01a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7de59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0100d50 0x7fb7e0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 0x7fb7e01a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41782/0 (socket says 172.21.15.45:41782) 2023-10-17T01:14:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 -- 172.21.15.45:0/2679249322 learned_addr learned my addr 172.21.15.45:0/2679249322 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ddd9b700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e0101f40 0x7fb7e01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 -- 172.21.15.45:0/2679249322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e0101f40 msgr2=0x7fb7e01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e0101f40 0x7fb7e01991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 -- 172.21.15.45:0/2679249322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0100d50 msgr2=0x7fb7e0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0100d50 0x7fb7e0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 -- 172.21.15.45:0/2679249322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d00095f0 con 0x7fb7e01028e0 2023-10-17T01:14:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7de59c700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0100d50 0x7fb7e0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.057+0000 7fb7ded9d700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 0x7fb7e01a0210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800ef30 tx=0x7fb7d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.058+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8018070 con 0x7fb7e01028e0 2023-10-17T01:14:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.058+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d800f040 con 0x7fb7e01028e0 2023-10-17T01:14:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.058+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800cd00 con 0x7fb7e01028e0 2023-10-17T01:14:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.058+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e01a0d90 con 0x7fb7e01028e0 2023-10-17T01:14:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.058+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e01a1210 con 0x7fb7e01028e0 2023-10-17T01:14:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.059+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e010c5b0 con 0x7fb7e01028e0 2023-10-17T01:14:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.061+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb7d8003a00 con 0x7fb7e01028e0 2023-10-17T01:14:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.062+0000 7fb7cf7fe700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb7c80a0bf0 0x7fb7c80a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.062+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb7d80cbbf0 con 0x7fb7e01028e0 2023-10-17T01:14:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.062+0000 7fb7de59c700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb7c80a0bf0 0x7fb7c80a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.063+0000 7fb7de59c700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb7c80a0bf0 0x7fb7c80a30b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009990 tx=0x7fb7d001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.066+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb7d80955e0 con 0x7fb7e01028e0 2023-10-17T01:14:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.279+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7e010c770 con 0x7fb7e01028e0 2023-10-17T01:14:34.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.281+0000 7fb7cf7fe700 1 -- 172.21.15.45:0/2679249322 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7fb7d8098c00 con 0x7fb7e01028e0 2023-10-17T01:14:34.283 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:34.283 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:34.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb7c80a0bf0 msgr2=0x7fb7c80a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb7c80a0bf0 0x7fb7c80a30b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009990 tx=0x7fb7d001a040 comp rx=0 tx=0).stop 2023-10-17T01:14:34.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 msgr2=0x7fb7e01a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 0x7fb7e01a0210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800ef30 tx=0x7fb7d800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 shutdown_connections 2023-10-17T01:14:34.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb7c80a0bf0 0x7fb7c80a30b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e01028e0 0x7fb7e01a0210 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e0100d50 0x7fb7e0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 --2- 172.21.15.45:0/2679249322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e0101f40 0x7fb7e01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 >> 172.21.15.45:0/2679249322 conn(0x7fb7e00fc280 msgr2=0x7fb7e00fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 shutdown_connections 2023-10-17T01:14:34.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.283+0000 7fb7e4dff700 1 -- 172.21.15.45:0/2679249322 wait complete. 2023-10-17T01:14:34.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-17T01:13:01.038089+0000', 'modified': '2023-10-17T01:13:08.449854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24587, 'mds_1': 24550, 'mds_2': 24539, 'mds_3': 24526, 'mds_4': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14544': {'gid': 14544, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/931419854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 931419854}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 931419854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14556': {'gid': 14556, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6841/1667366220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 1667366220}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 1667366220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6833/2648148860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 2648148860}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 2648148860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24473': {'gid': 24473, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6833/3345979209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 3345979209}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 3345979209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/1725566254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1725566254}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1725566254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6835/1888494346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1888494346}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 1888494346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.181:6837/1396793294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1396793294}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1396793294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.60:6837/1246290891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6836', 'nonce': 1246290891}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 1246290891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6839/1273587217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 1273587217}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 1273587217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6839/165785544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6838', 'nonce': 165785544}, {'type': 'v1', 'addr': '172.21.15.60:6839', 'nonce': 165785544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-17T01:14:34.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-17T01:14:34.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-17T01:14:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 -- 172.21.15.45:0/3602932857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20101f50 msgr2=0x7f4b201023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3602932857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20101f50 0x7f4b201023b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b140099c0 tx=0x7f4b14009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 -- 172.21.15.45:0/3602932857 shutdown_connections 2023-10-17T01:14:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3602932857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b201091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3602932857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b20100d60 0x7f4b20101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3602932857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20101f50 0x7f4b201023b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 -- 172.21.15.45:0/3602932857 >> 172.21.15.45:0/3602932857 conn(0x7f4b200fc2d0 msgr2=0x7f4b200fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 -- 172.21.15.45:0/3602932857 shutdown_connections 2023-10-17T01:14:34.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.511+0000 7f4b249ff700 1 -- 172.21.15.45:0/3602932857 wait complete. 2023-10-17T01:14:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.512+0000 7f4b249ff700 1 Processor -- start 2023-10-17T01:14:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 -- start start 2023-10-17T01:14:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20100d60 0x7f4b20074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b20101f50 0x7f4b20073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b20073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20073e20 con 0x7f4b201028f0 2023-10-17T01:14:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20073f90 con 0x7f4b20100d60 2023-10-17T01:14:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b249ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201a0ac0 con 0x7f4b20101f50 2023-10-17T01:14:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b20073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b20073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41810/0 (socket says 172.21.15.45:41810) 2023-10-17T01:14:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/3595650394 learned_addr learned my addr 172.21.15.45:0/3595650394 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20100d60 0x7f4b20074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.513+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b20101f50 0x7f4b20073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/3595650394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b20101f50 msgr2=0x7f4b20073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b20101f50 0x7f4b20073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/3595650394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20100d60 msgr2=0x7f4b20074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20100d60 0x7f4b20074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/3595650394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b140095f0 con 0x7f4b201028f0 2023-10-17T01:14:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b20073680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800ef30 tx=0x7f4b1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b0f7fe700 1 -- 172.21.15.45:0/3595650394 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18018070 con 0x7f4b201028f0 2023-10-17T01:14:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b0f7fe700 1 -- 172.21.15.45:0/3595650394 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b1800f040 con 0x7f4b201028f0 2023-10-17T01:14:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b201a0d40 con 0x7f4b201028f0 2023-10-17T01:14:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b0f7fe700 1 -- 172.21.15.45:0/3595650394 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800cd00 con 0x7f4b201028f0 2023-10-17T01:14:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.514+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201a11f0 con 0x7f4b201028f0 2023-10-17T01:14:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.517+0000 7f4b0f7fe700 1 -- 172.21.15.45:0/3595650394 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f4b18003a00 con 0x7f4b201028f0 2023-10-17T01:14:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 Processor -- start 2023-10-17T01:14:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b0f7fe700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b080a0b20 0x7f4b080a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 -- start start 2023-10-17T01:14:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b0f7fe700 1 -- 172.21.15.45:0/3595650394 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4b180cbc70 con 0x7f4b201028f0 2023-10-17T01:14:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b080a0b20 0x7f4b080a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 0x558023fbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fbccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fbd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023f139c0 con 0x558023fbd260 2023-10-17T01:14:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023f11380 con 0x558023fbc4b0 2023-10-17T01:14:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023f0cc70 con 0x558023fbbb50 2023-10-17T01:14:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.519+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fbd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 0x558023fbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 0x558023fbbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36034/0 (socket says 172.21.15.45:36034) 2023-10-17T01:14:34.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 -- 172.21.15.45:0/3850169726 learned_addr learned my addr 172.21.15.45:0/3850169726 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fbd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41812/0 (socket says 172.21.15.45:41812) 2023-10-17T01:14:34.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fbccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b080a0b20 0x7f4b080a2fe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400b580 tx=0x7f4b1400b8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 -- 172.21.15.45:0/3850169726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 msgr2=0x558023fbccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fbccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 -- 172.21.15.45:0/3850169726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 msgr2=0x558023fbd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fbd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 -- 172.21.15.45:0/3850169726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558023d3ad90 con 0x558023fbbb50 2023-10-17T01:14:34.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fbd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 0x558023fbbf70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009d30 tx=0x7f4b10011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41cdac14c4838585 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.520+0000 7f4b0cff9700 1 -- 172.21.15.45:0/3850169726 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10004220 con 0x558023fbbb50 2023-10-17T01:14:34.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b0cff9700 1 -- 172.21.15.45:0/3850169726 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b10004380 con 0x558023fbbb50 2023-10-17T01:14:34.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b0cff9700 1 -- 172.21.15.45:0/3850169726 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b100198d0 con 0x558023fbbb50 2023-10-17T01:14:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3850169726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 msgr2=0x558023fbbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 0x558023fbbf70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009d30 tx=0x7f4b10011720 comp rx=0 tx=0).stop 2023-10-17T01:14:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3850169726 shutdown_connections 2023-10-17T01:14:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fbd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbbb50 0x558023fbbf70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3850169726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fbccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3850169726 >> 172.21.15.45:0/3850169726 conn(0x558023f1b080 msgr2=0x558023fb6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3850169726 shutdown_connections 2023-10-17T01:14:34.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.521+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3850169726 wait complete. 2023-10-17T01:14:34.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.522+0000 7f4b383d1b80 1 -- start start 2023-10-17T01:14:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.522+0000 7f4b383d1b80 1 Processor -- start 2023-10-17T01:14:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 0x558023fc5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fc5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbd260 0x558023fccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023ea94c0 con 0x558023fbbb50 2023-10-17T01:14:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023ea9160 con 0x558023fbc4b0 2023-10-17T01:14:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023d908d0 con 0x558023fbd260 2023-10-17T01:14:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b383d1b80 10 client.0 _ll_register_callbacks cb 0x558023df6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 0x558023fc5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 0x558023fc5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41818/0 (socket says 172.21.15.45:41818) 2023-10-17T01:14:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbd260 0x558023fccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b1e59c700 1 -- 172.21.15.45:0/2071704790 learned_addr learned my addr 172.21.15.45:0/2071704790 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.523+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fc5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b1e59c700 1 -- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbd260 msgr2=0x558023fccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbd260 0x558023fccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b1e59c700 1 -- 172.21.15.45:0/2071704790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 msgr2=0x558023fc5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fc5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b1e59c700 1 -- 172.21.15.45:0/2071704790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201a11f0 con 0x558023fbbb50 2023-10-17T01:14:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 0x558023fc5860 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006010 tx=0x7f4b10003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e9f789dd4de15d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b05ffb700 1 -- 172.21.15.45:0/2071704790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10003a90 con 0x558023fbbb50 2023-10-17T01:14:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b05ffb700 1 -- 172.21.15.45:0/2071704790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4b10005040 con 0x558023fbbb50 2023-10-17T01:14:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.524+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558023f37cc0 con 0x558023fbbb50 2023-10-17T01:14:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b05ffb700 1 -- 172.21.15.45:0/2071704790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b10005200 con 0x558023fbbb50 2023-10-17T01:14:34.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 10 client.15150 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:34.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f4b10009960 con 0x558023fbbb50 2023-10-17T01:14:34.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b05ffb700 1 -- 172.21.15.45:0/2071704790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558023f37cc0 con 0x558023fbbb50 2023-10-17T01:14:34.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b05ffb700 1 -- 172.21.15.45:0/2071704790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4b10021040 con 0x558023fbbb50 2023-10-17T01:14:34.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 10 client.15150 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-17T01:14:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 20 client.15150 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558023fd09a0 0x558023fcfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 4 client.15150 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.525+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 --> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558023f8ace0 con 0x558023fd09a0 2023-10-17T01:14:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.526+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558023fd09a0 0x558023fcfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.526+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558023fd09a0 0x558023fcfe30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1178fa43f547c7d server_cookie=c1328d0336c8a287 in_seq=0 out_seq=0 2023-10-17T01:14:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.526+0000 7f4b05ffb700 10 client.15150 ms_handle_connect on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b05ffb700 1 -- 172.21.15.45:0/2071704790 <== mds.4 v2:172.21.15.181:6834/1888494346 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x558023f8ace0 con 0x558023fd09a0 2023-10-17T01:14:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b05ffb700 10 client.15150 handle_command_reply: tid=0 2023-10-17T01:14:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 client.15150 shutdown 2023-10-17T01:14:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 msgr2=0x558023fc5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 0x558023fc5860 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006010 tx=0x7f4b10003de0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 shutdown_connections 2023-10-17T01:14:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558023fd09a0 0x558023fcfe30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbd260 0x558023fccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbbb50 0x558023fc5860 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b05ffb700 0 client.15150 ms_handle_reset on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/2071704790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbc4b0 0x558023fc5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 >> 172.21.15.45:0/2071704790 conn(0x558023f1b080 msgr2=0x558023fb5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.527+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 shutdown_connections 2023-10-17T01:14:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.528+0000 7f4b383d1b80 1 -- 172.21.15.45:0/2071704790 wait complete. 2023-10-17T01:14:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.528+0000 7f4b383d1b80 20 client.15150 trim_cache size 0 max 16384 2023-10-17T01:14:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 Processor -- start 2023-10-17T01:14:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 -- start start 2023-10-17T01:14:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fb6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fcfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023eb6830 con 0x558023fbd260 2023-10-17T01:14:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023ed70b0 con 0x558023fbbb50 2023-10-17T01:14:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023f11cf0 con 0x558023fbc4b0 2023-10-17T01:14:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.547+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fcfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fcfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41830/0 (socket says 172.21.15.45:41830) 2023-10-17T01:14:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/159187999 learned_addr learned my addr 172.21.15.45:0/159187999 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fb6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/159187999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 msgr2=0x558023fc4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fc4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/159187999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 msgr2=0x558023fb6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fb6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 -- 172.21.15.45:0/159187999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558023d23c10 con 0x558023fbd260 2023-10-17T01:14:34.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.548+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fcfe30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4b180090a0 tx=0x7f4b18084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471572140594854b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.551+0000 7f4b0d7fa700 1 -- 172.21.15.45:0/159187999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18004ad0 con 0x558023fbd260 2023-10-17T01:14:34.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.551+0000 7f4b0d7fa700 1 -- 172.21.15.45:0/159187999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b1809de90 con 0x558023fbd260 2023-10-17T01:14:34.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.551+0000 7f4b0d7fa700 1 -- 172.21.15.45:0/159187999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1809d150 con 0x558023fbd260 2023-10-17T01:14:34.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 -- 172.21.15.45:0/159187999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 msgr2=0x558023fcfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fcfe30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4b180090a0 tx=0x7f4b18084040 comp rx=0 tx=0).stop 2023-10-17T01:14:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 -- 172.21.15.45:0/159187999 shutdown_connections 2023-10-17T01:14:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x558023fcfe30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/159187999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fc4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 -- 172.21.15.45:0/159187999 >> 172.21.15.45:0/159187999 conn(0x558023f1b080 msgr2=0x558023fb5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 -- 172.21.15.45:0/159187999 shutdown_connections 2023-10-17T01:14:34.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.552+0000 7f4b383d1b80 1 -- 172.21.15.45:0/159187999 wait complete. 2023-10-17T01:14:34.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.553+0000 7f4b383d1b80 1 -- start start 2023-10-17T01:14:34.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.553+0000 7f4b383d1b80 1 Processor -- start 2023-10-17T01:14:34.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fd4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fd4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x5580240222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023fd5540 con 0x558023fbd260 2023-10-17T01:14:34.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024022830 con 0x558023fbbb50 2023-10-17T01:14:34.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580240229a0 con 0x558023fbc4b0 2023-10-17T01:14:34.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b383d1b80 10 client.0 _ll_register_callbacks cb 0x7f4b180093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:34.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x5580240222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fd4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fd4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fd4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36048/0 (socket says 172.21.15.45:36048) 2023-10-17T01:14:34.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fd4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53908/0 (socket says 172.21.15.45:53908) 2023-10-17T01:14:34.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x5580240222f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41834/0 (socket says 172.21.15.45:41834) 2023-10-17T01:14:34.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 -- 172.21.15.45:0/3164381008 learned_addr learned my addr 172.21.15.45:0/3164381008 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 -- 172.21.15.45:0/3164381008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 msgr2=0x558023fd4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fd4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 -- 172.21.15.45:0/3164381008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 msgr2=0x5580240222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x5580240222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.554+0000 7f4b1dd9b700 1 -- 172.21.15.45:0/3164381008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201a0d40 con 0x558023fbc4b0 2023-10-17T01:14:34.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fd4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b1ed9d700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x5580240222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b1dd9b700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fd4ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4b14006010 tx=0x7f4b14027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaeafc47a70a8e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:14:34.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b05ffb700 1 -- 172.21.15.45:0/3164381008 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14005390 con 0x558023fbc4b0 2023-10-17T01:14:34.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b10004380 con 0x558023fbc4b0 2023-10-17T01:14:34.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b05ffb700 1 -- 172.21.15.45:0/3164381008 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4b140054f0 con 0x558023fbc4b0 2023-10-17T01:14:34.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b05ffb700 1 -- 172.21.15.45:0/3164381008 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b140056b0 con 0x558023fbc4b0 2023-10-17T01:14:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b383d1b80 10 client.25177 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.555+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558023d61200 con 0x558023fbc4b0 2023-10-17T01:14:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b05ffb700 1 -- 172.21.15.45:0/3164381008 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b10004380 con 0x558023fbc4b0 2023-10-17T01:14:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b05ffb700 1 -- 172.21.15.45:0/3164381008 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f4b1404a550 con 0x558023fbc4b0 2023-10-17T01:14:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b383d1b80 10 client.25177 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b383d1b80 10 client.25177 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-17T01:14:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b383d1b80 20 client.25177 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558024027790 0x5580240240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b383d1b80 4 client.25177 mds_command: new command op to 24514 tid=0[{"prefix": "status"}] 2023-10-17T01:14:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 --> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558023f05cf0 con 0x558024027790 2023-10-17T01:14:34.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.556+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558024027790 0x5580240240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b1e59c700 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558024027790 0x5580240240e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d77f7811054cd573 server_cookie=5d4e10b56b1f1b45 in_seq=0 out_seq=0 2023-10-17T01:14:34.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b05ffb700 10 client.25177 ms_handle_connect on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:34.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b05ffb700 1 -- 172.21.15.45:0/3164381008 <== mds.4 v2:172.21.15.181:6834/1888494346 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558023f05cf0 con 0x558024027790 2023-10-17T01:14:34.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b05ffb700 10 client.25177 handle_command_reply: tid=0 2023-10-17T01:14:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 client.25177 shutdown 2023-10-17T01:14:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 msgr2=0x558023fd4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fd4ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4b14006010 tx=0x7f4b14027000 comp rx=0 tx=0).stop 2023-10-17T01:14:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 shutdown_connections 2023-10-17T01:14:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x558024027790 0x5580240240e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558023fbd260 0x5580240222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b05ffb700 0 client.25177 ms_handle_reset on v2:172.21.15.181:6834/1888494346 2023-10-17T01:14:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558023fbbb50 0x558023fd4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 --2- 172.21.15.45:0/3164381008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558023fbc4b0 0x558023fd4ea0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 >> 172.21.15.45:0/3164381008 conn(0x558023f1b080 msgr2=0x558023fb6810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.557+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 shutdown_connections 2023-10-17T01:14:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b383d1b80 1 -- 172.21.15.45:0/3164381008 wait complete. 2023-10-17T01:14:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b383d1b80 20 client.25177 trim_cache size 0 max 16384 2023-10-17T01:14:34.571 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:34.572 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:34.572 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 4, 2023-10-17T01:14:34.572 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24514, 2023-10-17T01:14:34.572 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:34.572 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:34.573 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:34.573 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 87.109710794999998, 2023-10-17T01:14:34.573 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:34.573 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:34.573 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:34.574 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 114.472723307 2023-10-17T01:14:34.574 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b080a0b20 msgr2=0x7f4b080a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b080a0b20 0x7f4b080a2fe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400b580 tx=0x7f4b1400b8d0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 msgr2=0x7f4b20073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b20073680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800ef30 tx=0x7f4b1800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 shutdown_connections 2023-10-17T01:14:34.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b080a0b20 0x7f4b080a2fe0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b201028f0 0x7f4b20073680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b20100d60 0x7f4b20074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 --2- 172.21.15.45:0/3595650394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b20101f50 0x7f4b20073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 >> 172.21.15.45:0/3595650394 conn(0x7f4b200fc2d0 msgr2=0x7f4b200fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.558+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 shutdown_connections 2023-10-17T01:14:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.559+0000 7f4b249ff700 1 -- 172.21.15.45:0/3595650394 wait complete. 2023-10-17T01:14:34.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:34 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3800149440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:34.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:34 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2679249322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:34.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:34 smithi045 ceph-mon[98474]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:34.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-17T01:14:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 -- 172.21.15.45:0/781513332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4100d40 msgr2=0x7fbac4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/781513332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4100d40 0x7fbac4101160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbac00099c0 tx=0x7fbac0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 -- 172.21.15.45:0/781513332 shutdown_connections 2023-10-17T01:14:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/781513332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac41028d0 0x7fbac4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/781513332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4100d40 0x7fbac4101160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/781513332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac4101f30 0x7fbac4102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.784+0000 7fbacb5ff700 1 -- 172.21.15.45:0/781513332 >> 172.21.15.45:0/781513332 conn(0x7fbac40fc290 msgr2=0x7fbac40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.785+0000 7fbacb5ff700 1 -- 172.21.15.45:0/781513332 shutdown_connections 2023-10-17T01:14:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.785+0000 7fbacb5ff700 1 -- 172.21.15.45:0/781513332 wait complete. 2023-10-17T01:14:34.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.785+0000 7fbacb5ff700 1 Processor -- start 2023-10-17T01:14:34.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 -- start start 2023-10-17T01:14:34.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 0x7fbac4074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4101f30 0x7fbac4073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac41028d0 0x7fbac4073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac4073e20 con 0x7fbac4101f30 2023-10-17T01:14:34.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac4073f90 con 0x7fbac41028d0 2023-10-17T01:14:34.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbacb5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac41a0ac0 con 0x7fbac4100d40 2023-10-17T01:14:34.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbac939b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 0x7fbac4074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbac939b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 0x7fbac4074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36052/0 (socket says 172.21.15.45:36052) 2023-10-17T01:14:34.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbac8b9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4101f30 0x7fbac4073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbac939b700 1 -- 172.21.15.45:0/2493333238 learned_addr learned my addr 172.21.15.45:0/2493333238 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.786+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac41028d0 0x7fbac4073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac939b700 1 -- 172.21.15.45:0/2493333238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac41028d0 msgr2=0x7fbac4073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac939b700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac41028d0 0x7fbac4073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac939b700 1 -- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4101f30 msgr2=0x7fbac4073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac939b700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4101f30 0x7fbac4073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac939b700 1 -- 172.21.15.45:0/2493333238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00095f0 con 0x7fbac4100d40 2023-10-17T01:14:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4101f30 0x7fbac4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbac939b700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 0x7fbac4074b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009f50 tx=0x7fbac0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbaba7fc700 1 -- 172.21.15.45:0/2493333238 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac001d070 con 0x7fbac4100d40 2023-10-17T01:14:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbaba7fc700 1 -- 172.21.15.45:0/2493333238 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac0003f90 con 0x7fbac4100d40 2023-10-17T01:14:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbaba7fc700 1 -- 172.21.15.45:0/2493333238 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0022d00 con 0x7fbac4100d40 2023-10-17T01:14:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac41a0d40 con 0x7fbac4100d40 2023-10-17T01:14:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.787+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac41a1110 con 0x7fbac4100d40 2023-10-17T01:14:34.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.790+0000 7fbaba7fc700 1 -- 172.21.15.45:0/2493333238 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fbac0022470 con 0x7fbac4100d40 2023-10-17T01:14:34.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 Processor -- start 2023-10-17T01:14:34.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbaba7fc700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbab00a0b20 0x7fbab00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 -- start start 2023-10-17T01:14:34.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbaba7fc700 1 -- 172.21.15.45:0/2493333238 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbac00cd3f0 con 0x7fbac4100d40 2023-10-17T01:14:34.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d0611f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbab00a0b20 0x7fbab00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0612cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d06136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d05699c0 con 0x5582d0611b50 2023-10-17T01:14:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0567380 con 0x5582d06124b0 2023-10-17T01:14:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0562c70 con 0x5582d0613260 2023-10-17T01:14:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbac939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d0611f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbac939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d0611f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41866/0 (socket says 172.21.15.45:41866) 2023-10-17T01:14:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.792+0000 7fbac939b700 1 -- 172.21.15.45:0/431303275 learned_addr learned my addr 172.21.15.45:0/431303275 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0612cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d06136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac939b700 1 -- 172.21.15.45:0/431303275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 msgr2=0x5582d06136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac939b700 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d06136e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac939b700 1 -- 172.21.15.45:0/431303275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 msgr2=0x5582d0612cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac939b700 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0612cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac939b700 1 -- 172.21.15.45:0/431303275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582d0390d90 con 0x5582d0611b50 2023-10-17T01:14:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbab00a0b20 0x7fbab00a2fe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbac4075160 tx=0x7fbabc009ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbac939b700 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d0611f70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbac0096fa0 tx=0x7fbac0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b82ac5532022dd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbaaf7fe700 1 -- 172.21.15.45:0/431303275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac009aaa0 con 0x5582d0611b50 2023-10-17T01:14:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbaaf7fe700 1 -- 172.21.15.45:0/431303275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac009e8d0 con 0x5582d0611b50 2023-10-17T01:14:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.793+0000 7fbaaf7fe700 1 -- 172.21.15.45:0/431303275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0098f60 con 0x5582d0611b50 2023-10-17T01:14:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 -- 172.21.15.45:0/431303275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 msgr2=0x5582d0611f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d0611f70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbac0096fa0 tx=0x7fbac0005a00 comp rx=0 tx=0).stop 2023-10-17T01:14:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 -- 172.21.15.45:0/431303275 shutdown_connections 2023-10-17T01:14:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d06136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d0611f70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/431303275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0612cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 -- 172.21.15.45:0/431303275 >> 172.21.15.45:0/431303275 conn(0x5582d0571080 msgr2=0x5582d060c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 -- 172.21.15.45:0/431303275 shutdown_connections 2023-10-17T01:14:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.794+0000 7fbadee9fb80 1 -- 172.21.15.45:0/431303275 wait complete. 2023-10-17T01:14:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 -- start start 2023-10-17T01:14:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 Processor -- start 2023-10-17T01:14:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d0611b50 0x5582d061b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 0x5582d061ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d0622a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d04ff4c0 con 0x5582d06124b0 2023-10-17T01:14:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d04ff160 con 0x5582d0611b50 2023-10-17T01:14:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d03e68d0 con 0x5582d0613260 2023-10-17T01:14:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.795+0000 7fbadee9fb80 10 client.0 _ll_register_callbacks cb 0x5582d044ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac8b9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 0x5582d061ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac8b9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 0x5582d061ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41872/0 (socket says 172.21.15.45:41872) 2023-10-17T01:14:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac8b9a700 1 -- 172.21.15.45:0/1636722435 learned_addr learned my addr 172.21.15.45:0/1636722435 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d0611b50 0x5582d061b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d0622a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 -- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 msgr2=0x5582d0622a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d0622a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 -- 172.21.15.45:0/1636722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 msgr2=0x5582d061ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 0x5582d061ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 -- 172.21.15.45:0/1636722435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac41a1110 con 0x5582d0611b50 2023-10-17T01:14:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 0x5582d061ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbac939b700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d0611b50 0x5582d061b510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbac009aa30 tx=0x7fbac009a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a0594fee117ecd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbaacff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.796+0000 7fbaacff9700 1 -- 172.21.15.45:0/1636722435 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0004100 con 0x5582d0611b50 2023-10-17T01:14:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582d058dcc0 con 0x5582d0611b50 2023-10-17T01:14:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbaacff9700 1 -- 172.21.15.45:0/1636722435 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbac00a2a80 con 0x5582d0611b50 2023-10-17T01:14:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbaacff9700 1 -- 172.21.15.45:0/1636722435 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac0098f40 con 0x5582d0611b50 2023-10-17T01:14:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 10 client.25184 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5582d0379c10 con 0x5582d0611b50 2023-10-17T01:14:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbaacff9700 1 -- 172.21.15.45:0/1636722435 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac007d020 con 0x5582d0611b50 2023-10-17T01:14:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbaacff9700 1 -- 172.21.15.45:0/1636722435 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fbac00b6070 con 0x5582d0611b50 2023-10-17T01:14:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 10 client.25184 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-17T01:14:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 20 client.25184 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d0626920 0x5582d06259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 4 client.25184 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.797+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 --> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582d05e0ce0 con 0x5582d0626920 2023-10-17T01:14:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.798+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d0626920 0x5582d06259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.798+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d0626920 0x5582d06259c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4d7b2eb96e85c432 server_cookie=510bcdb477ae2d68 in_seq=0 out_seq=0 2023-10-17T01:14:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.798+0000 7fbaacff9700 10 client.25184 ms_handle_connect on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbaacff9700 1 -- 172.21.15.45:0/1636722435 <== mds.3 v2:172.21.15.181:6836/1396793294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5582d05e0ce0 con 0x5582d0626920 2023-10-17T01:14:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbaacff9700 10 client.25184 handle_command_reply: tid=0 2023-10-17T01:14:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 client.25184 shutdown 2023-10-17T01:14:34.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d0611b50 msgr2=0x5582d061b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d0611b50 0x5582d061b510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbac009aa30 tx=0x7fbac009a1c0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 shutdown_connections 2023-10-17T01:14:34.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d0622a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d0611b50 0x5582d061b510 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d0626920 0x5582d06259c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/1636722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d06124b0 0x5582d061ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 >> 172.21.15.45:0/1636722435 conn(0x5582d0571080 msgr2=0x5582d060bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbaacff9700 0 client.25184 ms_handle_reset on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:34.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 shutdown_connections 2023-10-17T01:14:34.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 1 -- 172.21.15.45:0/1636722435 wait complete. 2023-10-17T01:14:34.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.799+0000 7fbadee9fb80 20 client.25184 trim_cache size 0 max 16384 2023-10-17T01:14:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.817+0000 7fbadee9fb80 1 Processor -- start 2023-10-17T01:14:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.817+0000 7fbadee9fb80 1 -- start start 2023-10-17T01:14:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0611b50 0x5582d061abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0608820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 0x5582d061a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d052d0b0 con 0x5582d0613260 2023-10-17T01:14:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0567cf0 con 0x5582d06124b0 2023-10-17T01:14:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0502440 con 0x5582d0611b50 2023-10-17T01:14:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 0x5582d061a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 0x5582d061a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41876/0 (socket says 172.21.15.45:41876) 2023-10-17T01:14:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 -- 172.21.15.45:0/3286414037 learned_addr learned my addr 172.21.15.45:0/3286414037 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac939b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0611b50 0x5582d061abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0608820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 -- 172.21.15.45:0/3286414037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0611b50 msgr2=0x5582d061abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0611b50 0x5582d061abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 -- 172.21.15.45:0/3286414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 msgr2=0x5582d0608820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0608820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac9b9c700 1 -- 172.21.15.45:0/3286414037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582d03b7200 con 0x5582d0613260 2023-10-17T01:14:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.818+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0608820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 0x5582d061a260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0048e0 tx=0x7fbabc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2377bdb4873363 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbaaffff700 1 -- 172.21.15.45:0/3286414037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc008e80 con 0x5582d0613260 2023-10-17T01:14:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbaaffff700 1 -- 172.21.15.45:0/3286414037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbabc00f040 con 0x5582d0613260 2023-10-17T01:14:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbaaffff700 1 -- 172.21.15.45:0/3286414037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc018790 con 0x5582d0613260 2023-10-17T01:14:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 -- 172.21.15.45:0/3286414037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 msgr2=0x5582d061a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 0x5582d061a260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0048e0 tx=0x7fbabc015000 comp rx=0 tx=0).stop 2023-10-17T01:14:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 -- 172.21.15.45:0/3286414037 shutdown_connections 2023-10-17T01:14:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0613260 0x5582d061a260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0611b50 0x5582d061abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/3286414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d0608820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.819+0000 7fbadee9fb80 1 -- 172.21.15.45:0/3286414037 >> 172.21.15.45:0/3286414037 conn(0x5582d0571080 msgr2=0x5582d060bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 -- 172.21.15.45:0/3286414037 shutdown_connections 2023-10-17T01:14:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 -- 172.21.15.45:0/3286414037 wait complete. 2023-10-17T01:14:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 -- start start 2023-10-17T01:14:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 Processor -- start 2023-10-17T01:14:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d062c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d062e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d062ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0677c80 con 0x5582d0611b50 2023-10-17T01:14:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.820+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0677dc0 con 0x5582d06124b0 2023-10-17T01:14:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbadee9fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d0677f00 con 0x5582d0613260 2023-10-17T01:14:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbadee9fb80 10 client.0 _ll_register_callbacks cb 0x5582d044ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d062c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d062c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41878/0 (socket says 172.21.15.45:41878) 2023-10-17T01:14:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 -- 172.21.15.45:0/2585526720 learned_addr learned my addr 172.21.15.45:0/2585526720 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d062e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac9b9c700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d062ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 -- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 msgr2=0x5582d062ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d062ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 -- 172.21.15.45:0/2585526720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 msgr2=0x5582d062e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:34.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d062e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 -- 172.21.15.45:0/2585526720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc0045c0 con 0x5582d0611b50 2023-10-17T01:14:34.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d062e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:34.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbac939b700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d062c300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbac0097510 tx=0x7fbac009ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af28ba4079179901 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:34.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.821+0000 7fbaacff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbaacff9700 1 -- 172.21.15.45:0/2585526720 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac009a230 con 0x5582d0611b50 2023-10-17T01:14:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbac009e8d0 con 0x5582d0611b50 2023-10-17T01:14:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbaacff9700 1 -- 172.21.15.45:0/2585526720 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbac007e080 con 0x5582d0611b50 2023-10-17T01:14:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbaacff9700 1 -- 172.21.15.45:0/2585526720 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac00a2a80 con 0x5582d0611b50 2023-10-17T01:14:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 10 client.15174 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5582d038f260 con 0x5582d0611b50 2023-10-17T01:14:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbaacff9700 1 -- 172.21.15.45:0/2585526720 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac00a8070 con 0x5582d0611b50 2023-10-17T01:14:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbaacff9700 1 -- 172.21.15.45:0/2585526720 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fbac009ad00 con 0x5582d0611b50 2023-10-17T01:14:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 10 client.15174 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 10 client.15174 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-17T01:14:34.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 20 client.15174 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:34.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d067c6d0 0x5582d0679120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:34.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 4 client.15174 mds_command: new command op to 24526 tid=0[{"prefix": "status"}] 2023-10-17T01:14:34.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.822+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 --> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5582d055bcf0 con 0x5582d067c6d0 2023-10-17T01:14:34.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d067c6d0 0x5582d0679120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbac8b9a700 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d067c6d0 0x5582d0679120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=596445e726c30db6 server_cookie=9beaa44823e701ef in_seq=0 out_seq=0 2023-10-17T01:14:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbaacff9700 10 client.15174 ms_handle_connect on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbaacff9700 1 -- 172.21.15.45:0/2585526720 <== mds.3 v2:172.21.15.181:6836/1396793294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5582d055bcf0 con 0x5582d067c6d0 2023-10-17T01:14:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbaacff9700 10 client.15174 handle_command_reply: tid=0 2023-10-17T01:14:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbadee9fb80 1 client.15174 shutdown 2023-10-17T01:14:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 msgr2=0x5582d062c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.823+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d062c300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbac0097510 tx=0x7fbac009ea40 comp rx=0 tx=0).stop 2023-10-17T01:14:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 shutdown_connections 2023-10-17T01:14:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5582d0613260 0x5582d062ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x5582d067c6d0 0x5582d0679120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d0611b50 0x5582d062c300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 --2- 172.21.15.45:0/2585526720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582d06124b0 0x5582d062e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbaacff9700 0 client.15174 ms_handle_reset on v2:172.21.15.181:6836/1396793294 2023-10-17T01:14:34.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 >> 172.21.15.45:0/2585526720 conn(0x5582d0571080 msgr2=0x5582d060c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 shutdown_connections 2023-10-17T01:14:34.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 1 -- 172.21.15.45:0/2585526720 wait complete. 2023-10-17T01:14:34.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbadee9fb80 20 client.15174 trim_cache size 0 max 16384 2023-10-17T01:14:34.840 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:34.840 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:34.840 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 3, 2023-10-17T01:14:34.841 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24526, 2023-10-17T01:14:34.841 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:34.841 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:34.841 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:34.841 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 88.384222519000005, 2023-10-17T01:14:34.841 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:34.842 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:34.842 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:34.842 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 112.498069222 2023-10-17T01:14:34.842 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbab00a0b20 msgr2=0x7fbab00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbab00a0b20 0x7fbab00a2fe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbac4075160 tx=0x7fbabc009ba0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 msgr2=0x7fbac4074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.824+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 0x7fbac4074b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009f50 tx=0x7fbac0005740 comp rx=0 tx=0).stop 2023-10-17T01:14:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 shutdown_connections 2023-10-17T01:14:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbab00a0b20 0x7fbab00a2fe0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac41028d0 0x7fbac4073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4100d40 0x7fbac4074b00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 --2- 172.21.15.45:0/2493333238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac4101f30 0x7fbac4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 >> 172.21.15.45:0/2493333238 conn(0x7fbac40fc290 msgr2=0x7fbac40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 shutdown_connections 2023-10-17T01:14:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:34.825+0000 7fbacb5ff700 1 -- 172.21.15.45:0/2493333238 wait complete. 2023-10-17T01:14:34.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:34 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3800149440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:34.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:34 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2679249322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:34.846 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:34 smithi060 ceph-mon[134136]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:34.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-17T01:14:34.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:34 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3800149440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:34.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:34 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2679249322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:34.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:34 smithi181 ceph-mon[134520]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-17T01:14:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 -- 172.21.15.45:0/1306943832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 msgr2=0x7f25841023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 --2- 172.21.15.45:0/1306943832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841023b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25780099c0 tx=0x7f2578009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 -- 172.21.15.45:0/1306943832 shutdown_connections 2023-10-17T01:14:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 --2- 172.21.15.45:0/1306943832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 --2- 172.21.15.45:0/1306943832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 --2- 172.21.15.45:0/1306943832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841023b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 -- 172.21.15.45:0/1306943832 >> 172.21.15.45:0/1306943832 conn(0x7f25840fc2d0 msgr2=0x7f25840fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 -- 172.21.15.45:0/1306943832 shutdown_connections 2023-10-17T01:14:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.051+0000 7f2588bff700 1 -- 172.21.15.45:0/1306943832 wait complete. 2023-10-17T01:14:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.052+0000 7f2588bff700 1 Processor -- start 2023-10-17T01:14:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.052+0000 7f2588bff700 1 -- start start 2023-10-17T01:14:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2588bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2588bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2588bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2588bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841a07a0 con 0x7f2584101f50 2023-10-17T01:14:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2588bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841a0910 con 0x7f25841028f0 2023-10-17T01:14:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2588bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841a0ad0 con 0x7f2584100d60 2023-10-17T01:14:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f258259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f258259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36110/0 (socket says 172.21.15.45:36110) 2023-10-17T01:14:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53984/0 (socket says 172.21.15.45:53984) 2023-10-17T01:14:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f258259c700 1 -- 172.21.15.45:0/2008093802 learned_addr learned my addr 172.21.15.45:0/2008093802 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 -- 172.21.15.45:0/2008093802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 msgr2=0x7f2584198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 -- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 msgr2=0x7f25841991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2582d9d700 1 -- 172.21.15.45:0/2008093802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25780095f0 con 0x7f25841028f0 2023-10-17T01:14:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f258259c700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.053+0000 7f2581d9b700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.054+0000 7f2582d9d700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841a01d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f257c00efc0 tx=0x7f257c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.054+0000 7f25737fe700 1 -- 172.21.15.45:0/2008093802 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c018070 con 0x7f25841028f0 2023-10-17T01:14:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.054+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25841a0d50 con 0x7f25841028f0 2023-10-17T01:14:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.054+0000 7f25737fe700 1 -- 172.21.15.45:0/2008093802 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257c00f040 con 0x7f25841028f0 2023-10-17T01:14:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.054+0000 7f25737fe700 1 -- 172.21.15.45:0/2008093802 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00cd40 con 0x7f25841028f0 2023-10-17T01:14:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.054+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841a11f0 con 0x7f25841028f0 2023-10-17T01:14:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.057+0000 7f25737fe700 1 -- 172.21.15.45:0/2008093802 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f257c003a00 con 0x7f25841028f0 2023-10-17T01:14:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.058+0000 7f259c546b80 1 Processor -- start 2023-10-17T01:14:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.058+0000 7f259c546b80 1 -- start start 2023-10-17T01:14:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.058+0000 7f25737fe700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f256c0a0ba0 0x7f256c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f259c546b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e16392f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f259c546b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e16393cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f25737fe700 1 -- 172.21.15.45:0/2008093802 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f257c0cbc90 con 0x7f25841028f0 2023-10-17T01:14:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f259c546b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163946e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f259c546b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e162ea9c0 con 0x558e16394260 2023-10-17T01:14:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f259c546b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e162e8380 con 0x558e163934b0 2023-10-17T01:14:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f259c546b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e162e3c70 con 0x558e16392b50 2023-10-17T01:14:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f2581d9b700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f256c0a0ba0 0x7f256c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e16392f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e16392f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36116/0 (socket says 172.21.15.45:36116) 2023-10-17T01:14:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e16393cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 -- 172.21.15.45:0/1632874491 learned_addr learned my addr 172.21.15.45:0/1632874491 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f2582d9d700 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 -- 172.21.15.45:0/1632874491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 msgr2=0x558e16393cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e16393cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 -- 172.21.15.45:0/1632874491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 msgr2=0x558e163946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.059+0000 7f258259c700 1 -- 172.21.15.45:0/1632874491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e16111d90 con 0x558e16392b50 2023-10-17T01:14:35.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.060+0000 7f2581d9b700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f256c0a0ba0 0x7f256c0a3060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f257800b580 tx=0x7f257800bbc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.060+0000 7f258259c700 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e16392f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f25740115f0 tx=0x7f2574011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5983c7ec4fcc7e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.060+0000 7f2570ff9700 1 -- 172.21.15.45:0/1632874491 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574004a30 con 0x558e16392b50 2023-10-17T01:14:35.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.060+0000 7f2570ff9700 1 -- 172.21.15.45:0/1632874491 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25740054e0 con 0x558e16392b50 2023-10-17T01:14:35.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.060+0000 7f2570ff9700 1 -- 172.21.15.45:0/1632874491 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400e7a0 con 0x558e16392b50 2023-10-17T01:14:35.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 -- 172.21.15.45:0/1632874491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 msgr2=0x558e16392f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e16392f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f25740115f0 tx=0x7f2574011900 comp rx=0 tx=0).stop 2023-10-17T01:14:35.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 -- 172.21.15.45:0/1632874491 shutdown_connections 2023-10-17T01:14:35.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e16392f70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 --2- 172.21.15.45:0/1632874491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e16393cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 -- 172.21.15.45:0/1632874491 >> 172.21.15.45:0/1632874491 conn(0x558e162f2080 msgr2=0x558e1638d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 -- 172.21.15.45:0/1632874491 shutdown_connections 2023-10-17T01:14:35.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 -- 172.21.15.45:0/1632874491 wait complete. 2023-10-17T01:14:35.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.061+0000 7f259c546b80 1 -- start start 2023-10-17T01:14:35.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 Processor -- start 2023-10-17T01:14:35.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e1639c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e1639cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e162804c0 con 0x558e16394260 2023-10-17T01:14:35.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e16280160 con 0x558e163934b0 2023-10-17T01:14:35.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e161678d0 con 0x558e16392b50 2023-10-17T01:14:35.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.062+0000 7f259c546b80 10 client.0 _ll_register_callbacks cb 0x558e161cda30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:35.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41900/0 (socket says 172.21.15.45:41900) 2023-10-17T01:14:35.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 -- 172.21.15.45:0/687307872 learned_addr learned my addr 172.21.15.45:0/687307872 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2581d9b700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e1639cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f258259c700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e1639c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 -- 172.21.15.45:0/687307872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 msgr2=0x558e1639c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e1639c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 -- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 msgr2=0x558e1639cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e1639cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 -- 172.21.15.45:0/687307872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2574009dd0 con 0x558e16394260 2023-10-17T01:14:35.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2582d9d700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163a3e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f257c0c9760 tx=0x7f257c098ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ad5e038f3b5ea3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2569ffb700 1 -- 172.21.15.45:0/687307872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0997a0 con 0x558e16394260 2023-10-17T01:14:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f2569ffb700 1 -- 172.21.15.45:0/687307872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f257c004ad0 con 0x558e16394260 2023-10-17T01:14:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.063+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e1630ecc0 con 0x558e16394260 2023-10-17T01:14:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f2569ffb700 1 -- 172.21.15.45:0/687307872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257c099a40 con 0x558e16394260 2023-10-17T01:14:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 10 client.15186 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558e160fac10 con 0x558e16394260 2023-10-17T01:14:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f2569ffb700 1 -- 172.21.15.45:0/687307872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e1630ecc0 con 0x558e16394260 2023-10-17T01:14:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f2569ffb700 1 -- 172.21.15.45:0/687307872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f257c098830 con 0x558e16394260 2023-10-17T01:14:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 10 client.15186 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-17T01:14:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 20 client.15186 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163a7b00 0x558e163a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 4 client.15186 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.064+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 --> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e16361ce0 con 0x558e163a7b00 2023-10-17T01:14:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.065+0000 7f258259c700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163a7b00 0x558e163a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.065+0000 7f258259c700 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163a7b00 0x558e163a6f70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da9faface911bfc0 server_cookie=5438b7c8de725817 in_seq=0 out_seq=0 2023-10-17T01:14:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.065+0000 7f2569ffb700 10 client.15186 ms_handle_connect on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f2569ffb700 1 -- 172.21.15.45:0/687307872 <== mds.2 v2:172.21.15.60:6836/1246290891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x558e16361ce0 con 0x558e163a7b00 2023-10-17T01:14:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f2569ffb700 10 client.15186 handle_command_reply: tid=0 2023-10-17T01:14:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 client.15186 shutdown 2023-10-17T01:14:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 msgr2=0x558e163a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163a3e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f257c0c9760 tx=0x7f257c098ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 shutdown_connections 2023-10-17T01:14:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163a7b00 0x558e163a6f70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e16394260 0x558e163a3e10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e1639c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f2569ffb700 0 client.15186 ms_handle_reset on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 --2- 172.21.15.45:0/687307872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e163934b0 0x558e1639cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 >> 172.21.15.45:0/687307872 conn(0x558e162f2080 msgr2=0x558e1638cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 shutdown_connections 2023-10-17T01:14:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.066+0000 7f259c546b80 1 -- 172.21.15.45:0/687307872 wait complete. 2023-10-17T01:14:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.067+0000 7f259c546b80 20 client.15186 trim_cache size 0 max 16384 2023-10-17T01:14:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.084+0000 7f259c546b80 1 Processor -- start 2023-10-17T01:14:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 -- start start 2023-10-17T01:14:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16392b50 0x558e163a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e16397210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16394260 0x558e16397750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1638e850 con 0x558e163934b0 2023-10-17T01:14:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e162ae0b0 con 0x558e16392b50 2023-10-17T01:14:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f259c546b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e162e8cf0 con 0x558e16394260 2023-10-17T01:14:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e16397210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e16397210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41908/0 (socket says 172.21.15.45:41908) 2023-10-17T01:14:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f2581d9b700 1 -- 172.21.15.45:0/3727307538 learned_addr learned my addr 172.21.15.45:0/3727307538 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f258259c700 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16392b50 0x558e163a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f2582d9d700 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16394260 0x558e16397750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.085+0000 7f2581d9b700 1 -- 172.21.15.45:0/3727307538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16394260 msgr2=0x558e16397750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f2581d9b700 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16394260 0x558e16397750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f2581d9b700 1 -- 172.21.15.45:0/3727307538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16392b50 msgr2=0x558e163a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f2581d9b700 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16392b50 0x558e163a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f2581d9b700 1 -- 172.21.15.45:0/3727307538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e16138200 con 0x558e163934b0 2023-10-17T01:14:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f258259c700 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16392b50 0x558e163a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f2581d9b700 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e16397210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f25780058e0 tx=0x7f257801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cc7a48d1b178b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f25717fa700 1 -- 172.21.15.45:0/3727307538 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257802e040 con 0x558e163934b0 2023-10-17T01:14:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f25717fa700 1 -- 172.21.15.45:0/3727307538 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257801e070 con 0x558e163934b0 2023-10-17T01:14:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.086+0000 7f25717fa700 1 -- 172.21.15.45:0/3727307538 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800f680 con 0x558e163934b0 2023-10-17T01:14:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 -- 172.21.15.45:0/3727307538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 msgr2=0x558e16397210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e16397210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f25780058e0 tx=0x7f257801a000 comp rx=0 tx=0).stop 2023-10-17T01:14:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 -- 172.21.15.45:0/3727307538 shutdown_connections 2023-10-17T01:14:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16394260 0x558e16397750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16392b50 0x558e163a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 --2- 172.21.15.45:0/3727307538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e16397210 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 -- 172.21.15.45:0/3727307538 >> 172.21.15.45:0/3727307538 conn(0x558e162f2080 msgr2=0x558e1639a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 -- 172.21.15.45:0/3727307538 shutdown_connections 2023-10-17T01:14:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 -- 172.21.15.45:0/3727307538 wait complete. 2023-10-17T01:14:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 -- start start 2023-10-17T01:14:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.087+0000 7f259c546b80 1 Processor -- start 2023-10-17T01:14:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e163add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e163ae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16394260 0x558e163af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e163af7b0 con 0x558e163934b0 2023-10-17T01:14:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e163af920 con 0x558e16394260 2023-10-17T01:14:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e163afae0 con 0x558e16392b50 2023-10-17T01:14:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f259c546b80 10 client.0 _ll_register_callbacks cb 0x7f257c099110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e163ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e163ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41918/0 (socket says 172.21.15.45:41918) 2023-10-17T01:14:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 -- 172.21.15.45:0/2417258010 learned_addr learned my addr 172.21.15.45:0/2417258010 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f258259c700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e163add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2582d9d700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16394260 0x558e163af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 -- 172.21.15.45:0/2417258010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 msgr2=0x558e163add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e163add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 -- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16394260 msgr2=0x558e163af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16394260 0x558e163af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f2581d9b700 1 -- 172.21.15.45:0/2417258010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2584109af0 con 0x558e163934b0 2023-10-17T01:14:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.088+0000 7f258259c700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e163add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f2581d9b700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e163ae260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f257800a010 tx=0x7f257800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15705f59b346f5a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f2569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f2569ffb700 1 -- 172.21.15.45:0/2417258010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257802e070 con 0x558e163934b0 2023-10-17T01:14:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f2569ffb700 1 -- 172.21.15.45:0/2417258010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2578028870 con 0x558e163934b0 2023-10-17T01:14:35.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25740054e0 con 0x558e163934b0 2023-10-17T01:14:35.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f2569ffb700 1 -- 172.21.15.45:0/2417258010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2578003ea0 con 0x558e163934b0 2023-10-17T01:14:35.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f259c546b80 10 client.15192 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:35.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558e16110260 con 0x558e163934b0 2023-10-17T01:14:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.089+0000 7f2569ffb700 1 -- 172.21.15.45:0/2417258010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25740054e0 con 0x558e163934b0 2023-10-17T01:14:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f2569ffb700 1 -- 172.21.15.45:0/2417258010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f257803b420 con 0x558e163934b0 2023-10-17T01:14:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f259c546b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f259c546b80 10 client.15192 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-17T01:14:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f259c546b80 20 client.15192 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f259c546b80 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163fe9e0 0x558e163fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f259c546b80 4 client.15192 mds_command: new command op to 24539 tid=0[{"prefix": "status"}] 2023-10-17T01:14:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 --> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e162dccf0 con 0x558e163fe9e0 2023-10-17T01:14:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.090+0000 7f258259c700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163fe9e0 0x558e163fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f258259c700 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163fe9e0 0x558e163fb4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98d7012a33c17e1b server_cookie=15093bd1ce4aea2 in_seq=0 out_seq=0 2023-10-17T01:14:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f2569ffb700 10 client.15192 ms_handle_connect on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f2569ffb700 1 -- 172.21.15.45:0/2417258010 <== mds.2 v2:172.21.15.60:6836/1246290891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558e162dccf0 con 0x558e163fe9e0 2023-10-17T01:14:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f2569ffb700 10 client.15192 handle_command_reply: tid=0 2023-10-17T01:14:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 client.15192 shutdown 2023-10-17T01:14:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 msgr2=0x558e163ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e163ae260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f257800a010 tx=0x7f257800b6c0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 shutdown_connections 2023-10-17T01:14:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x558e163fe9e0 0x558e163fb4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e16394260 0x558e163af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e16392b50 0x558e163add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 --2- 172.21.15.45:0/2417258010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e163934b0 0x558e163ae260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f2569ffb700 0 client.15192 ms_handle_reset on v2:172.21.15.60:6836/1246290891 2023-10-17T01:14:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 >> 172.21.15.45:0/2417258010 conn(0x558e162f2080 msgr2=0x558e163a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 shutdown_connections 2023-10-17T01:14:35.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.091+0000 7f259c546b80 1 -- 172.21.15.45:0/2417258010 wait complete. 2023-10-17T01:14:35.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f259c546b80 20 client.15192 trim_cache size 0 max 16384 2023-10-17T01:14:35.113 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:35.114 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:35.114 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2023-10-17T01:14:35.114 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24539, 2023-10-17T01:14:35.114 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:35.115 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:35.115 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:35.115 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 89.665289619000006, 2023-10-17T01:14:35.115 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:35.115 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:35.116 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:35.116 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 110.597653818 2023-10-17T01:14:35.116 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f256c0a0ba0 msgr2=0x7f256c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f256c0a0ba0 0x7f256c0a3060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f257800b580 tx=0x7f257800bbc0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 msgr2=0x7f25841a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841a01d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f257c00efc0 tx=0x7f257c00c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 shutdown_connections 2023-10-17T01:14:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f256c0a0ba0 0x7f256c0a3060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25841028f0 0x7f25841a01d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2584100d60 0x7f2584198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 --2- 172.21.15.45:0/2008093802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2584101f50 0x7f25841991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 >> 172.21.15.45:0/2008093802 conn(0x7f25840fc2d0 msgr2=0x7f2584107830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 shutdown_connections 2023-10-17T01:14:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.092+0000 7f2588bff700 1 -- 172.21.15.45:0/2008093802 wait complete. 2023-10-17T01:14:35.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-17T01:14:35.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 -- 172.21.15.45:0/2343805742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 msgr2=0x7fec38101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/2343805742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38101180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fec280099c0 tx=0x7fec28009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 -- 172.21.15.45:0/2343805742 shutdown_connections 2023-10-17T01:14:35.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/2343805742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec381028f0 0x7fec381091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/2343805742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38101180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/2343805742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec38101f50 0x7fec381023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 -- 172.21.15.45:0/2343805742 >> 172.21.15.45:0/2343805742 conn(0x7fec380fc2d0 msgr2=0x7fec380fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 -- 172.21.15.45:0/2343805742 shutdown_connections 2023-10-17T01:14:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.324+0000 7fec3d5ff700 1 -- 172.21.15.45:0/2343805742 wait complete. 2023-10-17T01:14:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.325+0000 7fec3d5ff700 1 Processor -- start 2023-10-17T01:14:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.325+0000 7fec3d5ff700 1 -- start start 2023-10-17T01:14:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.325+0000 7fec3d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.325+0000 7fec3d5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 0x7fec381991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec3d5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec381028f0 0x7fec381a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec3d5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381a07f0 con 0x7fec38101f50 2023-10-17T01:14:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec3d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381a0960 con 0x7fec38100d60 2023-10-17T01:14:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec3d5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381a0b20 con 0x7fec381028f0 2023-10-17T01:14:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 0x7fec381991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 0x7fec381991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41932/0 (socket says 172.21.15.45:41932) 2023-10-17T01:14:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec377fe700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec381028f0 0x7fec381a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec367fc700 1 -- 172.21.15.45:0/3149819021 learned_addr learned my addr 172.21.15.45:0/3149819021 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 -- 172.21.15.45:0/3149819021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec381028f0 msgr2=0x7fec381a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec381028f0 0x7fec381a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 -- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 msgr2=0x7fec381991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 0x7fec381991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 -- 172.21.15.45:0/3149819021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280095f0 con 0x7fec38100d60 2023-10-17T01:14:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec377fe700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec381028f0 0x7fec381a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec367fc700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 0x7fec381991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.326+0000 7fec36ffd700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38198c80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec280058e0 tx=0x7fec2800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.327+0000 7fec1ffff700 1 -- 172.21.15.45:0/3149819021 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2801d070 con 0x7fec38100d60 2023-10-17T01:14:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.327+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec381a0da0 con 0x7fec38100d60 2023-10-17T01:14:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.327+0000 7fec1ffff700 1 -- 172.21.15.45:0/3149819021 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec28005020 con 0x7fec38100d60 2023-10-17T01:14:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.327+0000 7fec1ffff700 1 -- 172.21.15.45:0/3149819021 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28004a90 con 0x7fec38100d60 2023-10-17T01:14:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.327+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec381a11c0 con 0x7fec38100d60 2023-10-17T01:14:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.331+0000 7fec1ffff700 1 -- 172.21.15.45:0/3149819021 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fec28005400 con 0x7fec38100d60 2023-10-17T01:14:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.332+0000 7fec50e97b80 1 Processor -- start 2023-10-17T01:14:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 -- start start 2023-10-17T01:14:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee70469f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee7046acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7046b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee703c19c0 con 0x55ee7046a4b0 2023-10-17T01:14:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee703bf380 con 0x55ee7046b260 2023-10-17T01:14:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec50e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee703bac70 con 0x55ee70469b50 2023-10-17T01:14:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec1ffff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fec200a0ba0 0x7fec200a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec36ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee7046acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee70469f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec1ffff700 1 -- 172.21.15.45:0/3149819021 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fec280ccd10 con 0x7fec38100d60 2023-10-17T01:14:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec36ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee7046acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41938/0 (socket says 172.21.15.45:41938) 2023-10-17T01:14:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec36ffd700 1 -- 172.21.15.45:0/2422459375 learned_addr learned my addr 172.21.15.45:0/2422459375 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec367fc700 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7046b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.333+0000 7fec377fe700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fec200a0ba0 0x7fec200a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec36ffd700 1 -- 172.21.15.45:0/2422459375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 msgr2=0x55ee70469f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec36ffd700 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee70469f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec36ffd700 1 -- 172.21.15.45:0/2422459375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 msgr2=0x55ee7046b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec36ffd700 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7046b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec36ffd700 1 -- 172.21.15.45:0/2422459375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee701e8d90 con 0x55ee7046a4b0 2023-10-17T01:14:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec367fc700 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7046b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec36ffd700 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee7046acd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec280ca7e0 tx=0x7fec2808fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c9da72f55b475 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec377fe700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fec200a0ba0 0x7fec200a3060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fec30009550 tx=0x7fec3000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec1d7fa700 1 -- 172.21.15.45:0/2422459375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28099d20 con 0x55ee7046a4b0 2023-10-17T01:14:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec1d7fa700 1 -- 172.21.15.45:0/2422459375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec28005190 con 0x55ee7046a4b0 2023-10-17T01:14:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.334+0000 7fec1d7fa700 1 -- 172.21.15.45:0/2422459375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec280a23a0 con 0x55ee7046a4b0 2023-10-17T01:14:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 -- 172.21.15.45:0/2422459375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 msgr2=0x55ee7046acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee7046acd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec280ca7e0 tx=0x7fec2808fda0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 -- 172.21.15.45:0/2422459375 shutdown_connections 2023-10-17T01:14:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7046b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee70469f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2422459375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee7046acd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 -- 172.21.15.45:0/2422459375 >> 172.21.15.45:0/2422459375 conn(0x55ee703c9080 msgr2=0x55ee70464900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 -- 172.21.15.45:0/2422459375 shutdown_connections 2023-10-17T01:14:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.335+0000 7fec50e97b80 1 -- 172.21.15.45:0/2422459375 wait complete. 2023-10-17T01:14:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 -- start start 2023-10-17T01:14:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 Processor -- start 2023-10-17T01:14:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 0x55ee70473520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70473a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee703574c0 con 0x55ee70469b50 2023-10-17T01:14:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee70357160 con 0x55ee7046b260 2023-10-17T01:14:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7023e8d0 con 0x55ee7046a4b0 2023-10-17T01:14:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.336+0000 7fec50e97b80 10 client.0 _ll_register_callbacks cb 0x55ee702a4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:35.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 0x55ee70473520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 0x55ee70473520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41940/0 (socket says 172.21.15.45:41940) 2023-10-17T01:14:35.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 -- 172.21.15.45:0/466377594 learned_addr learned my addr 172.21.15.45:0/466377594 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec377fe700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec367fc700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70473a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 -- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 msgr2=0x55ee70473a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70473a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 -- 172.21.15.45:0/466377594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 msgr2=0x55ee7047aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 -- 172.21.15.45:0/466377594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c009d60 con 0x55ee70469b50 2023-10-17T01:14:35.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec377fe700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:35.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec36ffd700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 0x55ee70473520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec2809a350 tx=0x7fec2809a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ed45d3e3551421 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:35.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec1aff5700 1 -- 172.21.15.45:0/466377594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec280071c0 con 0x55ee70469b50 2023-10-17T01:14:35.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec1aff5700 1 -- 172.21.15.45:0/466377594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fec2809aac0 con 0x55ee70469b50 2023-10-17T01:14:35.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.337+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee703e5cc0 con 0x55ee70469b50 2023-10-17T01:14:35.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec1aff5700 1 -- 172.21.15.45:0/466377594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec280a02c0 con 0x55ee70469b50 2023-10-17T01:14:35.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 10 client.15204 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:35.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fec381a11c0 con 0x55ee70469b50 2023-10-17T01:14:35.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec1aff5700 1 -- 172.21.15.45:0/466377594 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec28093080 con 0x55ee70469b50 2023-10-17T01:14:35.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec1aff5700 1 -- 172.21.15.45:0/466377594 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fec2809a6b0 con 0x55ee70469b50 2023-10-17T01:14:35.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:35.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 10 client.15204 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-17T01:14:35.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 20 client.15204 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:35.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee7047e300 0x55ee7047d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 4 client.15204 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:35.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.338+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 --> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee70438ce0 con 0x55ee7047e300 2023-10-17T01:14:35.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.339+0000 7fec367fc700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee7047e300 0x55ee7047d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.339+0000 7fec367fc700 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee7047e300 0x55ee7047d9d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a844caabd79b539 server_cookie=a026fceabeb3ccc0 in_seq=0 out_seq=0 2023-10-17T01:14:35.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.339+0000 7fec1aff5700 10 client.15204 ms_handle_connect on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:35.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec1aff5700 1 -- 172.21.15.45:0/466377594 <== mds.1 v2:172.21.15.181:6838/1273587217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ee70438ce0 con 0x55ee7047e300 2023-10-17T01:14:35.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec1aff5700 10 client.15204 handle_command_reply: tid=0 2023-10-17T01:14:35.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 client.15204 shutdown 2023-10-17T01:14:35.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 msgr2=0x55ee70473520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 0x55ee70473520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec2809a350 tx=0x7fec2809a470 comp rx=0 tx=0).stop 2023-10-17T01:14:35.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 shutdown_connections 2023-10-17T01:14:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee7047e300 0x55ee7047d9d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee70469b50 0x55ee70473520 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 --2- 172.21.15.45:0/466377594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70473a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec1aff5700 0 client.15204 ms_handle_reset on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 >> 172.21.15.45:0/466377594 conn(0x55ee703c9080 msgr2=0x55ee70463fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 shutdown_connections 2023-10-17T01:14:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 1 -- 172.21.15.45:0/466377594 wait complete. 2023-10-17T01:14:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.340+0000 7fec50e97b80 20 client.15204 trim_cache size 0 max 16384 2023-10-17T01:14:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.358+0000 7fec50e97b80 1 Processor -- start 2023-10-17T01:14:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 -- start start 2023-10-17T01:14:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee7047d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee70465600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7046a890 con 0x55ee7046a4b0 2023-10-17T01:14:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee70438ce0 con 0x55ee7046b260 2023-10-17T01:14:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec50e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee70364830 con 0x55ee70469b50 2023-10-17T01:14:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee70465600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee70465600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41952/0 (socket says 172.21.15.45:41952) 2023-10-17T01:14:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 -- 172.21.15.45:0/2960234271 learned_addr learned my addr 172.21.15.45:0/2960234271 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec36ffd700 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee7047d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec377fe700 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 -- 172.21.15.45:0/2960234271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 msgr2=0x55ee7047d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee7047d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 -- 172.21.15.45:0/2960234271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 msgr2=0x55ee7047b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec367fc700 1 -- 172.21.15.45:0/2960234271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee701d1c10 con 0x55ee7046a4b0 2023-10-17T01:14:35.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.359+0000 7fec36ffd700 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee7047d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.360+0000 7fec367fc700 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee70465600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005d90 tx=0x7fec2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f504897c265367 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.360+0000 7fec1dffb700 1 -- 172.21.15.45:0/2960234271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c01e270 con 0x55ee7046a4b0 2023-10-17T01:14:35.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.360+0000 7fec1dffb700 1 -- 172.21.15.45:0/2960234271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec2c017040 con 0x55ee7046a4b0 2023-10-17T01:14:35.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.360+0000 7fec1dffb700 1 -- 172.21.15.45:0/2960234271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c010960 con 0x55ee7046a4b0 2023-10-17T01:14:35.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 -- 172.21.15.45:0/2960234271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 msgr2=0x55ee70465600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee70465600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005d90 tx=0x7fec2c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:35.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 -- 172.21.15.45:0/2960234271 shutdown_connections 2023-10-17T01:14:35.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee7046b260 0x55ee7047b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee70469b50 0x55ee7047d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 --2- 172.21.15.45:0/2960234271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046a4b0 0x55ee70465600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 -- 172.21.15.45:0/2960234271 >> 172.21.15.45:0/2960234271 conn(0x55ee703c9080 msgr2=0x55ee70472260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 -- 172.21.15.45:0/2960234271 shutdown_connections 2023-10-17T01:14:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 -- 172.21.15.45:0/2960234271 wait complete. 2023-10-17T01:14:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.361+0000 7fec50e97b80 1 -- start start 2023-10-17T01:14:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 Processor -- start 2023-10-17T01:14:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee70469b50 0x55ee704822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70482800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 0x55ee704857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee70485d00 con 0x55ee7046b260 2023-10-17T01:14:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee70485e70 con 0x55ee70469b50 2023-10-17T01:14:35.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee70486030 con 0x55ee7046a4b0 2023-10-17T01:14:35.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec50e97b80 10 client.0 _ll_register_callbacks cb 0x7fec2c0095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:35.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.362+0000 7fec377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 0x55ee704857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee70469b50 0x55ee704822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 0x55ee704857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41958/0 (socket says 172.21.15.45:41958) 2023-10-17T01:14:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec377fe700 1 -- 172.21.15.45:0/1145166698 learned_addr learned my addr 172.21.15.45:0/1145166698 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec367fc700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70482800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 -- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 msgr2=0x55ee70482800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70482800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 -- 172.21.15.45:0/1145166698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 msgr2=0x55ee704857c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 0x55ee704857c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 -- 172.21.15.45:0/1145166698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec381a11c0 con 0x55ee70469b50 2023-10-17T01:14:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec367fc700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70482800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec377fe700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 0x55ee704857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec36ffd700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee70469b50 0x55ee704822c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec280968c0 tx=0x7fec28085190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462f460b7a83706a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec1aff5700 1 -- 172.21.15.45:0/1145166698 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28096700 con 0x55ee70469b50 2023-10-17T01:14:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec1aff5700 1 -- 172.21.15.45:0/1145166698 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fec280071c0 con 0x55ee70469b50 2023-10-17T01:14:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.363+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec28005190 con 0x55ee70469b50 2023-10-17T01:14:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec1aff5700 1 -- 172.21.15.45:0/1145166698 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2809aac0 con 0x55ee70469b50 2023-10-17T01:14:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 10 client.25226 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fec2c0044e0 con 0x55ee70469b50 2023-10-17T01:14:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec1aff5700 1 -- 172.21.15.45:0/1145166698 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec28005190 con 0x55ee70469b50 2023-10-17T01:14:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec1aff5700 1 -- 172.21.15.45:0/1145166698 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fec28025820 con 0x55ee70469b50 2023-10-17T01:14:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 10 client.25226 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 10 client.25226 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-17T01:14:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 20 client.25226 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee704d50f0 0x55ee704d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 4 client.25226 mds_command: new command op to 24550 tid=0[{"prefix": "status"}] 2023-10-17T01:14:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 --> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ee703b3cf0 con 0x55ee704d50f0 2023-10-17T01:14:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.364+0000 7fec367fc700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee704d50f0 0x55ee704d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec367fc700 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee704d50f0 0x55ee704d1c60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a29544b161717f1 server_cookie=2a7d31de58b5f4b3 in_seq=0 out_seq=0 2023-10-17T01:14:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec1aff5700 10 client.25226 ms_handle_connect on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec1aff5700 1 -- 172.21.15.45:0/1145166698 <== mds.1 v2:172.21.15.181:6838/1273587217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ee703b3cf0 con 0x55ee704d50f0 2023-10-17T01:14:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec1aff5700 10 client.25226 handle_command_reply: tid=0 2023-10-17T01:14:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec50e97b80 1 client.25226 shutdown 2023-10-17T01:14:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee70469b50 msgr2=0x55ee704822c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.365+0000 7fec50e97b80 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee70469b50 0x55ee704822c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec280968c0 tx=0x7fec28085190 comp rx=0 tx=0).stop 2023-10-17T01:14:35.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 shutdown_connections 2023-10-17T01:14:35.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x55ee704d50f0 0x55ee704d1c60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee7046b260 0x55ee704857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ee70469b50 0x55ee704822c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec1aff5700 0 client.25226 ms_handle_reset on v2:172.21.15.181:6838/1273587217 2023-10-17T01:14:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 --2- 172.21.15.45:0/1145166698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ee7046a4b0 0x55ee70482800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 >> 172.21.15.45:0/1145166698 conn(0x55ee703c9080 msgr2=0x55ee7047d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 shutdown_connections 2023-10-17T01:14:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 1 -- 172.21.15.45:0/1145166698 wait complete. 2023-10-17T01:14:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec50e97b80 20 client.25226 trim_cache size 0 max 16384 2023-10-17T01:14:35.411 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:35.411 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:35.411 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2023-10-17T01:14:35.411 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24550, 2023-10-17T01:14:35.412 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:35.412 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:35.412 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:35.412 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 90.953677006999996, 2023-10-17T01:14:35.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:35.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:35.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:35.413 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 108.72413437199999 2023-10-17T01:14:35.413 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:35.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fec200a0ba0 msgr2=0x7fec200a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fec200a0ba0 0x7fec200a3060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fec30009550 tx=0x7fec3000b040 comp rx=0 tx=0).stop 2023-10-17T01:14:35.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 msgr2=0x7fec38198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38198c80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec280058e0 tx=0x7fec2800f690 comp rx=0 tx=0).stop 2023-10-17T01:14:35.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 shutdown_connections 2023-10-17T01:14:35.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fec200a0ba0 0x7fec200a3060 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec381028f0 0x7fec381a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec38100d60 0x7fec38198c80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 --2- 172.21.15.45:0/3149819021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec38101f50 0x7fec381991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.366+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 >> 172.21.15.45:0/3149819021 conn(0x7fec380fc2d0 msgr2=0x7fec380fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.367+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 shutdown_connections 2023-10-17T01:14:35.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.367+0000 7fec3d5ff700 1 -- 172.21.15.45:0/3149819021 wait complete. 2023-10-17T01:14:35.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-17T01:14:35.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 -- 172.21.15.45:0/2930077002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 msgr2=0x7f8df0109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 --2- 172.21.15.45:0/2930077002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0109180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8de800b0d0 tx=0x7f8de800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 -- 172.21.15.45:0/2930077002 shutdown_connections 2023-10-17T01:14:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 --2- 172.21.15.45:0/2930077002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0109180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 --2- 172.21.15.45:0/2930077002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 --2- 172.21.15.45:0/2930077002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df0101f30 0x7f8df0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 -- 172.21.15.45:0/2930077002 >> 172.21.15.45:0/2930077002 conn(0x7f8df00fc290 msgr2=0x7f8df00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 -- 172.21.15.45:0/2930077002 shutdown_connections 2023-10-17T01:14:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.622+0000 7f8df63ff700 1 -- 172.21.15.45:0/2930077002 wait complete. 2023-10-17T01:14:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.624+0000 7f8df63ff700 1 Processor -- start 2023-10-17T01:14:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.624+0000 7f8df63ff700 1 -- start start 2023-10-17T01:14:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.624+0000 7f8df63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df63ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df0101f30 0x7f8df0107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df63ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df63ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0103df0 con 0x7f8df01028d0 2023-10-17T01:14:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0103f60 con 0x7f8df0100d40 2023-10-17T01:14:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df63ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0104120 con 0x7f8df0101f30 2023-10-17T01:14:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54054/0 (socket says 172.21.15.45:54054) 2023-10-17T01:14:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 -- 172.21.15.45:0/3380123184 learned_addr learned my addr 172.21.15.45:0/3380123184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df499c700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8def7fe700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df0101f30 0x7f8df0107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 -- 172.21.15.45:0/3380123184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df0101f30 msgr2=0x7f8df0107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df0101f30 0x7f8df0107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 -- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 msgr2=0x7f8df0103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 -- 172.21.15.45:0/3380123184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8009d00 con 0x7f8df0100d40 2023-10-17T01:14:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8df499c700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.625+0000 7f8deffff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0106ac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e970 tx=0x7f8de000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.626+0000 7f8ded7fa700 1 -- 172.21.15.45:0/3380123184 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0009790 con 0x7f8df0100d40 2023-10-17T01:14:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.626+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df01a52d0 con 0x7f8df0100d40 2023-10-17T01:14:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.626+0000 7f8ded7fa700 1 -- 172.21.15.45:0/3380123184 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de00044e0 con 0x7f8df0100d40 2023-10-17T01:14:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.626+0000 7f8ded7fa700 1 -- 172.21.15.45:0/3380123184 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003e30 con 0x7f8df0100d40 2023-10-17T01:14:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.626+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df01a57b0 con 0x7f8df0100d40 2023-10-17T01:14:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.631+0000 7f8ded7fa700 1 -- 172.21.15.45:0/3380123184 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8de0003f90 con 0x7f8df0100d40 2023-10-17T01:14:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.631+0000 7f8e09d00b80 1 Processor -- start 2023-10-17T01:14:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 -- start start 2023-10-17T01:14:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 0x56003c530f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c5314b0 0x56003c531cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c5326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c4889c0 con 0x56003c530b50 2023-10-17T01:14:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c486380 con 0x56003c5314b0 2023-10-17T01:14:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c481c70 con 0x56003c532260 2023-10-17T01:14:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8df499c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 0x56003c530f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8ded7fa700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8dd80a0b20 0x7f8dd80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8df499c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 0x56003c530f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41966/0 (socket says 172.21.15.45:41966) 2023-10-17T01:14:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8df499c700 1 -- 172.21.15.45:0/3604519035 learned_addr learned my addr 172.21.15.45:0/3604519035 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8deffff700 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c5314b0 0x56003c531cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8def7fe700 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c5326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8df499c700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8dd80a0b20 0x7f8dd80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.632+0000 7f8ded7fa700 1 -- 172.21.15.45:0/3380123184 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8de0014070 con 0x7f8df0100d40 2023-10-17T01:14:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 -- 172.21.15.45:0/3604519035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 msgr2=0x56003c5326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c5326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 -- 172.21.15.45:0/3604519035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c5314b0 msgr2=0x56003c531cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c5314b0 0x56003c531cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 -- 172.21.15.45:0/3604519035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56003c2afd90 con 0x56003c530b50 2023-10-17T01:14:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8deffff700 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c5314b0 0x56003c531cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 0x56003c530f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8de8015000 tx=0x7f8de8008e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a79bf7e5466c69 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8dd67fc700 1 -- 172.21.15.45:0/3604519035 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800e040 con 0x56003c530b50 2023-10-17T01:14:35.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8df499c700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8dd80a0b20 0x7f8dd80a2fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8de8008000 tx=0x7f8de8026000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8dd67fc700 1 -- 172.21.15.45:0/3604519035 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de800be30 con 0x56003c530b50 2023-10-17T01:14:35.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.633+0000 7f8dd67fc700 1 -- 172.21.15.45:0/3604519035 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8005040 con 0x56003c530b50 2023-10-17T01:14:35.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3604519035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 msgr2=0x56003c530f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 0x56003c530f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8de8015000 tx=0x7f8de8008e70 comp rx=0 tx=0).stop 2023-10-17T01:14:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3604519035 shutdown_connections 2023-10-17T01:14:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c5326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c530b50 0x56003c530f70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3604519035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c5314b0 0x56003c531cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3604519035 >> 172.21.15.45:0/3604519035 conn(0x56003c490080 msgr2=0x56003c52b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3604519035 shutdown_connections 2023-10-17T01:14:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.634+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3604519035 wait complete. 2023-10-17T01:14:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 -- start start 2023-10-17T01:14:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 Processor -- start 2023-10-17T01:14:35.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c53a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c53ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c541de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c41e4c0 con 0x56003c5314b0 2023-10-17T01:14:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c41e160 con 0x56003c530b50 2023-10-17T01:14:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c3058d0 con 0x56003c532260 2023-10-17T01:14:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.635+0000 7f8e09d00b80 10 client.0 _ll_register_callbacks cb 0x56003c36ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c53ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c53ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41978/0 (socket says 172.21.15.45:41978) 2023-10-17T01:14:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c53a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8def7fe700 1 -- 172.21.15.45:0/3719530554 learned_addr learned my addr 172.21.15.45:0/3719530554 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8df499c700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c541de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 -- 172.21.15.45:0/3719530554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 msgr2=0x56003c541de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c541de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 -- 172.21.15.45:0/3719530554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 msgr2=0x56003c53ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c53ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 -- 172.21.15.45:0/3719530554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df01a57b0 con 0x56003c530b50 2023-10-17T01:14:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8def7fe700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c53ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8deffff700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c53a860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8de0099aa0 tx=0x7f8de0069b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac5271b000278b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.636+0000 7f8dcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/3719530554 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0099470 con 0x56003c530b50 2023-10-17T01:14:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56003c4accc0 con 0x56003c530b50 2023-10-17T01:14:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/3719530554 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8de0065700 con 0x56003c530b50 2023-10-17T01:14:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/3719530554 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de009a210 con 0x56003c530b50 2023-10-17T01:14:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8e09d00b80 10 client.25238 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f8df0109ad0 con 0x56003c530b50 2023-10-17T01:14:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/3719530554 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56003c4accc0 con 0x56003c530b50 2023-10-17T01:14:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.637+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/3719530554 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8de00cc100 con 0x56003c530b50 2023-10-17T01:14:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8e09d00b80 10 client.25238 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8e09d00b80 10 client.25238 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-17T01:14:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8e09d00b80 20 client.25238 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c545ab0 0x56003c544f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8e09d00b80 4 client.25238 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56003c4ffce0 con 0x56003c545ab0 2023-10-17T01:14:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.638+0000 7f8def7fe700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c545ab0 0x56003c544f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.639+0000 7f8def7fe700 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c545ab0 0x56003c544f40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7df256ab6a909ab server_cookie=829cd57c0296be in_seq=0 out_seq=0 2023-10-17T01:14:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.639+0000 7f8dcf7fe700 10 client.25238 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.639+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/3719530554 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x56003c4ffce0 con 0x56003c545ab0 2023-10-17T01:14:35.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.639+0000 7f8dcf7fe700 10 client.25238 handle_command_reply: tid=0 2023-10-17T01:14:35.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.639+0000 7f8e09d00b80 1 client.25238 shutdown 2023-10-17T01:14:35.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 msgr2=0x56003c53a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c53a860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8de0099aa0 tx=0x7f8de0069b00 comp rx=0 tx=0).stop 2023-10-17T01:14:35.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 shutdown_connections 2023-10-17T01:14:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c532260 0x56003c541de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c545ab0 0x56003c544f40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c53a860 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/3719530554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c53ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 >> 172.21.15.45:0/3719530554 conn(0x56003c490080 msgr2=0x56003c52afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8dcf7fe700 0 client.25238 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 shutdown_connections 2023-10-17T01:14:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 1 -- 172.21.15.45:0/3719530554 wait complete. 2023-10-17T01:14:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.640+0000 7f8e09d00b80 20 client.25238 trim_cache size 0 max 16384 2023-10-17T01:14:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 Processor -- start 2023-10-17T01:14:35.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 -- start start 2023-10-17T01:14:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c542610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c5314b0 0x56003c538740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 0x56003c544f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c4accc0 con 0x56003c532260 2023-10-17T01:14:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c531b60 con 0x56003c530b50 2023-10-17T01:14:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c52d240 con 0x56003c5314b0 2023-10-17T01:14:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8df499c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 0x56003c544f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8df499c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 0x56003c544f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41980/0 (socket says 172.21.15.45:41980) 2023-10-17T01:14:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.658+0000 7f8df499c700 1 -- 172.21.15.45:0/816480118 learned_addr learned my addr 172.21.15.45:0/816480118 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8deffff700 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c542610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8def7fe700 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c5314b0 0x56003c538740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8df499c700 1 -- 172.21.15.45:0/816480118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c5314b0 msgr2=0x56003c538740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8df499c700 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c5314b0 0x56003c538740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8df499c700 1 -- 172.21.15.45:0/816480118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 msgr2=0x56003c542610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8df499c700 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c542610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8df499c700 1 -- 172.21.15.45:0/816480118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56003c298c10 con 0x56003c532260 2023-10-17T01:14:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8df499c700 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 0x56003c544f40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x56003c5308f0 tx=0x7f8de80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff73f0222b89bcb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8dd6ffd700 1 -- 172.21.15.45:0/816480118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800e050 con 0x56003c532260 2023-10-17T01:14:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8dd6ffd700 1 -- 172.21.15.45:0/816480118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de8005780 con 0x56003c532260 2023-10-17T01:14:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.659+0000 7f8dd6ffd700 1 -- 172.21.15.45:0/816480118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80316e0 con 0x56003c532260 2023-10-17T01:14:35.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 -- 172.21.15.45:0/816480118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 msgr2=0x56003c544f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 0x56003c544f40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x56003c5308f0 tx=0x7f8de80099e0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 -- 172.21.15.45:0/816480118 shutdown_connections 2023-10-17T01:14:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c532260 0x56003c544f40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c530b50 0x56003c542610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/816480118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c5314b0 0x56003c538740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 -- 172.21.15.45:0/816480118 >> 172.21.15.45:0/816480118 conn(0x56003c490080 msgr2=0x56003c52afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 -- 172.21.15.45:0/816480118 shutdown_connections 2023-10-17T01:14:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 -- 172.21.15.45:0/816480118 wait complete. 2023-10-17T01:14:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.660+0000 7f8e09d00b80 1 -- start start 2023-10-17T01:14:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 Processor -- start 2023-10-17T01:14:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c530b50 0x56003c54ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c54bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c532260 0x56003c54cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c54d420 con 0x56003c5314b0 2023-10-17T01:14:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c54d590 con 0x56003c532260 2023-10-17T01:14:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003c54d750 con 0x56003c530b50 2023-10-17T01:14:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8e09d00b80 10 client.0 _ll_register_callbacks cb 0x7f8de0094550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c54bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8def7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c54bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41996/0 (socket says 172.21.15.45:41996) 2023-10-17T01:14:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.661+0000 7f8def7fe700 1 -- 172.21.15.45:0/2348481279 learned_addr learned my addr 172.21.15.45:0/2348481279 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8deffff700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c530b50 0x56003c54ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8df499c700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c532260 0x56003c54cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8def7fe700 1 -- 172.21.15.45:0/2348481279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c530b50 msgr2=0x56003c54ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8def7fe700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c530b50 0x56003c54ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8def7fe700 1 -- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c532260 msgr2=0x56003c54cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:35.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8def7fe700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c532260 0x56003c54cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8def7fe700 1 -- 172.21.15.45:0/2348481279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56003c2afd90 con 0x56003c5314b0 2023-10-17T01:14:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8df499c700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c532260 0x56003c54cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8def7fe700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c54bf40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56003c542390 tx=0x7f8de4004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c917e4655acb12f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8dcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/2348481279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4017030 con 0x56003c5314b0 2023-10-17T01:14:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8de800be30 con 0x56003c5314b0 2023-10-17T01:14:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.662+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/2348481279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8de4020e30 con 0x56003c5314b0 2023-10-17T01:14:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/2348481279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de4012a70 con 0x56003c5314b0 2023-10-17T01:14:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 10 client.15228 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56003c2d6200 con 0x56003c5314b0 2023-10-17T01:14:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/2348481279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de800be30 con 0x56003c5314b0 2023-10-17T01:14:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/2348481279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8de4032050 con 0x56003c5314b0 2023-10-17T01:14:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-17T01:14:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 20 client.15228 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c59cc80 0x56003c5995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 4 client.15228 mds_command: new command op to 24587 tid=0[{"prefix": "status"}] 2023-10-17T01:14:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.663+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56003c47acf0 con 0x56003c59cc80 2023-10-17T01:14:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8deffff700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c59cc80 0x56003c5995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8deffff700 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c59cc80 0x56003c5995d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97a2891a016423e server_cookie=1d179405798ff995 in_seq=0 out_seq=0 2023-10-17T01:14:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8dcf7fe700 10 client.15228 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8dcf7fe700 1 -- 172.21.15.45:0/2348481279 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56003c47acf0 con 0x56003c59cc80 2023-10-17T01:14:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8dcf7fe700 10 client.15228 handle_command_reply: tid=0 2023-10-17T01:14:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 client.15228 shutdown 2023-10-17T01:14:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 msgr2=0x56003c54bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c54bf40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56003c542390 tx=0x7f8de4004690 comp rx=0 tx=0).stop 2023-10-17T01:14:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 shutdown_connections 2023-10-17T01:14:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56003c59cc80 0x56003c5995d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56003c532260 0x56003c54cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56003c530b50 0x56003c54ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8e09d00b80 1 --2- 172.21.15.45:0/2348481279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56003c5314b0 0x56003c54bf40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.664+0000 7f8dcf7fe700 0 client.15228 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:35.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 >> 172.21.15.45:0/2348481279 conn(0x56003c490080 msgr2=0x56003c538740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 shutdown_connections 2023-10-17T01:14:35.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8e09d00b80 1 -- 172.21.15.45:0/2348481279 wait complete. 2023-10-17T01:14:35.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8e09d00b80 20 client.15228 trim_cache size 0 max 16384 2023-10-17T01:14:35.683 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:35.683 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "2804d2ac-6c89-11ee-8db6-212e2dc638e7", 2023-10-17T01:14:35.683 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-17T01:14:35.683 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24587, 2023-10-17T01:14:35.684 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-17T01:14:35.684 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-17T01:14:35.684 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-17T01:14:35.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 94.623963509999996, 2023-10-17T01:14:35.684 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-10-17T01:14:35.685 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 85, 2023-10-17T01:14:35.685 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 83, 2023-10-17T01:14:35.685 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 100.34004041999999 2023-10-17T01:14:35.685 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:35.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8dd80a0b20 msgr2=0x7f8dd80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8dd80a0b20 0x7f8dd80a2fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8de8008000 tx=0x7f8de8026000 comp rx=0 tx=0).stop 2023-10-17T01:14:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 msgr2=0x7f8df0106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0106ac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e970 tx=0x7f8de000ed30 comp rx=0 tx=0).stop 2023-10-17T01:14:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 shutdown_connections 2023-10-17T01:14:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8dd80a0b20 0x7f8dd80a2fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8df01028d0 0x7f8df0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8df0100d40 0x7f8df0106ac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 --2- 172.21.15.45:0/3380123184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df0101f30 0x7f8df0107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.665+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 >> 172.21.15.45:0/3380123184 conn(0x7f8df00fc290 msgr2=0x7f8df01077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.666+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 shutdown_connections 2023-10-17T01:14:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:35.666+0000 7f8df63ff700 1 -- 172.21.15.45:0/3380123184 wait complete. 2023-10-17T01:14:35.689 INFO:teuthology.run_tasks:Running task kclient... 2023-10-17T01:14:35.712 INFO:tasks.kclient:Mounting kernel clients... 2023-10-17T01:14:35.713 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-17T01:14:35.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:35.714 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-17T01:14:35.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:35.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-10-17T01:14:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2023-10-17T01:14:35.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:14:35.876 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-10-17T01:14:35.876 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-17T01:14:35.876 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-10-17T01:14:35.877 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-10-17T01:14:35.877 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-17T01:14:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:14:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:37 smithi045 ceph-mon[98474]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-17T01:14:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:37 smithi060 ceph-mon[134136]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-17T01:14:37.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:37 smithi181 ceph-mon[134520]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-17T01:14:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 -- 172.21.15.45:0/2202192209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 msgr2=0x7f25241013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2202192209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f25241013b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2514009990 tx=0x7f2514009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 -- 172.21.15.45:0/2202192209 shutdown_connections 2023-10-17T01:14:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2202192209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241060b0 0x7f252410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2202192209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f25241013b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2202192209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524101980 0x7f25241059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 -- 172.21.15.45:0/2202192209 >> 172.21.15.45:0/2202192209 conn(0x7f25240fc840 msgr2=0x7f25240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 -- 172.21.15.45:0/2202192209 shutdown_connections 2023-10-17T01:14:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.054+0000 7f252a0d9700 1 -- 172.21.15.45:0/2202192209 wait complete. 2023-10-17T01:14:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 Processor -- start 2023-10-17T01:14:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 -- start start 2023-10-17T01:14:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f2524073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524101980 0x7f2524073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241060b0 0x7f2524073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:38.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241a5340 con 0x7f2524100fd0 2023-10-17T01:14:38.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241a54b0 con 0x7f25241060b0 2023-10-17T01:14:38.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f252a0d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241a5670 con 0x7f2524101980 2023-10-17T01:14:38.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.056+0000 7f25237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f2524073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:38.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f2524073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38918/0 (socket says 172.21.15.45:38918) 2023-10-17T01:14:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 -- 172.21.15.45:0/2326165074 learned_addr learned my addr 172.21.15.45:0/2326165074 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f2523fff700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241060b0 0x7f2524073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f2522ffd700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524101980 0x7f2524073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 -- 172.21.15.45:0/2326165074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524101980 msgr2=0x7f2524073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524101980 0x7f2524073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 -- 172.21.15.45:0/2326165074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241060b0 msgr2=0x7f2524073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241060b0 0x7f2524073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 -- 172.21.15.45:0/2326165074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c009c40 con 0x7f2524100fd0 2023-10-17T01:14:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f25237fe700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f2524073050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2514005910 tx=0x7f2514004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.057+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251401d070 con 0x7f2524100fd0 2023-10-17T01:14:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.058+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140095f0 con 0x7f2524100fd0 2023-10-17T01:14:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.058+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2514022470 con 0x7f2524100fd0 2023-10-17T01:14:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.058+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514017650 con 0x7f2524100fd0 2023-10-17T01:14:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.058+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25241a5b70 con 0x7f2524100fd0 2023-10-17T01:14:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.059+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252404ea50 con 0x7f2524100fd0 2023-10-17T01:14:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.061+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f25140225e0 con 0x7f2524100fd0 2023-10-17T01:14:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.062+0000 7f2520ff9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f250c0a0b20 0x7f250c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.063+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f25140cd840 con 0x7f2524100fd0 2023-10-17T01:14:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.063+0000 7f2522ffd700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f250c0a0b20 0x7f250c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.063+0000 7f2522ffd700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f250c0a0b20 0x7f250c0a2fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2524074840 tx=0x7f2518009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.064+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2514097230 con 0x7f2524100fd0 2023-10-17T01:14:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.321+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2524074580 con 0x7f2524100fd0 2023-10-17T01:14:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.321+0000 7f2520ff9700 1 -- 172.21.15.45:0/2326165074 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f251409a850 con 0x7f2524100fd0 2023-10-17T01:14:38.323 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-17T01:14:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f250c0a0b20 msgr2=0x7f250c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f250c0a0b20 0x7f250c0a2fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2524074840 tx=0x7f2518009260 comp rx=0 tx=0).stop 2023-10-17T01:14:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 msgr2=0x7f2524073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f2524073050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2514005910 tx=0x7f2514004710 comp rx=0 tx=0).stop 2023-10-17T01:14:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 shutdown_connections 2023-10-17T01:14:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f250c0a0b20 0x7f250c0a2fe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241060b0 0x7f2524073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2524100fd0 0x7f2524073050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 --2- 172.21.15.45:0/2326165074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524101980 0x7f2524073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 >> 172.21.15.45:0/2326165074 conn(0x7f25240fc840 msgr2=0x7f25240fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 shutdown_connections 2023-10-17T01:14:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:38.324+0000 7f252a0d9700 1 -- 172.21.15.45:0/2326165074 wait complete. 2023-10-17T01:14:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:38 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2326165074' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-17T01:14:38.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-17T01:14:38.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-17T01:14:38.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-17T01:14:38.804 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-17T01:14:38.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-10-17T01:14:38.805 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-17T01:14:38.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-17T01:14:38.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-17T01:14:38.805 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-10-17T01:14:38.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:38.806 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-17T01:14:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:38 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2326165074' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-17T01:14:38.852 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-17T01:14:38.852 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-17T01:14:38.852 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-17T01:14:38.853 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-17T01:14:38.853 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-17T01:14:38.853 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-17T01:14:38.853 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-17T01:14:38.853 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-17T01:14:38.854 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-17T01:14:38.854 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-17T01:14:38.854 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-17T01:14:38.854 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-17T01:14:38.854 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-17T01:14:38.855 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-17T01:14:38.855 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-17T01:14:38.855 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-17T01:14:38.855 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41617sec preferred_lft 41617sec 2023-10-17T01:14:38.855 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-17T01:14:38.856 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-17T01:14:38.856 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-17T01:14:38.856 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-17T01:14:38.856 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-17T01:14:38.857 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-17T01:14:38.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:38.858 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-17T01:14:38.858 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-17T01:14:38.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2023-10-17T01:14:38.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2023-10-17T01:14:38.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2023-10-17T01:14:38.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-17T01:14:38.860 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-17T01:14:38.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:38 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2326165074' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-17T01:14:39.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:39.332 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-17T01:14:39.405 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-17T01:14:39.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:39.409 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-17T01:14:39.469 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-17T01:14:39.469 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-17T01:14:39.470 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-17T01:14:39.470 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-17T01:14:39.470 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-17T01:14:39.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:39.471 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-17T01:14:39.471 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-17T01:14:39.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-17T01:14:39.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-17T01:14:39.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-17T01:14:39.473 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-17T01:14:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:39 smithi045 ceph-mon[98474]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-17T01:14:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:39 smithi060 ceph-mon[134136]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-17T01:14:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:39 smithi181 ceph-mon[134520]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-17T01:14:40.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:40.212 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-17T01:14:40.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:40.257 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-10-17T01:14:40.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:40.342 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-17T01:14:40.342 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-17T01:14:40.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--tmp-cephfs 2023-10-17T01:14:40.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-10-17T01:14:40.343 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-17T01:14:40.530 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-10-17T01:14:40.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:40.531 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-17T01:14:40.531 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-17T01:14:40.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-10-17T01:14:40.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-17T01:14:40.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-10-17T01:14:40.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-10-17T01:14:40.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-10-17T01:14:40.533 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-17T01:14:40.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:40.817 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-17T01:14:40.817 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-17T01:14:40.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-17T01:14:40.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2023-10-17T01:14:40.818 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-17T01:14:40.996 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-17T01:14:40.996 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /tmp/cephfs 2023-10-17T01:14:41.041 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/tmp/cephfs' 2023-10-17T01:14:41.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-17T01:14:41.042 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /tmp/cephfs 2023-10-17T01:14:41.124 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-17T01:14:41.124 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-17T01:14:41.124 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/c18bfcb3-8f74-497f-901a-c6d13f726359 2023-10-17T01:14:41.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:41.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/c18bfcb3-8f74-497f-901a-c6d13f726359 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,context=system_u:object_r:postgresql_db_t:s0 2023-10-17T01:14:41.279 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2023-10-17T01:14:41.280 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-17T01:14:41.280 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-10-17T01:14:41.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:41 smithi045 ceph-mon[98474]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-17T01:14:41.738 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2023-10-17T01:14:41.738 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-17T01:14:41.738 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-10-17T01:14:41.739 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.60:3300,172.21.15.181:3300" 2023-10-17T01:14:41.739 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:3300,172.21.15.60:3300,172.21.15.181:3300:/volumes/_nogroup/sv_1/c18bfcb3-8f74-497f-901a-c6d13f726359 2023-10-17T01:14:41.739 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=2804d2ac-6c89-11ee-8db6-212e2dc638e7" will pass to kernel 2023-10-17T01:14:41.744 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-17T01:14:41.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:41.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /tmp/cephfs 2023-10-17T01:14:41.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:41.823 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-17T01:14:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:41 smithi060 ceph-mon[134136]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-17T01:14:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:41 smithi181 ceph-mon[134520]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-17T01:14:41.921 INFO:teuthology.orchestra.run.smithi045.stdout:client25253 2023-10-17T01:14:41.921 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:14:41.922 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-17T01:14:41.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:41.967 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-17T01:14:42.094 INFO:teuthology.orchestra.run.smithi045.stdout:client25253 2023-10-17T01:14:42.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:42.095 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-17T01:14:42.183 INFO:teuthology.orchestra.run.smithi045.stdout:client25253 2023-10-17T01:14:42.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:14:42.184 DEBUG:teuthology.orchestra.run.smithi045:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7.client25253/status') 2023-10-17T01:14:42.293 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.25253 (3)172.21.15.45:0/3460609968 2023-10-17T01:14:42.293 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2023-10-17T01:14:42.294 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2023-10-17T01:14:42.294 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2023-10-17T01:14:42.294 INFO:teuthology.orchestra.run.smithi045.stderr:71 bytes copied, 5.6345e-05 s, 1.3 MB/s 2023-10-17T01:14:42.297 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-17T01:14:42.309 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-17T01:14:42.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:42.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 -- 172.21.15.45:0/1457693630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48101f80 msgr2=0x7f2c481023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:42.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/1457693630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48101f80 0x7f2c481023e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c380099c0 tx=0x7f2c38009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 -- 172.21.15.45:0/1457693630 shutdown_connections 2023-10-17T01:14:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/1457693630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c48102920 0x7f2c481091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/1457693630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 0x7f2c481011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/1457693630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48101f80 0x7f2c481023e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 -- 172.21.15.45:0/1457693630 >> 172.21.15.45:0/1457693630 conn(0x7f2c480fc300 msgr2=0x7f2c480fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:42.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 -- 172.21.15.45:0/1457693630 shutdown_connections 2023-10-17T01:14:42.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.538+0000 7f2c4fbff700 1 -- 172.21.15.45:0/1457693630 wait complete. 2023-10-17T01:14:42.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.539+0000 7f2c4fbff700 1 Processor -- start 2023-10-17T01:14:42.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.539+0000 7f2c4fbff700 1 -- start start 2023-10-17T01:14:42.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4fbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 0x7f2c48198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:42.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4fbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c48101f80 0x7f2c481991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4fbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 0x7f2c481a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4fbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481a0810 con 0x7f2c48102920 2023-10-17T01:14:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4fbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481a0980 con 0x7f2c48101f80 2023-10-17T01:14:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4fbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481a0b40 con 0x7f2c48100d90 2023-10-17T01:14:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4e19c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 0x7f2c481a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4e19c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 0x7f2c481a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38926/0 (socket says 172.21.15.45:38926) 2023-10-17T01:14:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4e19c700 1 -- 172.21.15.45:0/3058372386 learned_addr learned my addr 172.21.15.45:0/3058372386 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4d99b700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 0x7f2c48198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.540+0000 7f2c4d19a700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c48101f80 0x7f2c481991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:42.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4d99b700 1 -- 172.21.15.45:0/3058372386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c48101f80 msgr2=0x7f2c481991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:42.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4d99b700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c48101f80 0x7f2c481991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4d99b700 1 -- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 msgr2=0x7f2c481a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:42.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4d99b700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 0x7f2c481a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4d99b700 1 -- 172.21.15.45:0/3058372386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380095f0 con 0x7f2c48100d90 2023-10-17T01:14:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4e19c700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 0x7f2c481a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4d99b700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 0x7f2c48198ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9c0 tx=0x7f2c4400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44004230 con 0x7f2c48100d90 2023-10-17T01:14:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c44004390 con 0x7f2c48100d90 2023-10-17T01:14:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c481a0e20 con 0x7f2c48100d90 2023-10-17T01:14:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44003d30 con 0x7f2c48100d90 2023-10-17T01:14:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.541+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c481a11f0 con 0x7f2c48100d90 2023-10-17T01:14:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.542+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4810c610 con 0x7f2c48100d90 2023-10-17T01:14:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.544+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2c44005030 con 0x7f2c48100d90 2023-10-17T01:14:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.546+0000 7f2c3effd700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c340a0b70 0x7f2c340a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.546+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2c44014070 con 0x7f2c48100d90 2023-10-17T01:14:42.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.546+0000 7f2c4d19a700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c340a0b70 0x7f2c340a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:42.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.546+0000 7f2c4d19a700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c340a0b70 0x7f2c340a3030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b580 tx=0x7f2c38005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:42.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.548+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2c44095fb0 con 0x7f2c48100d90 2023-10-17T01:14:42.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.763+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4810c7a0 con 0x7f2c48100d90 2023-10-17T01:14:42.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.766+0000 7f2c3effd700 1 -- 172.21.15.45:0/3058372386 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f2c440995d0 con 0x7f2c48100d90 2023-10-17T01:14:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:42.767 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:42.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.767+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c340a0b70 msgr2=0x7f2c340a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:42.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.767+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c340a0b70 0x7f2c340a3030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b580 tx=0x7f2c38005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.767+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 msgr2=0x7f2c48198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:42.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.767+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 0x7f2c48198ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9c0 tx=0x7f2c4400ecd0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 shutdown_connections 2023-10-17T01:14:42.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2c340a0b70 0x7f2c340a3030 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c48102920 0x7f2c481a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c48100d90 0x7f2c48198ca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 --2- 172.21.15.45:0/3058372386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c48101f80 0x7f2c481991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 >> 172.21.15.45:0/3058372386 conn(0x7f2c480fc300 msgr2=0x7f2c481077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:42.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 shutdown_connections 2023-10-17T01:14:42.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.768+0000 7f2c4fbff700 1 -- 172.21.15.45:0/3058372386 wait complete. 2023-10-17T01:14:42.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:42.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:42.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 -- 172.21.15.45:0/1553467562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c641028f0 msgr2=0x7f8c641091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:42.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 --2- 172.21.15.45:0/1553467562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c641028f0 0x7f8c641091a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00b0d0 tx=0x7f8c5c00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 -- 172.21.15.45:0/1553467562 shutdown_connections 2023-10-17T01:14:42.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 --2- 172.21.15.45:0/1553467562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c641028f0 0x7f8c641091a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 --2- 172.21.15.45:0/1553467562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 --2- 172.21.15.45:0/1553467562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c64101f50 0x7f8c641023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 -- 172.21.15.45:0/1553467562 >> 172.21.15.45:0/1553467562 conn(0x7f8c640fc2d0 msgr2=0x7f8c640fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 -- 172.21.15.45:0/1553467562 shutdown_connections 2023-10-17T01:14:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.996+0000 7f8c69dff700 1 -- 172.21.15.45:0/1553467562 wait complete. 2023-10-17T01:14:43.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 Processor -- start 2023-10-17T01:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 -- start start 2023-10-17T01:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c64101f50 0x7f8c64197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c641028f0 0x7f8c6419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419e6d0 con 0x7f8c64101f50 2023-10-17T01:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419e810 con 0x7f8c64100d60 2023-10-17T01:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.997+0000 7f8c69dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419e9a0 con 0x7f8c641028f0 2023-10-17T01:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c62ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c64101f50 0x7f8c64197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38130/0 (socket says 172.21.15.45:38130) 2023-10-17T01:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c62ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c64101f50 0x7f8c64197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38954/0 (socket says 172.21.15.45:38954) 2023-10-17T01:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 -- 172.21.15.45:0/940011125 learned_addr learned my addr 172.21.15.45:0/940011125 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c63fff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c641028f0 0x7f8c6419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 -- 172.21.15.45:0/940011125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c641028f0 msgr2=0x7f8c6419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c641028f0 0x7f8c6419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 -- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c64101f50 msgr2=0x7f8c64197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c64101f50 0x7f8c64197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 -- 172.21.15.45:0/940011125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c009d00 con 0x7f8c64100d60 2023-10-17T01:14:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.998+0000 7f8c637fe700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64196b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400d980 tx=0x7f8c5400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.999+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004020 con 0x7f8c64100d60 2023-10-17T01:14:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.999+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c64072fe0 con 0x7f8c64100d60 2023-10-17T01:14:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.999+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5400f690 con 0x7f8c64100d60 2023-10-17T01:14:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.999+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54003a60 con 0x7f8c64100d60 2023-10-17T01:14:43.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:42.999+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c64073540 con 0x7f8c64100d60 2023-10-17T01:14:43.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.000+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c641082f0 con 0x7f8c64100d60 2023-10-17T01:14:43.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.002+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8c5400fb00 con 0x7f8c64100d60 2023-10-17T01:14:43.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.003+0000 7f8c60ff9700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8c4c0a0b20 0x7f8c4c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.003+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c540cc0d0 con 0x7f8c64100d60 2023-10-17T01:14:43.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.003+0000 7f8c62ffd700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8c4c0a0b20 0x7f8c4c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.004+0000 7f8c62ffd700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8c4c0a0b20 0x7f8c4c0a2fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c58010d80 tx=0x7f8c58012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:43.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.006+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f8c54095a30 con 0x7f8c64100d60 2023-10-17T01:14:43.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.216+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c6402cee0 con 0x7f8c64100d60 2023-10-17T01:14:43.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.217+0000 7f8c60ff9700 1 -- 172.21.15.45:0/940011125 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c54099050 con 0x7f8c64100d60 2023-10-17T01:14:43.219 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:43.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8c4c0a0b20 msgr2=0x7f8c4c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8c4c0a0b20 0x7f8c4c0a2fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c58010d80 tx=0x7f8c58012040 comp rx=0 tx=0).stop 2023-10-17T01:14:43.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 msgr2=0x7f8c64196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64196b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400d980 tx=0x7f8c5400dc90 comp rx=0 tx=0).stop 2023-10-17T01:14:43.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 shutdown_connections 2023-10-17T01:14:43.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8c4c0a0b20 0x7f8c4c0a2fe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c641028f0 0x7f8c6419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64100d60 0x7f8c64196b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 --2- 172.21.15.45:0/940011125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c64101f50 0x7f8c64197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 >> 172.21.15.45:0/940011125 conn(0x7f8c640fc2d0 msgr2=0x7f8c64105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:43.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 shutdown_connections 2023-10-17T01:14:43.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.220+0000 7f8c69dff700 1 -- 172.21.15.45:0/940011125 wait complete. 2023-10-17T01:14:43.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:43.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 -- 172.21.15.45:0/3276433809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c108c30 msgr2=0x7fdd1c10b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 --2- 172.21.15.45:0/3276433809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c108c30 0x7fdd1c10b0c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b0a0 tx=0x7fdd1400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 -- 172.21.15.45:0/3276433809 shutdown_connections 2023-10-17T01:14:43.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 --2- 172.21.15.45:0/3276433809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c108c30 0x7fdd1c10b0c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 --2- 172.21.15.45:0/3276433809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c074d10 0x7fdd1c073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 --2- 172.21.15.45:0/3276433809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1c073730 0x7fdd1c073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 -- 172.21.15.45:0/3276433809 >> 172.21.15.45:0/3276433809 conn(0x7fdd1c0fc290 msgr2=0x7fdd1c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:43.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 -- 172.21.15.45:0/3276433809 shutdown_connections 2023-10-17T01:14:43.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.448+0000 7fdd213ff700 1 -- 172.21.15.45:0/3276433809 wait complete. 2023-10-17T01:14:43.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 Processor -- start 2023-10-17T01:14:43.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 -- start start 2023-10-17T01:14:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1c073730 0x7fdd1c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 0x7fdd1c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c108c30 0x7fdd1c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1a08c0 con 0x7fdd1c074d10 2023-10-17T01:14:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1a0a00 con 0x7fdd1c108c30 2023-10-17T01:14:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.449+0000 7fdd213ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1a0b90 con 0x7fdd1c073730 2023-10-17T01:14:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 0x7fdd1c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c108c30 0x7fdd1c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 0x7fdd1c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38966/0 (socket says 172.21.15.45:38966) 2023-10-17T01:14:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 -- 172.21.15.45:0/1836747876 learned_addr learned my addr 172.21.15.45:0/1836747876 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1affd700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1c073730 0x7fdd1c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 -- 172.21.15.45:0/1836747876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1c073730 msgr2=0x7fdd1c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1c073730 0x7fdd1c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 -- 172.21.15.45:0/1836747876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c108c30 msgr2=0x7fdd1c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c108c30 0x7fdd1c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 -- 172.21.15.45:0/1836747876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009d00 con 0x7fdd1c074d10 2023-10-17T01:14:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1b7fe700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c108c30 0x7fdd1c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.450+0000 7fdd1a7fc700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 0x7fdd1c199280 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000e9a0 tx=0x7fdd1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.451+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10009790 con 0x7fdd1c074d10 2023-10-17T01:14:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.451+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd10010430 con 0x7fdd1c074d10 2023-10-17T01:14:43.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.451+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10018680 con 0x7fdd1c074d10 2023-10-17T01:14:43.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.451+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c1a0e70 con 0x7fdd1c074d10 2023-10-17T01:14:43.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.451+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c1a12a0 con 0x7fdd1c074d10 2023-10-17T01:14:43.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.452+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c107330 con 0x7fdd1c074d10 2023-10-17T01:14:43.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.454+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdd100105a0 con 0x7fdd1c074d10 2023-10-17T01:14:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.455+0000 7fdd03fff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdd040a0b20 0x7fdd040a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.455+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdd10014070 con 0x7fdd1c074d10 2023-10-17T01:14:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.455+0000 7fdd1affd700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdd040a0b20 0x7fdd040a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.456+0000 7fdd1affd700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdd040a0b20 0x7fdd040a2fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009c60 tx=0x7fdd0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.459+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fdd100520b0 con 0x7fdd1c074d10 2023-10-17T01:14:43.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.674+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd1c061a60 con 0x7fdd1c074d10 2023-10-17T01:14:43.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.674+0000 7fdd03fff700 1 -- 172.21.15.45:0/1836747876 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd10095c90 con 0x7fdd1c074d10 2023-10-17T01:14:43.676 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:43.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdd040a0b20 msgr2=0x7fdd040a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdd040a0b20 0x7fdd040a2fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009c60 tx=0x7fdd0c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:43.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 msgr2=0x7fdd1c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 0x7fdd1c199280 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000e9a0 tx=0x7fdd1000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 shutdown_connections 2023-10-17T01:14:43.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdd040a0b20 0x7fdd040a2fe0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd1c108c30 0x7fdd1c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd1c073730 0x7fdd1c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 --2- 172.21.15.45:0/1836747876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd1c074d10 0x7fdd1c199280 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 >> 172.21.15.45:0/1836747876 conn(0x7fdd1c0fc290 msgr2=0x7fdd1c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:43.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 shutdown_connections 2023-10-17T01:14:43.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.677+0000 7fdd213ff700 1 -- 172.21.15.45:0/1836747876 wait complete. 2023-10-17T01:14:43.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:43 smithi045 ceph-mon[98474]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2023-10-17T01:14:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:43 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3058372386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:43 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/940011125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:43 smithi060 ceph-mon[134136]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2023-10-17T01:14:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:43 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3058372386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:43 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/940011125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:43.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 -- 172.21.15.45:0/2064064814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70100d60 msgr2=0x7f3a70101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2064064814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70100d60 0x7f3a70101180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009990 tx=0x7f3a60009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 -- 172.21.15.45:0/2064064814 shutdown_connections 2023-10-17T01:14:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2064064814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a701028f0 0x7f3a701091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2064064814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70100d60 0x7f3a70101180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2064064814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70101f50 0x7f3a701023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 -- 172.21.15.45:0/2064064814 >> 172.21.15.45:0/2064064814 conn(0x7f3a700fc2d0 msgr2=0x7f3a700fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.904+0000 7f3a75bff700 1 -- 172.21.15.45:0/2064064814 shutdown_connections 2023-10-17T01:14:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.905+0000 7f3a75bff700 1 -- 172.21.15.45:0/2064064814 wait complete. 2023-10-17T01:14:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:43 smithi181 ceph-mon[134520]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2023-10-17T01:14:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:43 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3058372386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:43.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:43 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/940011125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:43.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.906+0000 7f3a75bff700 1 Processor -- start 2023-10-17T01:14:43.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.906+0000 7f3a75bff700 1 -- start start 2023-10-17T01:14:43.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a75bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 0x7f3a70196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a75bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70101f50 0x7f3a70197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a75bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a701028f0 0x7f3a7019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a75bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7019e670 con 0x7f3a701028f0 2023-10-17T01:14:43.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a75bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7019e7b0 con 0x7f3a70100d60 2023-10-17T01:14:43.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a75bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7019e940 con 0x7f3a70101f50 2023-10-17T01:14:43.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 0x7f3a70196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 0x7f3a70196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38170/0 (socket says 172.21.15.45:38170) 2023-10-17T01:14:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/2749357596 learned_addr learned my addr 172.21.15.45:0/2749357596 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6ffff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a701028f0 0x7f3a7019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6effd700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70101f50 0x7f3a70197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/2749357596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70101f50 msgr2=0x7f3a70197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70101f50 0x7f3a70197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a701028f0 msgr2=0x7f3a7019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a701028f0 0x7f3a7019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.907+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/2749357596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600095f0 con 0x7f3a70100d60 2023-10-17T01:14:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.908+0000 7f3a6f7fe700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 0x7f3a70196af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005720 tx=0x7f3a60009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.908+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6001d070 con 0x7f3a70100d60 2023-10-17T01:14:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.908+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a60021410 con 0x7f3a70100d60 2023-10-17T01:14:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.908+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60017650 con 0x7f3a70100d60 2023-10-17T01:14:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.908+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a70072f80 con 0x7f3a70100d60 2023-10-17T01:14:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.908+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a700733e0 con 0x7f3a70100d60 2023-10-17T01:14:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.909+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a70108340 con 0x7f3a70100d60 2023-10-17T01:14:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.911+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f3a60021580 con 0x7f3a70100d60 2023-10-17T01:14:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.912+0000 7f3a6cff9700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3a580a0b20 0x7f3a580a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.913+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3a600cd7c0 con 0x7f3a70100d60 2023-10-17T01:14:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.913+0000 7f3a6effd700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3a580a0b20 0x7f3a580a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.913+0000 7f3a6effd700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3a580a0b20 0x7f3a580a2fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009c60 tx=0x7f3a64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:43.915+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3a600971b0 con 0x7f3a70100d60 2023-10-17T01:14:44.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.125+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a7002ce80 con 0x7f3a70100d60 2023-10-17T01:14:44.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.125+0000 7f3a6cff9700 1 -- 172.21.15.45:0/2749357596 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a6009a7d0 con 0x7f3a70100d60 2023-10-17T01:14:44.127 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3a580a0b20 msgr2=0x7f3a580a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3a580a0b20 0x7f3a580a2fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009c60 tx=0x7f3a64009260 comp rx=0 tx=0).stop 2023-10-17T01:14:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 msgr2=0x7f3a70196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 0x7f3a70196af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005720 tx=0x7f3a60009580 comp rx=0 tx=0).stop 2023-10-17T01:14:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 shutdown_connections 2023-10-17T01:14:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f3a580a0b20 0x7f3a580a2fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a701028f0 0x7f3a7019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a70100d60 0x7f3a70196af0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 --2- 172.21.15.45:0/2749357596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a70101f50 0x7f3a70197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 >> 172.21.15.45:0/2749357596 conn(0x7f3a700fc2d0 msgr2=0x7f3a70105480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 shutdown_connections 2023-10-17T01:14:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.128+0000 7f3a75bff700 1 -- 172.21.15.45:0/2749357596 wait complete. 2023-10-17T01:14:44.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 -- 172.21.15.45:0/351457806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 msgr2=0x7f95e01045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 --2- 172.21.15.45:0/351457806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01045e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009990 tx=0x7f95cc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 -- 172.21.15.45:0/351457806 shutdown_connections 2023-10-17T01:14:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 --2- 172.21.15.45:0/351457806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 0x7f95e010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 --2- 172.21.15.45:0/351457806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e01033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 --2- 172.21.15.45:0/351457806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01045e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 -- 172.21.15.45:0/351457806 >> 172.21.15.45:0/351457806 conn(0x7f95e00fe500 msgr2=0x7f95e0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:44.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 -- 172.21.15.45:0/351457806 shutdown_connections 2023-10-17T01:14:44.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.352+0000 7f95e6bff700 1 -- 172.21.15.45:0/351457806 wait complete. 2023-10-17T01:14:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.353+0000 7f95e6bff700 1 Processor -- start 2023-10-17T01:14:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 -- start start 2023-10-17T01:14:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 0x7f95e01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01a0730 con 0x7f95e0104b20 2023-10-17T01:14:44.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01a0870 con 0x7f95e0102f90 2023-10-17T01:14:44.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e6bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01a0a00 con 0x7f95e0104180 2023-10-17T01:14:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e519c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 0x7f95e01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38188/0 (socket says 172.21.15.45:38188) 2023-10-17T01:14:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95e499b700 1 -- 172.21.15.45:0/1234066471 learned_addr learned my addr 172.21.15.45:0/1234066471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.354+0000 7f95dffff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e499b700 1 -- 172.21.15.45:0/1234066471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 msgr2=0x7f95e01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e499b700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e499b700 1 -- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 msgr2=0x7f95e01a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e499b700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 0x7f95e01a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e499b700 1 -- 172.21.15.45:0/1234066471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0095f0 con 0x7f95e0102f90 2023-10-17T01:14:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95dffff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e519c700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 0x7f95e01a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e499b700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e0198bb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95d400e970 tx=0x7f95d400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4009790 con 0x7f95e0102f90 2023-10-17T01:14:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d400eec0 con 0x7f95e0102f90 2023-10-17T01:14:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e01a0ce0 con 0x7f95e0102f90 2023-10-17T01:14:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4003cf0 con 0x7f95e0102f90 2023-10-17T01:14:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.355+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e01a1190 con 0x7f95e0102f90 2023-10-17T01:14:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.356+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e010e810 con 0x7f95e0102f90 2023-10-17T01:14:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.359+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f95d400ccd0 con 0x7f95e0102f90 2023-10-17T01:14:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.360+0000 7f95ddffb700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95d00a0bf0 0x7f95d00a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.360+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f95d4014070 con 0x7f95e0102f90 2023-10-17T01:14:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.360+0000 7f95dffff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95d00a0bf0 0x7f95d00a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.361+0000 7f95dffff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95d00a0bf0 0x7f95d00a30b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b2e0 tx=0x7f95cc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.362+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f95d4095e90 con 0x7f95e0102f90 2023-10-17T01:14:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.582+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95e0109df0 con 0x7f95e0102f90 2023-10-17T01:14:44.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.585+0000 7f95ddffb700 1 -- 172.21.15.45:0/1234066471 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f95d40994b0 con 0x7f95e0102f90 2023-10-17T01:14:44.586 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:44.586 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95d00a0bf0 msgr2=0x7f95d00a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95d00a0bf0 0x7f95d00a30b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b2e0 tx=0x7f95cc005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 msgr2=0x7f95e0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e0198bb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95d400e970 tx=0x7f95d400ec80 comp rx=0 tx=0).stop 2023-10-17T01:14:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 shutdown_connections 2023-10-17T01:14:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95d00a0bf0 0x7f95d00a30b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95e0104b20 0x7f95e01a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e0102f90 0x7f95e0198bb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 --2- 172.21.15.45:0/1234066471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e0104180 0x7f95e01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 >> 172.21.15.45:0/1234066471 conn(0x7f95e00fe500 msgr2=0x7f95e01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:44.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 shutdown_connections 2023-10-17T01:14:44.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.586+0000 7f95e6bff700 1 -- 172.21.15.45:0/1234066471 wait complete. 2023-10-17T01:14:44.607 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:44.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:44 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1836747876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:44 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2749357596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:44.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.814+0000 7f02e77ff700 1 -- 172.21.15.45:0/2057605755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 msgr2=0x7f02e01064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.814+0000 7f02e77ff700 1 --2- 172.21.15.45:0/2057605755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01064e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009990 tx=0x7f02d0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 -- 172.21.15.45:0/2057605755 shutdown_connections 2023-10-17T01:14:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 --2- 172.21.15.45:0/2057605755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e0106a20 0x7f02e0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 --2- 172.21.15.45:0/2057605755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 0x7f02e0103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 --2- 172.21.15.45:0/2057605755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01064e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 -- 172.21.15.45:0/2057605755 >> 172.21.15.45:0/2057605755 conn(0x7f02e00fb1b0 msgr2=0x7f02e00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:44.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 -- 172.21.15.45:0/2057605755 shutdown_connections 2023-10-17T01:14:44.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.815+0000 7f02e77ff700 1 -- 172.21.15.45:0/2057605755 wait complete. 2023-10-17T01:14:44.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 Processor -- start 2023-10-17T01:14:44.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 -- start start 2023-10-17T01:14:44.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 0x7f02e0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e0106a20 0x7f02e01a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01a07f0 con 0x7f02e01040f0 2023-10-17T01:14:44.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01a0960 con 0x7f02e01017c0 2023-10-17T01:14:44.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e77ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01a0b20 con 0x7f02e0106a20 2023-10-17T01:14:44.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e4d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.816+0000 7f02e559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 0x7f02e0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e4d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39018/0 (socket says 172.21.15.45:39018) 2023-10-17T01:14:44.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e4d9a700 1 -- 172.21.15.45:0/3509547521 learned_addr learned my addr 172.21.15.45:0/3509547521 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:44.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e5d9c700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e0106a20 0x7f02e01a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e559b700 1 -- 172.21.15.45:0/3509547521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e0106a20 msgr2=0x7f02e01a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e559b700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e0106a20 0x7f02e01a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e559b700 1 -- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 msgr2=0x7f02e01991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:44.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e559b700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:44.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e559b700 1 -- 172.21.15.45:0/3509547521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00095f0 con 0x7f02e01017c0 2023-10-17T01:14:44.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e4d9a700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:44.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02e559b700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 0x7f02e0198c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9f0 tx=0x7f02dc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:44.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.817+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00c9c0 con 0x7f02e01017c0 2023-10-17T01:14:44.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.818+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02dc010430 con 0x7f02e01017c0 2023-10-17T01:14:44.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.818+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e01a0e00 con 0x7f02e01017c0 2023-10-17T01:14:44.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.818+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc018680 con 0x7f02e01017c0 2023-10-17T01:14:44.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.818+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e01a1220 con 0x7f02e01017c0 2023-10-17T01:14:44.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.819+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e01085f0 con 0x7f02e01017c0 2023-10-17T01:14:44.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.821+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f02dc010860 con 0x7f02e01017c0 2023-10-17T01:14:44.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.822+0000 7f02d67fc700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02cc0a0b20 0x7f02cc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:44.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.822+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02dc014070 con 0x7f02e01017c0 2023-10-17T01:14:44.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.822+0000 7f02e4d9a700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02cc0a0b20 0x7f02cc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:44.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.823+0000 7f02e4d9a700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02cc0a0b20 0x7f02cc0a2fe0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f02e019a180 tx=0x7f02d000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:44.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:44.824+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f02dc095dd0 con 0x7f02e01017c0 2023-10-17T01:14:44.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:44 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1836747876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:44.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:44 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2749357596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:44 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1836747876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:44 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2749357596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.033+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02e002cde0 con 0x7f02e01017c0 2023-10-17T01:14:45.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.034+0000 7f02d67fc700 1 -- 172.21.15.45:0/3509547521 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02dc0993f0 con 0x7f02e01017c0 2023-10-17T01:14:45.035 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02cc0a0b20 msgr2=0x7f02cc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02cc0a0b20 0x7f02cc0a2fe0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f02e019a180 tx=0x7f02d000b260 comp rx=0 tx=0).stop 2023-10-17T01:14:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 msgr2=0x7f02e0198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 0x7f02e0198c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9f0 tx=0x7f02dc00ed00 comp rx=0 tx=0).stop 2023-10-17T01:14:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 shutdown_connections 2023-10-17T01:14:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02cc0a0b20 0x7f02cc0a2fe0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e0106a20 0x7f02e01a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02e01017c0 0x7f02e0198c80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 --2- 172.21.15.45:0/3509547521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02e01040f0 0x7f02e01991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 >> 172.21.15.45:0/3509547521 conn(0x7f02e00fb1b0 msgr2=0x7f02e00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 shutdown_connections 2023-10-17T01:14:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.036+0000 7f02e77ff700 1 -- 172.21.15.45:0/3509547521 wait complete. 2023-10-17T01:14:45.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:45.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 -- 172.21.15.45:0/1270581038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888074d90 msgr2=0x7f98880731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 --2- 172.21.15.45:0/1270581038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888074d90 0x7f98880731e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9884009990 tx=0x7f9884009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 -- 172.21.15.45:0/1270581038 shutdown_connections 2023-10-17T01:14:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 --2- 172.21.15.45:0/1270581038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9888108c00 0x7f988810b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 --2- 172.21.15.45:0/1270581038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888074d90 0x7f98880731e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 --2- 172.21.15.45:0/1270581038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880737b0 0x7f9888073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.266+0000 7f988fbff700 1 -- 172.21.15.45:0/1270581038 >> 172.21.15.45:0/1270581038 conn(0x7f98880fc290 msgr2=0x7f98880fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.267+0000 7f988fbff700 1 -- 172.21.15.45:0/1270581038 shutdown_connections 2023-10-17T01:14:45.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.267+0000 7f988fbff700 1 -- 172.21.15.45:0/1270581038 wait complete. 2023-10-17T01:14:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.267+0000 7f988fbff700 1 Processor -- start 2023-10-17T01:14:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 -- start start 2023-10-17T01:14:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98880737b0 0x7f9888198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9888074d90 0x7f98881991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 0x7f98881a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881a07c0 con 0x7f9888108c00 2023-10-17T01:14:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881a0930 con 0x7f9888074d90 2023-10-17T01:14:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988fbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881a0af0 con 0x7f98880737b0 2023-10-17T01:14:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988e19c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 0x7f98881a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988d19a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9888074d90 0x7f98881991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988e19c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 0x7f98881a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39022/0 (socket says 172.21.15.45:39022) 2023-10-17T01:14:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988e19c700 1 -- 172.21.15.45:0/603939430 learned_addr learned my addr 172.21.15.45:0/603939430 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.268+0000 7f988d99b700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98880737b0 0x7f9888198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988e19c700 1 -- 172.21.15.45:0/603939430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98880737b0 msgr2=0x7f9888198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988e19c700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98880737b0 0x7f9888198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988e19c700 1 -- 172.21.15.45:0/603939430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9888074d90 msgr2=0x7f98881991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988e19c700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9888074d90 0x7f98881991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988e19c700 1 -- 172.21.15.45:0/603939430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840095f0 con 0x7f9888108c00 2023-10-17T01:14:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988d99b700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98880737b0 0x7f9888198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988e19c700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 0x7f98881a01f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f988000ef30 tx=0x7f988000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880018070 con 0x7f9888108c00 2023-10-17T01:14:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98881a0d70 con 0x7f9888108c00 2023-10-17T01:14:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988000f040 con 0x7f9888108c00 2023-10-17T01:14:45.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000cdb0 con 0x7f9888108c00 2023-10-17T01:14:45.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.269+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98881a11c0 con 0x7f9888108c00 2023-10-17T01:14:45.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.270+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98881072b0 con 0x7f9888108c00 2023-10-17T01:14:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.272+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9880003a00 con 0x7f9888108c00 2023-10-17T01:14:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.274+0000 7f987effd700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f98740a0b20 0x7f98740a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.274+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f98800cbd60 con 0x7f9888108c00 2023-10-17T01:14:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.274+0000 7f988d99b700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f98740a0b20 0x7f98740a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.274+0000 7f988d99b700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f98740a0b20 0x7f98740a2fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9884006010 tx=0x7f988400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.278+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f9880095750 con 0x7f9888108c00 2023-10-17T01:14:45.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.492+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9888061a30 con 0x7f9888108c00 2023-10-17T01:14:45.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.493+0000 7f987effd700 1 -- 172.21.15.45:0/603939430 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9880098d70 con 0x7f9888108c00 2023-10-17T01:14:45.494 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:45.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f98740a0b20 msgr2=0x7f98740a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f98740a0b20 0x7f98740a2fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9884006010 tx=0x7f988400b260 comp rx=0 tx=0).stop 2023-10-17T01:14:45.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 msgr2=0x7f98881a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 0x7f98881a01f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f988000ef30 tx=0x7f988000c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 shutdown_connections 2023-10-17T01:14:45.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f98740a0b20 0x7f98740a2fe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9888108c00 0x7f98881a01f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98880737b0 0x7f9888198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 --2- 172.21.15.45:0/603939430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9888074d90 0x7f98881991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 >> 172.21.15.45:0/603939430 conn(0x7f98880fc290 msgr2=0x7f98880fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:45.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 shutdown_connections 2023-10-17T01:14:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.495+0000 7f988fbff700 1 -- 172.21.15.45:0/603939430 wait complete. 2023-10-17T01:14:45.515 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:45.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 -- 172.21.15.45:0/1779727894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60100d40 msgr2=0x7fab60101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 --2- 172.21.15.45:0/1779727894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60100d40 0x7fab60101160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab500098f0 tx=0x7fab50009c00 comp rx=0 tx=0).stop 2023-10-17T01:14:45.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 -- 172.21.15.45:0/1779727894 shutdown_connections 2023-10-17T01:14:45.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 --2- 172.21.15.45:0/1779727894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab601028d0 0x7fab60109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 --2- 172.21.15.45:0/1779727894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60100d40 0x7fab60101160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 --2- 172.21.15.45:0/1779727894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab60101f30 0x7fab60102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.722+0000 7fab64fff700 1 -- 172.21.15.45:0/1779727894 >> 172.21.15.45:0/1779727894 conn(0x7fab600fc290 msgr2=0x7fab600fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:45.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.723+0000 7fab64fff700 1 -- 172.21.15.45:0/1779727894 shutdown_connections 2023-10-17T01:14:45.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.723+0000 7fab64fff700 1 -- 172.21.15.45:0/1779727894 wait complete. 2023-10-17T01:14:45.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.724+0000 7fab64fff700 1 Processor -- start 2023-10-17T01:14:45.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.724+0000 7fab64fff700 1 -- start start 2023-10-17T01:14:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab64fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab60100d40 0x7fab60196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab64fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60101f30 0x7fab60197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab64fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 0x7fab6019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab64fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6019e660 con 0x7fab601028d0 2023-10-17T01:14:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab64fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6019e7a0 con 0x7fab60100d40 2023-10-17T01:14:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab64fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6019e930 con 0x7fab60101f30 2023-10-17T01:14:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 0x7fab6019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 0x7fab6019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39038/0 (socket says 172.21.15.45:39038) 2023-10-17T01:14:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 -- 172.21.15.45:0/926162701 learned_addr learned my addr 172.21.15.45:0/926162701 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:45.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5dd9b700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60101f30 0x7fab60197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5e59c700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab60100d40 0x7fab60196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 -- 172.21.15.45:0/926162701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60101f30 msgr2=0x7fab60197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60101f30 0x7fab60197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 -- 172.21.15.45:0/926162701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab60100d40 msgr2=0x7fab60196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab60100d40 0x7fab60196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.725+0000 7fab5ed9d700 1 -- 172.21.15.45:0/926162701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009520 con 0x7fab601028d0 2023-10-17T01:14:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.726+0000 7fab5ed9d700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 0x7fab6019e090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab5800ef30 tx=0x7fab5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.726+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58018070 con 0x7fab601028d0 2023-10-17T01:14:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.726+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5800f040 con 0x7fab601028d0 2023-10-17T01:14:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.726+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800ccf0 con 0x7fab601028d0 2023-10-17T01:14:45.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.726+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6019ebb0 con 0x7fab601028d0 2023-10-17T01:14:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.726+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6019f070 con 0x7fab601028d0 2023-10-17T01:14:45.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.727+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6010c5a0 con 0x7fab601028d0 2023-10-17T01:14:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.729+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fab58003a00 con 0x7fab601028d0 2023-10-17T01:14:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.731+0000 7fab4f7fe700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fab480a0b20 0x7fab480a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.732+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fab580cbc60 con 0x7fab601028d0 2023-10-17T01:14:45.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.732+0000 7fab5e59c700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fab480a0b20 0x7fab480a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:45.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.732+0000 7fab5e59c700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fab480a0b20 0x7fab480a2fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fab50009fd0 tx=0x7fab50019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:45.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.736+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fab58095650 con 0x7fab601028d0 2023-10-17T01:14:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:45 smithi045 ceph-mon[98474]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-17T01:14:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:45 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1234066471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:45 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3509547521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:45 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/603939430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:45 smithi060 ceph-mon[134136]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-17T01:14:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:45 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1234066471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:45 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3509547521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:45 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/603939430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:45 smithi181 ceph-mon[134520]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-17T01:14:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:45 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1234066471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:45 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3509547521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:45 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/603939430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:45.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.946+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab60107cd0 con 0x7fab601028d0 2023-10-17T01:14:45.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.946+0000 7fab4f7fe700 1 -- 172.21.15.45:0/926162701 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab58098c70 con 0x7fab601028d0 2023-10-17T01:14:45.948 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:45.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fab480a0b20 msgr2=0x7fab480a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fab480a0b20 0x7fab480a2fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fab50009fd0 tx=0x7fab50019040 comp rx=0 tx=0).stop 2023-10-17T01:14:45.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 msgr2=0x7fab6019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:45.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 0x7fab6019e090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab5800ef30 tx=0x7fab5800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 shutdown_connections 2023-10-17T01:14:45.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fab480a0b20 0x7fab480a2fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab601028d0 0x7fab6019e090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab60100d40 0x7fab60196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 --2- 172.21.15.45:0/926162701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab60101f30 0x7fab60197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:45.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 >> 172.21.15.45:0/926162701 conn(0x7fab600fc290 msgr2=0x7fab600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:45.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 shutdown_connections 2023-10-17T01:14:45.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:45.949+0000 7fab64fff700 1 -- 172.21.15.45:0/926162701 wait complete. 2023-10-17T01:14:45.969 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-17T01:14:45.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:46.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.174+0000 7f95315ff700 1 -- 172.21.15.45:0/1841665140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c1028e0 msgr2=0x7f952c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.174+0000 7f95315ff700 1 --2- 172.21.15.45:0/1841665140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c1028e0 0x7f952c109190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f952400b0d0 tx=0x7f952400b3e0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 -- 172.21.15.45:0/1841665140 shutdown_connections 2023-10-17T01:14:46.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 --2- 172.21.15.45:0/1841665140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c1028e0 0x7f952c109190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 --2- 172.21.15.45:0/1841665140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952c100d50 0x7f952c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 --2- 172.21.15.45:0/1841665140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c101f40 0x7f952c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 -- 172.21.15.45:0/1841665140 >> 172.21.15.45:0/1841665140 conn(0x7f952c0fc280 msgr2=0x7f952c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 -- 172.21.15.45:0/1841665140 shutdown_connections 2023-10-17T01:14:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.175+0000 7f95315ff700 1 -- 172.21.15.45:0/1841665140 wait complete. 2023-10-17T01:14:46.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 Processor -- start 2023-10-17T01:14:46.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 -- start start 2023-10-17T01:14:46.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952c100d50 0x7f952c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 0x7f952c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c1028e0 0x7f952c1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1a0860 con 0x7f952c101f40 2023-10-17T01:14:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1a09d0 con 0x7f952c1028e0 2023-10-17T01:14:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f95315ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1a0b90 con 0x7f952c100d50 2023-10-17T01:14:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f952a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 0x7f952c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f952a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 0x7f952c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39058/0 (socket says 172.21.15.45:39058) 2023-10-17T01:14:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.176+0000 7f952a7fc700 1 -- 172.21.15.45:0/468677591 learned_addr learned my addr 172.21.15.45:0/468677591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952affd700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952c100d50 0x7f952c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952b7fe700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c1028e0 0x7f952c1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952a7fc700 1 -- 172.21.15.45:0/468677591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952c100d50 msgr2=0x7f952c198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952a7fc700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952c100d50 0x7f952c198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952a7fc700 1 -- 172.21.15.45:0/468677591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c1028e0 msgr2=0x7f952c1a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952a7fc700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c1028e0 0x7f952c1a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952a7fc700 1 -- 172.21.15.45:0/468677591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524009d00 con 0x7f952c101f40 2023-10-17T01:14:46.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f952a7fc700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 0x7f952c199260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f952000e9a0 tx=0x7f952000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:46.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520009790 con 0x7f952c101f40 2023-10-17T01:14:46.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f952000cbd0 con 0x7f952c101f40 2023-10-17T01:14:46.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.177+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c1a0e70 con 0x7f952c101f40 2023-10-17T01:14:46.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.178+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520018680 con 0x7f952c101f40 2023-10-17T01:14:46.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.178+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c1a1290 con 0x7f952c101f40 2023-10-17T01:14:46.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.179+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c10c5c0 con 0x7f952c101f40 2023-10-17T01:14:46.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.181+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f952000cd40 con 0x7f952c101f40 2023-10-17T01:14:46.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.182+0000 7f9513fff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95140a0b70 0x7f95140a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.182+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9520014070 con 0x7f952c101f40 2023-10-17T01:14:46.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.182+0000 7f952affd700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95140a0b70 0x7f95140a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.183+0000 7f952affd700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95140a0b70 0x7f95140a3030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f952c19a1f0 tx=0x7f951c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:46.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.186+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f95200520b0 con 0x7f952c101f40 2023-10-17T01:14:46.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.444+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f952c1a18d0 con 0x7f952c101f40 2023-10-17T01:14:46.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.444+0000 7f9513fff700 1 -- 172.21.15.45:0/468677591 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f9520095dd0 con 0x7f952c101f40 2023-10-17T01:14:46.446 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:46.446 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:46.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95140a0b70 msgr2=0x7f95140a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95140a0b70 0x7f95140a3030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f952c19a1f0 tx=0x7f951c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:46.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 msgr2=0x7f952c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 0x7f952c199260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f952000e9a0 tx=0x7f952000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 shutdown_connections 2023-10-17T01:14:46.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f95140a0b70 0x7f95140a3030 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c1028e0 0x7f952c1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952c100d50 0x7f952c198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 --2- 172.21.15.45:0/468677591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952c101f40 0x7f952c199260 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 >> 172.21.15.45:0/468677591 conn(0x7f952c0fc280 msgr2=0x7f952c107780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:46.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 shutdown_connections 2023-10-17T01:14:46.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.447+0000 7f95315ff700 1 -- 172.21.15.45:0/468677591 wait complete. 2023-10-17T01:14:46.453 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:46.468 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-17T01:14:46.468 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-17T01:14:46.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-17T01:14:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 -- 172.21.15.45:0/3741848475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d41028f0 msgr2=0x7fd6d41091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/3741848475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d41028f0 0x7fd6d41091a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b0d0 tx=0x7fd6cc00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 -- 172.21.15.45:0/3741848475 shutdown_connections 2023-10-17T01:14:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/3741848475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d41028f0 0x7fd6d41091a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/3741848475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d4100d60 0x7fd6d4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/3741848475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d4101f50 0x7fd6d41023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 -- 172.21.15.45:0/3741848475 >> 172.21.15.45:0/3741848475 conn(0x7fd6d40fc290 msgr2=0x7fd6d40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 -- 172.21.15.45:0/3741848475 shutdown_connections 2023-10-17T01:14:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 -- 172.21.15.45:0/3741848475 wait complete. 2023-10-17T01:14:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 Processor -- start 2023-10-17T01:14:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.675+0000 7fd6d9fff700 1 -- start start 2023-10-17T01:14:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d9fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 0x7fd6d4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d9fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d4101f50 0x7fd6d4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d9fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d41028f0 0x7fd6d41a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d9fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41a0860 con 0x7fd6d4100d60 2023-10-17T01:14:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d9fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41a09d0 con 0x7fd6d4101f50 2023-10-17T01:14:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d9fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41a0b90 con 0x7fd6d41028f0 2023-10-17T01:14:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 0x7fd6d4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 0x7fd6d4198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39084/0 (socket says 172.21.15.45:39084) 2023-10-17T01:14:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 -- 172.21.15.45:0/575807979 learned_addr learned my addr 172.21.15.45:0/575807979 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d3fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d41028f0 0x7fd6d41a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d2ffd700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d4101f50 0x7fd6d4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 -- 172.21.15.45:0/575807979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d41028f0 msgr2=0x7fd6d41a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d41028f0 0x7fd6d41a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 -- 172.21.15.45:0/575807979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d4101f50 msgr2=0x7fd6d4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d4101f50 0x7fd6d4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 -- 172.21.15.45:0/575807979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc009d00 con 0x7fd6d4100d60 2023-10-17T01:14:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d3fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d41028f0 0x7fd6d41a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.676+0000 7fd6d37fe700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 0x7fd6d4198d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e970 tx=0x7fd6c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.677+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4009790 con 0x7fd6d4100d60 2023-10-17T01:14:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.677+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c40044e0 con 0x7fd6d4100d60 2023-10-17T01:14:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.677+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d41a0e70 con 0x7fd6d4100d60 2023-10-17T01:14:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.677+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4003e30 con 0x7fd6d4100d60 2023-10-17T01:14:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.677+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d41a12a0 con 0x7fd6d4100d60 2023-10-17T01:14:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.678+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d410c5d0 con 0x7fd6d4100d60 2023-10-17T01:14:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.680+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd6c4004000 con 0x7fd6d4100d60 2023-10-17T01:14:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.682+0000 7fd6d0ff9700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd6bc0a0bf0 0x7fd6bc0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.682+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd6c4014070 con 0x7fd6d4100d60 2023-10-17T01:14:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.682+0000 7fd6d2ffd700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd6bc0a0bf0 0x7fd6bc0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.682+0000 7fd6d2ffd700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd6bc0a0bf0 0x7fd6bc0a30b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009bf0 tx=0x7fd6c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.686+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fd6c4095e10 con 0x7fd6d4100d60 2023-10-17T01:14:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:46 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/926162701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:46.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:46 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/468677591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:46 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/926162701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:46.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:46 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/468677591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:46.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.892+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6d41087b0 con 0x7fd6bc0a0bf0 2023-10-17T01:14:46.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.904+0000 7fd6d0ff9700 1 -- 172.21.15.45:0/575807979 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7fd6d41087b0 con 0x7fd6bc0a0bf0 2023-10-17T01:14:46.910 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:46.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:46 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/926162701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:46.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:46 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/468677591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd6bc0a0bf0 msgr2=0x7fd6bc0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd6bc0a0bf0 0x7fd6bc0a30b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009bf0 tx=0x7fd6c8009260 comp rx=0 tx=0).stop 2023-10-17T01:14:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 msgr2=0x7fd6d4198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:46.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 0x7fd6d4198d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e970 tx=0x7fd6c400ed30 comp rx=0 tx=0).stop 2023-10-17T01:14:46.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 shutdown_connections 2023-10-17T01:14:46.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd6bc0a0bf0 0x7fd6bc0a30b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d41028f0 0x7fd6d41a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d4100d60 0x7fd6d4198d20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 --2- 172.21.15.45:0/575807979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d4101f50 0x7fd6d4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 >> 172.21.15.45:0/575807979 conn(0x7fd6d40fc290 msgr2=0x7fd6d40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 shutdown_connections 2023-10-17T01:14:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:46.909+0000 7fd6d9fff700 1 -- 172.21.15.45:0/575807979 wait complete. 2023-10-17T01:14:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:14:46.929 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-17T01:14:46.458100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655086,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1122,"num_read_kb":677,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334820,"kb_used_data":11396,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607004,"statfs":{"total":1151940427776,"available":1151597572096,"internally_reserved":0,"allocated":11669504,"data_stored":7279338,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1022,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":63,"num_write":14,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003018"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474988+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:14:32.474988+0000","last_peered":"2023-10-17T01:14:32.474988+0000","last_clean":"2023-10-17T01:14:32.474988+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:14:32.474988+0000","last_undegraded":"2023-10-17T01:14:32.474988+0000","last_fullsized":"2023-10-17T01:14:32.474988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475967+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:14:32.475967+0000","last_peered":"2023-10-17T01:14:32.475967+0000","last_clean":"2023-10-17T01:14:32.475967+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:14:32.475967+0000","last_undegraded":"2023-10-17T01:14:32.475967+0000","last_fullsized":"2023-10-17T01:14:32.475967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479353+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:14:32.479353+0000","last_peered":"2023-10-17T01:14:32.479353+0000","last_clean":"2023-10-17T01:14:32.479353+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:14:32.479353+0000","last_undegraded":"2023-10-17T01:14:32.479353+0000","last_fullsized":"2023-10-17T01:14:32.479353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.462397+0000","last_change":"2023-10-17T01:13:33.514935+0000","last_active":"2023-10-17T01:14:32.462397+0000","last_peered":"2023-10-17T01:14:32.462397+0000","last_clean":"2023-10-17T01:14:32.462397+0000","last_became_active":"2023-10-17T01:13:33.514537+0000","last_became_peered":"2023-10-17T01:13:33.514537+0000","last_unstale":"2023-10-17T01:14:32.462397+0000","last_undegraded":"2023-10-17T01:14:32.462397+0000","last_fullsized":"2023-10-17T01:14:32.462397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:56.247357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490466+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:14:32.490466+0000","last_peered":"2023-10-17T01:14:32.490466+0000","last_clean":"2023-10-17T01:14:32.490466+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:14:32.490466+0000","last_undegraded":"2023-10-17T01:14:32.490466+0000","last_fullsized":"2023-10-17T01:14:32.490466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.466611+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:14:32.466611+0000","last_peered":"2023-10-17T01:14:32.466611+0000","last_clean":"2023-10-17T01:14:32.466611+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:14:32.466611+0000","last_undegraded":"2023-10-17T01:14:32.466611+0000","last_fullsized":"2023-10-17T01:14:32.466611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.457606+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:14:32.457606+0000","last_peered":"2023-10-17T01:14:32.457606+0000","last_clean":"2023-10-17T01:14:32.457606+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:14:32.457606+0000","last_undegraded":"2023-10-17T01:14:32.457606+0000","last_fullsized":"2023-10-17T01:14:32.457606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490514+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:14:32.490514+0000","last_peered":"2023-10-17T01:14:32.490514+0000","last_clean":"2023-10-17T01:14:32.490514+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:14:32.490514+0000","last_undegraded":"2023-10-17T01:14:32.490514+0000","last_fullsized":"2023-10-17T01:14:32.490514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.476373+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:14:32.476373+0000","last_peered":"2023-10-17T01:14:32.476373+0000","last_clean":"2023-10-17T01:14:32.476373+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:14:32.476373+0000","last_undegraded":"2023-10-17T01:14:32.476373+0000","last_fullsized":"2023-10-17T01:14:32.476373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.472094+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:14:32.472094+0000","last_peered":"2023-10-17T01:14:32.472094+0000","last_clean":"2023-10-17T01:14:32.472094+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:14:32.472094+0000","last_undegraded":"2023-10-17T01:14:32.472094+0000","last_fullsized":"2023-10-17T01:14:32.472094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.480952+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:14:32.480952+0000","last_peered":"2023-10-17T01:14:32.480952+0000","last_clean":"2023-10-17T01:14:32.480952+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:14:32.480952+0000","last_undegraded":"2023-10-17T01:14:32.480952+0000","last_fullsized":"2023-10-17T01:14:32.480952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.461642+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:14:32.461642+0000","last_peered":"2023-10-17T01:14:32.461642+0000","last_clean":"2023-10-17T01:14:32.461642+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:14:32.461642+0000","last_undegraded":"2023-10-17T01:14:32.461642+0000","last_fullsized":"2023-10-17T01:14:32.461642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490977+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:14:32.490977+0000","last_peered":"2023-10-17T01:14:32.490977+0000","last_clean":"2023-10-17T01:14:32.490977+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:14:32.490977+0000","last_undegraded":"2023-10-17T01:14:32.490977+0000","last_fullsized":"2023-10-17T01:14:32.490977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474419+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:14:32.474419+0000","last_peered":"2023-10-17T01:14:32.474419+0000","last_clean":"2023-10-17T01:14:32.474419+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:14:32.474419+0000","last_undegraded":"2023-10-17T01:14:32.474419+0000","last_fullsized":"2023-10-17T01:14:32.474419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.477729+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:14:32.477729+0000","last_peered":"2023-10-17T01:14:32.477729+0000","last_clean":"2023-10-17T01:14:32.477729+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:14:32.477729+0000","last_undegraded":"2023-10-17T01:14:32.477729+0000","last_fullsized":"2023-10-17T01:14:32.477729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.478337+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:14:32.478337+0000","last_peered":"2023-10-17T01:14:32.478337+0000","last_clean":"2023-10-17T01:14:32.478337+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:14:32.478337+0000","last_undegraded":"2023-10-17T01:14:32.478337+0000","last_fullsized":"2023-10-17T01:14:32.478337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.478690+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:14:32.478690+0000","last_peered":"2023-10-17T01:14:32.478690+0000","last_clean":"2023-10-17T01:14:32.478690+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:14:32.478690+0000","last_undegraded":"2023-10-17T01:14:32.478690+0000","last_fullsized":"2023-10-17T01:14:32.478690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.477870+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:14:32.477870+0000","last_peered":"2023-10-17T01:14:32.477870+0000","last_clean":"2023-10-17T01:14:32.477870+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:14:32.477870+0000","last_undegraded":"2023-10-17T01:14:32.477870+0000","last_fullsized":"2023-10-17T01:14:32.477870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484919+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:14:32.484919+0000","last_peered":"2023-10-17T01:14:32.484919+0000","last_clean":"2023-10-17T01:14:32.484919+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:14:32.484919+0000","last_undegraded":"2023-10-17T01:14:32.484919+0000","last_fullsized":"2023-10-17T01:14:32.484919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.595468+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:14:43.595468+0000","last_peered":"2023-10-17T01:14:43.595468+0000","last_clean":"2023-10-17T01:14:43.595468+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:14:43.595468+0000","last_undegraded":"2023-10-17T01:14:43.595468+0000","last_fullsized":"2023-10-17T01:14:43.595468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.482277+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:14:32.482277+0000","last_peered":"2023-10-17T01:14:32.482277+0000","last_clean":"2023-10-17T01:14:32.482277+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:14:32.482277+0000","last_undegraded":"2023-10-17T01:14:32.482277+0000","last_fullsized":"2023-10-17T01:14:32.482277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512845+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:32.512845+0000","last_peered":"2023-10-17T01:13:32.512845+0000","last_clean":"2023-10-17T01:13:32.512845+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:32.512845+0000","last_undegraded":"2023-10-17T01:13:32.512845+0000","last_fullsized":"2023-10-17T01:13:32.512845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479641+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:14:32.479641+0000","last_peered":"2023-10-17T01:14:32.479641+0000","last_clean":"2023-10-17T01:14:32.479641+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:14:32.479641+0000","last_undegraded":"2023-10-17T01:14:32.479641+0000","last_fullsized":"2023-10-17T01:14:32.479641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516202+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:32.516202+0000","last_peered":"2023-10-17T01:13:32.516202+0000","last_clean":"2023-10-17T01:13:32.516202+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:32.516202+0000","last_undegraded":"2023-10-17T01:13:32.516202+0000","last_fullsized":"2023-10-17T01:13:32.516202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486971+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:14:32.486971+0000","last_peered":"2023-10-17T01:14:32.486971+0000","last_clean":"2023-10-17T01:14:32.486971+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:14:32.486971+0000","last_undegraded":"2023-10-17T01:14:32.486971+0000","last_fullsized":"2023-10-17T01:14:32.486971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516290+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:32.516290+0000","last_peered":"2023-10-17T01:13:32.516290+0000","last_clean":"2023-10-17T01:13:32.516290+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:32.516290+0000","last_undegraded":"2023-10-17T01:13:32.516290+0000","last_fullsized":"2023-10-17T01:13:32.516290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475047+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:14:32.475047+0000","last_peered":"2023-10-17T01:14:32.475047+0000","last_clean":"2023-10-17T01:14:32.475047+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:14:32.475047+0000","last_undegraded":"2023-10-17T01:14:32.475047+0000","last_fullsized":"2023-10-17T01:14:32.475047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.596973+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:14:43.596973+0000","last_peered":"2023-10-17T01:14:43.596973+0000","last_clean":"2023-10-17T01:14:43.596973+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:14:43.596973+0000","last_undegraded":"2023-10-17T01:14:43.596973+0000","last_fullsized":"2023-10-17T01:14:43.596973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1107,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.460354+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:14:32.460354+0000","last_peered":"2023-10-17T01:14:32.460354+0000","last_clean":"2023-10-17T01:14:32.460354+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:14:32.460354+0000","last_undegraded":"2023-10-17T01:14:32.460354+0000","last_fullsized":"2023-10-17T01:14:32.460354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.029617+0000","last_change":"2023-10-17T01:13:34.029617+0000","last_active":"2023-10-17T01:13:34.029617+0000","last_peered":"2023-10-17T01:13:34.029617+0000","last_clean":"2023-10-17T01:13:34.029617+0000","last_became_active":"2023-10-17T01:13:34.023843+0000","last_became_peered":"2023-10-17T01:13:34.023843+0000","last_unstale":"2023-10-17T01:13:34.029617+0000","last_undegraded":"2023-10-17T01:13:34.029617+0000","last_fullsized":"2023-10-17T01:13:34.029617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:03:37.817216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484628+0000","last_change":"2023-10-17T01:13:34.130813+0000","last_active":"2023-10-17T01:14:32.484628+0000","last_peered":"2023-10-17T01:14:32.484628+0000","last_clean":"2023-10-17T01:14:32.484628+0000","last_became_active":"2023-10-17T01:13:34.130345+0000","last_became_peered":"2023-10-17T01:13:34.130345+0000","last_unstale":"2023-10-17T01:14:32.484628+0000","last_undegraded":"2023-10-17T01:14:32.484628+0000","last_fullsized":"2023-10-17T01:14:32.484628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:56:47.808654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508250+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:32.508250+0000","last_peered":"2023-10-17T01:13:32.508250+0000","last_clean":"2023-10-17T01:13:32.508250+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:32.508250+0000","last_undegraded":"2023-10-17T01:13:32.508250+0000","last_fullsized":"2023-10-17T01:13:32.508250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.491221+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:14:32.491221+0000","last_peered":"2023-10-17T01:14:32.491221+0000","last_clean":"2023-10-17T01:14:32.491221+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:14:32.491221+0000","last_undegraded":"2023-10-17T01:14:32.491221+0000","last_fullsized":"2023-10-17T01:14:32.491221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507782+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:32.507782+0000","last_peered":"2023-10-17T01:13:32.507782+0000","last_clean":"2023-10-17T01:13:32.507782+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:32.507782+0000","last_undegraded":"2023-10-17T01:13:32.507782+0000","last_fullsized":"2023-10-17T01:13:32.507782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484000+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:14:32.484000+0000","last_peered":"2023-10-17T01:14:32.484000+0000","last_clean":"2023-10-17T01:14:32.484000+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:14:32.484000+0000","last_undegraded":"2023-10-17T01:14:32.484000+0000","last_fullsized":"2023-10-17T01:14:32.484000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":208,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:41.593389+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:14:41.593389+0000","last_peered":"2023-10-17T01:14:41.593389+0000","last_clean":"2023-10-17T01:14:41.593389+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:14:41.593389+0000","last_undegraded":"2023-10-17T01:14:41.593389+0000","last_fullsized":"2023-10-17T01:14:41.593389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.476176+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:14:32.476176+0000","last_peered":"2023-10-17T01:14:32.476176+0000","last_clean":"2023-10-17T01:14:32.476176+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:14:32.476176+0000","last_undegraded":"2023-10-17T01:14:32.476176+0000","last_fullsized":"2023-10-17T01:14:32.476176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.471633+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:14:32.471633+0000","last_peered":"2023-10-17T01:14:32.471633+0000","last_clean":"2023-10-17T01:14:32.471633+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:14:32.471633+0000","last_undegraded":"2023-10-17T01:14:32.471633+0000","last_fullsized":"2023-10-17T01:14:32.471633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508333+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:32.508333+0000","last_peered":"2023-10-17T01:13:32.508333+0000","last_clean":"2023-10-17T01:13:32.508333+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:32.508333+0000","last_undegraded":"2023-10-17T01:13:32.508333+0000","last_fullsized":"2023-10-17T01:13:32.508333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479738+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:14:32.479738+0000","last_peered":"2023-10-17T01:14:32.479738+0000","last_clean":"2023-10-17T01:14:32.479738+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:14:32.479738+0000","last_undegraded":"2023-10-17T01:14:32.479738+0000","last_fullsized":"2023-10-17T01:14:32.479738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":243,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:41.593836+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:14:41.593836+0000","last_peered":"2023-10-17T01:14:41.593836+0000","last_clean":"2023-10-17T01:14:41.593836+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:14:41.593836+0000","last_undegraded":"2023-10-17T01:14:41.593836+0000","last_fullsized":"2023-10-17T01:14:41.593836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57393,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":153,"num_write":35,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470653+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:14:32.470653+0000","last_peered":"2023-10-17T01:14:32.470653+0000","last_clean":"2023-10-17T01:14:32.470653+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:14:32.470653+0000","last_undegraded":"2023-10-17T01:14:32.470653+0000","last_fullsized":"2023-10-17T01:14:32.470653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513865+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:32.513865+0000","last_peered":"2023-10-17T01:13:32.513865+0000","last_clean":"2023-10-17T01:13:32.513865+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:32.513865+0000","last_undegraded":"2023-10-17T01:13:32.513865+0000","last_fullsized":"2023-10-17T01:13:32.513865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486907+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:14:32.486907+0000","last_peered":"2023-10-17T01:14:32.486907+0000","last_clean":"2023-10-17T01:14:32.486907+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:14:32.486907+0000","last_undegraded":"2023-10-17T01:14:32.486907+0000","last_fullsized":"2023-10-17T01:14:32.486907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508044+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:32.508044+0000","last_peered":"2023-10-17T01:13:32.508044+0000","last_clean":"2023-10-17T01:13:32.508044+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:32.508044+0000","last_undegraded":"2023-10-17T01:13:32.508044+0000","last_fullsized":"2023-10-17T01:13:32.508044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490807+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:14:32.490807+0000","last_peered":"2023-10-17T01:14:32.490807+0000","last_clean":"2023-10-17T01:14:32.490807+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:14:32.490807+0000","last_undegraded":"2023-10-17T01:14:32.490807+0000","last_fullsized":"2023-10-17T01:14:32.490807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.597131+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:14:43.597131+0000","last_peered":"2023-10-17T01:14:43.597131+0000","last_clean":"2023-10-17T01:14:43.597131+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:14:43.597131+0000","last_undegraded":"2023-10-17T01:14:43.597131+0000","last_fullsized":"2023-10-17T01:14:43.597131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.472490+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:14:32.472490+0000","last_peered":"2023-10-17T01:14:32.472490+0000","last_clean":"2023-10-17T01:14:32.472490+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:14:32.472490+0000","last_undegraded":"2023-10-17T01:14:32.472490+0000","last_fullsized":"2023-10-17T01:14:32.472490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.858283+0000","last_change":"2023-10-17T01:13:33.858283+0000","last_active":"2023-10-17T01:13:33.858283+0000","last_peered":"2023-10-17T01:13:33.858283+0000","last_clean":"2023-10-17T01:13:33.858283+0000","last_became_active":"2023-10-17T01:13:33.844620+0000","last_became_peered":"2023-10-17T01:13:33.844620+0000","last_unstale":"2023-10-17T01:13:33.858283+0000","last_undegraded":"2023-10-17T01:13:33.858283+0000","last_fullsized":"2023-10-17T01:13:33.858283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:48:34.953384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.466233+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:14:32.466233+0000","last_peered":"2023-10-17T01:14:32.466233+0000","last_clean":"2023-10-17T01:14:32.466233+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:14:32.466233+0000","last_undegraded":"2023-10-17T01:14:32.466233+0000","last_fullsized":"2023-10-17T01:14:32.466233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516499+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:32.516499+0000","last_peered":"2023-10-17T01:13:32.516499+0000","last_clean":"2023-10-17T01:13:32.516499+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:32.516499+0000","last_undegraded":"2023-10-17T01:13:32.516499+0000","last_fullsized":"2023-10-17T01:13:32.516499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467302+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:14:32.467302+0000","last_peered":"2023-10-17T01:14:32.467302+0000","last_clean":"2023-10-17T01:14:32.467302+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:14:32.467302+0000","last_undegraded":"2023-10-17T01:14:32.467302+0000","last_fullsized":"2023-10-17T01:14:32.467302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521188+0000","last_change":"2023-10-17T01:13:33.521188+0000","last_active":"2023-10-17T01:13:33.521188+0000","last_peered":"2023-10-17T01:13:33.521188+0000","last_clean":"2023-10-17T01:13:33.521188+0000","last_became_active":"2023-10-17T01:13:33.520645+0000","last_became_peered":"2023-10-17T01:13:33.520645+0000","last_unstale":"2023-10-17T01:13:33.521188+0000","last_undegraded":"2023-10-17T01:13:33.521188+0000","last_fullsized":"2023-10-17T01:13:33.521188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:55:42.028467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485550+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:14:32.485550+0000","last_peered":"2023-10-17T01:14:32.485550+0000","last_clean":"2023-10-17T01:14:32.485550+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:14:32.485550+0000","last_undegraded":"2023-10-17T01:14:32.485550+0000","last_fullsized":"2023-10-17T01:14:32.485550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:42.594670+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:14:42.594670+0000","last_peered":"2023-10-17T01:14:42.594670+0000","last_clean":"2023-10-17T01:14:42.594670+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:14:42.594670+0000","last_undegraded":"2023-10-17T01:14:42.594670+0000","last_fullsized":"2023-10-17T01:14:42.594670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470919+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:14:32.470919+0000","last_peered":"2023-10-17T01:14:32.470919+0000","last_clean":"2023-10-17T01:14:32.470919+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:14:32.470919+0000","last_undegraded":"2023-10-17T01:14:32.470919+0000","last_fullsized":"2023-10-17T01:14:32.470919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467321+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:14:32.467321+0000","last_peered":"2023-10-17T01:14:32.467321+0000","last_clean":"2023-10-17T01:14:32.467321+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:14:32.467321+0000","last_undegraded":"2023-10-17T01:14:32.467321+0000","last_fullsized":"2023-10-17T01:14:32.467321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:44.597987+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:14:44.597987+0000","last_peered":"2023-10-17T01:14:44.597987+0000","last_clean":"2023-10-17T01:14:44.597987+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:14:44.597987+0000","last_undegraded":"2023-10-17T01:14:44.597987+0000","last_fullsized":"2023-10-17T01:14:44.597987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.483880+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:14:32.483880+0000","last_peered":"2023-10-17T01:14:32.483880+0000","last_clean":"2023-10-17T01:14:32.483880+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:14:32.483880+0000","last_undegraded":"2023-10-17T01:14:32.483880+0000","last_fullsized":"2023-10-17T01:14:32.483880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512805+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:32.512805+0000","last_peered":"2023-10-17T01:13:32.512805+0000","last_clean":"2023-10-17T01:13:32.512805+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:32.512805+0000","last_undegraded":"2023-10-17T01:13:32.512805+0000","last_fullsized":"2023-10-17T01:13:32.512805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514380+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:32.514380+0000","last_peered":"2023-10-17T01:13:32.514380+0000","last_clean":"2023-10-17T01:13:32.514380+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:32.514380+0000","last_undegraded":"2023-10-17T01:13:32.514380+0000","last_fullsized":"2023-10-17T01:13:32.514380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.483168+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:14:32.483168+0000","last_peered":"2023-10-17T01:14:32.483168+0000","last_clean":"2023-10-17T01:14:32.483168+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:14:32.483168+0000","last_undegraded":"2023-10-17T01:14:32.483168+0000","last_fullsized":"2023-10-17T01:14:32.483168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521833+0000","last_change":"2023-10-17T01:13:33.521833+0000","last_active":"2023-10-17T01:13:33.521833+0000","last_peered":"2023-10-17T01:13:33.521833+0000","last_clean":"2023-10-17T01:13:33.521833+0000","last_became_active":"2023-10-17T01:13:33.518951+0000","last_became_peered":"2023-10-17T01:13:33.518951+0000","last_unstale":"2023-10-17T01:13:33.521833+0000","last_undegraded":"2023-10-17T01:13:33.521833+0000","last_fullsized":"2023-10-17T01:13:33.521833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:26.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516020+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:32.516020+0000","last_peered":"2023-10-17T01:13:32.516020+0000","last_clean":"2023-10-17T01:13:32.516020+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:32.516020+0000","last_undegraded":"2023-10-17T01:13:32.516020+0000","last_fullsized":"2023-10-17T01:13:32.516020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485317+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:14:32.485317+0000","last_peered":"2023-10-17T01:14:32.485317+0000","last_clean":"2023-10-17T01:14:32.485317+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:14:32.485317+0000","last_undegraded":"2023-10-17T01:14:32.485317+0000","last_fullsized":"2023-10-17T01:14:32.485317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:35.331586+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:14:35.331586+0000","last_peered":"2023-10-17T01:14:35.331586+0000","last_clean":"2023-10-17T01:14:35.331586+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:14:35.331586+0000","last_undegraded":"2023-10-17T01:14:35.331586+0000","last_fullsized":"2023-10-17T01:14:35.331586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486127+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:14:32.486127+0000","last_peered":"2023-10-17T01:14:32.486127+0000","last_clean":"2023-10-17T01:14:32.486127+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:14:32.486127+0000","last_undegraded":"2023-10-17T01:14:32.486127+0000","last_fullsized":"2023-10-17T01:14:32.486127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512549+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:32.512549+0000","last_peered":"2023-10-17T01:13:32.512549+0000","last_clean":"2023-10-17T01:13:32.512549+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:32.512549+0000","last_undegraded":"2023-10-17T01:13:32.512549+0000","last_fullsized":"2023-10-17T01:13:32.512549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484397+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:14:32.484397+0000","last_peered":"2023-10-17T01:14:32.484397+0000","last_clean":"2023-10-17T01:14:32.484397+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:14:32.484397+0000","last_undegraded":"2023-10-17T01:14:32.484397+0000","last_fullsized":"2023-10-17T01:14:32.484397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516068+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:32.516068+0000","last_peered":"2023-10-17T01:13:32.516068+0000","last_clean":"2023-10-17T01:13:32.516068+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:32.516068+0000","last_undegraded":"2023-10-17T01:13:32.516068+0000","last_fullsized":"2023-10-17T01:13:32.516068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.450934+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:14:32.450934+0000","last_peered":"2023-10-17T01:14:32.450934+0000","last_clean":"2023-10-17T01:14:32.450934+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:14:32.450934+0000","last_undegraded":"2023-10-17T01:14:32.450934+0000","last_fullsized":"2023-10-17T01:14:32.450934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513931+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:32.513931+0000","last_peered":"2023-10-17T01:13:32.513931+0000","last_clean":"2023-10-17T01:13:32.513931+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:32.513931+0000","last_undegraded":"2023-10-17T01:13:32.513931+0000","last_fullsized":"2023-10-17T01:13:32.513931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467086+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:14:32.467086+0000","last_peered":"2023-10-17T01:14:32.467086+0000","last_clean":"2023-10-17T01:14:32.467086+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:14:32.467086+0000","last_undegraded":"2023-10-17T01:14:32.467086+0000","last_fullsized":"2023-10-17T01:14:32.467086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.511972+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:33.511972+0000","last_peered":"2023-10-17T01:13:33.511972+0000","last_clean":"2023-10-17T01:13:33.511972+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:33.511972+0000","last_undegraded":"2023-10-17T01:13:33.511972+0000","last_fullsized":"2023-10-17T01:13:33.511972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484207+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:14:32.484207+0000","last_peered":"2023-10-17T01:14:32.484207+0000","last_clean":"2023-10-17T01:14:32.484207+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:14:32.484207+0000","last_undegraded":"2023-10-17T01:14:32.484207+0000","last_fullsized":"2023-10-17T01:14:32.484207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515566+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:32.515566+0000","last_peered":"2023-10-17T01:13:32.515566+0000","last_clean":"2023-10-17T01:13:32.515566+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:32.515566+0000","last_undegraded":"2023-10-17T01:13:32.515566+0000","last_fullsized":"2023-10-17T01:13:32.515566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.487926+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:14:32.487926+0000","last_peered":"2023-10-17T01:14:32.487926+0000","last_clean":"2023-10-17T01:14:32.487926+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:14:32.487926+0000","last_undegraded":"2023-10-17T01:14:32.487926+0000","last_fullsized":"2023-10-17T01:14:32.487926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:42.595441+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:14:42.595441+0000","last_peered":"2023-10-17T01:14:42.595441+0000","last_clean":"2023-10-17T01:14:42.595441+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:14:42.595441+0000","last_undegraded":"2023-10-17T01:14:42.595441+0000","last_fullsized":"2023-10-17T01:14:42.595441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1438,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.481056+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:14:32.481056+0000","last_peered":"2023-10-17T01:14:32.481056+0000","last_clean":"2023-10-17T01:14:32.481056+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:14:32.481056+0000","last_undegraded":"2023-10-17T01:14:32.481056+0000","last_fullsized":"2023-10-17T01:14:32.481056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508206+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:32.508206+0000","last_peered":"2023-10-17T01:13:32.508206+0000","last_clean":"2023-10-17T01:13:32.508206+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:32.508206+0000","last_undegraded":"2023-10-17T01:13:32.508206+0000","last_fullsized":"2023-10-17T01:13:32.508206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.457845+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:14:32.457845+0000","last_peered":"2023-10-17T01:14:32.457845+0000","last_clean":"2023-10-17T01:14:32.457845+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:14:32.457845+0000","last_undegraded":"2023-10-17T01:14:32.457845+0000","last_fullsized":"2023-10-17T01:14:32.457845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508369+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:32.508369+0000","last_peered":"2023-10-17T01:13:32.508369+0000","last_clean":"2023-10-17T01:13:32.508369+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:32.508369+0000","last_undegraded":"2023-10-17T01:13:32.508369+0000","last_fullsized":"2023-10-17T01:13:32.508369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.469788+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:14:32.469788+0000","last_peered":"2023-10-17T01:14:32.469788+0000","last_clean":"2023-10-17T01:14:32.469788+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:14:32.469788+0000","last_undegraded":"2023-10-17T01:14:32.469788+0000","last_fullsized":"2023-10-17T01:14:32.469788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470156+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:14:32.470156+0000","last_peered":"2023-10-17T01:14:32.470156+0000","last_clean":"2023-10-17T01:14:32.470156+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:14:32.470156+0000","last_undegraded":"2023-10-17T01:14:32.470156+0000","last_fullsized":"2023-10-17T01:14:32.470156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.471400+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:14:32.471400+0000","last_peered":"2023-10-17T01:14:32.471400+0000","last_clean":"2023-10-17T01:14:32.471400+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:14:32.471400+0000","last_undegraded":"2023-10-17T01:14:32.471400+0000","last_fullsized":"2023-10-17T01:14:32.471400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.482598+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:14:32.482598+0000","last_peered":"2023-10-17T01:14:32.482598+0000","last_clean":"2023-10-17T01:14:32.482598+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:14:32.482598+0000","last_undegraded":"2023-10-17T01:14:32.482598+0000","last_fullsized":"2023-10-17T01:14:32.482598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490997+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:14:32.490997+0000","last_peered":"2023-10-17T01:14:32.490997+0000","last_clean":"2023-10-17T01:14:32.490997+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:14:32.490997+0000","last_undegraded":"2023-10-17T01:14:32.490997+0000","last_fullsized":"2023-10-17T01:14:32.490997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485901+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:14:32.485901+0000","last_peered":"2023-10-17T01:14:32.485901+0000","last_clean":"2023-10-17T01:14:32.485901+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:14:32.485901+0000","last_undegraded":"2023-10-17T01:14:32.485901+0000","last_fullsized":"2023-10-17T01:14:32.485901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467479+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:14:32.467479+0000","last_peered":"2023-10-17T01:14:32.467479+0000","last_clean":"2023-10-17T01:14:32.467479+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:14:32.467479+0000","last_undegraded":"2023-10-17T01:14:32.467479+0000","last_fullsized":"2023-10-17T01:14:32.467479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474399+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:14:32.474399+0000","last_peered":"2023-10-17T01:14:32.474399+0000","last_clean":"2023-10-17T01:14:32.474399+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:14:32.474399+0000","last_undegraded":"2023-10-17T01:14:32.474399+0000","last_fullsized":"2023-10-17T01:14:32.474399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485736+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:14:32.485736+0000","last_peered":"2023-10-17T01:14:32.485736+0000","last_clean":"2023-10-17T01:14:32.485736+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:14:32.485736+0000","last_undegraded":"2023-10-17T01:14:32.485736+0000","last_fullsized":"2023-10-17T01:14:32.485736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475798+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:14:32.475798+0000","last_peered":"2023-10-17T01:14:32.475798+0000","last_clean":"2023-10-17T01:14:32.475798+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:14:32.475798+0000","last_undegraded":"2023-10-17T01:14:32.475798+0000","last_fullsized":"2023-10-17T01:14:32.475798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485548+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:14:32.485548+0000","last_peered":"2023-10-17T01:14:32.485548+0000","last_clean":"2023-10-17T01:14:32.485548+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:14:32.485548+0000","last_undegraded":"2023-10-17T01:14:32.485548+0000","last_fullsized":"2023-10-17T01:14:32.485548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.488268+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:14:32.488268+0000","last_peered":"2023-10-17T01:14:32.488268+0000","last_clean":"2023-10-17T01:14:32.488268+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:14:32.488268+0000","last_undegraded":"2023-10-17T01:14:32.488268+0000","last_fullsized":"2023-10-17T01:14:32.488268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490569+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:14:32.490569+0000","last_peered":"2023-10-17T01:14:32.490569+0000","last_clean":"2023-10-17T01:14:32.490569+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:14:32.490569+0000","last_undegraded":"2023-10-17T01:14:32.490569+0000","last_fullsized":"2023-10-17T01:14:32.490569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470712+0000","last_change":"2023-10-17T01:13:33.522423+0000","last_active":"2023-10-17T01:14:32.470712+0000","last_peered":"2023-10-17T01:14:32.470712+0000","last_clean":"2023-10-17T01:14:32.470712+0000","last_became_active":"2023-10-17T01:13:33.521927+0000","last_became_peered":"2023-10-17T01:13:33.521927+0000","last_unstale":"2023-10-17T01:14:32.470712+0000","last_undegraded":"2023-10-17T01:14:32.470712+0000","last_fullsized":"2023-10-17T01:14:32.470712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:07:25.868960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64472,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":996,"num_read_kb":568,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":193416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72599999999999998,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.51500000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51000000000000001,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63700000000000001,"5min":1.468,"15min":1.468},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66000000000000003,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.54400000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83999999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76900000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80300000000000005,"5min":1.409,"15min":1.409},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":1.159,"15min":1.159},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.749,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0980000000000001,"5min":1.236,"15min":1.236},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89000000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1259999999999999,"5min":1.359,"15min":1.359},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.68999999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.95699999999999996,"5min":1.21,"15min":1.21},"last":0.872},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.68200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94699999999999995,"5min":1.413,"15min":1.413},"last":0.83599999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":1092223,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.61599999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.91000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67500000000000004,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83899999999999997,"5min":0.87,"15min":0.87},"last":0.59799999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76000000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.621}]},{"osd":5,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.86099999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95299999999999996,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.78500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88700000000000001,"5min":1.117,"15min":1.117},"last":0.874},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.131,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.879},"min":{"1min":0.60099999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.325,"15min":1.325},"last":0.78700000000000003}]},{"osd":11,"last update":"Tue Oct 17 01:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81499999999999995,"15min":0.81599999999999995},"min":{"1min":0.66600000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.96499999999999997,"5min":1.028,"15min":1.028},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84699999999999998,"15min":0.84899999999999998},"min":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.89300000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.51500000000000001,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.32500000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.379}]},{"osd":1,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.66800000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.437},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.751},"min":{"1min":0.48799999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77800000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.216,"15min":1.216},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.71099999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68799999999999994,"15min":0.73399999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76100000000000001,"15min":0.80400000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.877,"15min":0.86599999999999999},"min":{"1min":0.51500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90400000000000003,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.83899999999999997,"15min":0.88100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.198,"15min":1.198},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.84099999999999997,"15min":0.89500000000000002},"min":{"1min":0.59499999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.94299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":6.9379999999999997,"15min":6.9379999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97399999999999998,"5min":1.27,"15min":1.27},"last":0.66200000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.94499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.27,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92300000000000004,"15min":0.96699999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.996,"5min":1.242,"15min":1.242},"last":0.79700000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.93799999999999994},"min":{"1min":0.627,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.077,"5min":1.218,"15min":1.218},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0189999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Tue Oct 17 01:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.749},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86499999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55000000000000004}]},{"osd":1,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.81499999999999995,"5min":0.874,"15min":0.874},"last":0.626},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.57899999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45100000000000001},"min":{"1min":0.39100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.436},{"interface":"front","average":{"1min":0.504,"5min":0.47799999999999998,"15min":0.46700000000000003},"min":{"1min":0.40100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.61599999999999999,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.498,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.50900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94799999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.61899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.67400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0009999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.755,"15min":0.753},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93799999999999994,"5min":1.51,"15min":1.51},"last":0.77500000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.53200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79700000000000004,"15min":0.77600000000000002},"min":{"1min":0.67200000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92400000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.755,"15min":0.748},"min":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.67700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98799999999999999,"5min":0.996,"15min":0.996},"last":0.748}]},{"osd":11,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.89300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":1036658,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64700000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":1.373,"15min":1.373},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85499999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.74099999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83199999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81499999999999995,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84299999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99199999999999999,"5min":1.879,"15min":1.879},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0660000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.64200000000000002}]},{"osd":8,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77300000000000002,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.014,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95399999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65900000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.86499999999999999,"15min":0.89200000000000002},"min":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85899999999999999,"15min":0.89500000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.76200000000000001}]},{"osd":10,"last update":"Tue Oct 17 01:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.83799999999999997,"15min":0.84999999999999998},"min":{"1min":0.50800000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.02,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.56000000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":1.149,"15min":1.149},"last":0.872}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.443},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68899999999999995,"5min":1.339,"15min":1.339},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":1.458,"15min":1.458},"last":0.748},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.752,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.80700000000000005}]},{"osd":3,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67800000000000005,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90200000000000002,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73999999999999999,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0640000000000001,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.61399999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1319999999999999,"5min":1.474,"15min":1.474},"last":0.70799999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95599999999999996,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.57999999999999996}]},{"osd":8,"last update":"Tue Oct 17 01:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.91100000000000003,"15min":0.90900000000000003},"min":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3859999999999999,"5min":1.417,"15min":1.417},"last":0.77300000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.63,"5min":0.625,"15min":0.625},"max":{"1min":1.2909999999999999,"5min":1.657,"15min":1.657},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2270000000000001,"5min":1.532,"15min":1.532},"last":0.82099999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:13:57 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.754,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.73399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.41099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67200000000000004,"5min":0.83499999999999996,"15min":0.85899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.52000000000000002},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.60899999999999999,"15min":0.54500000000000004},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.59799999999999998,"15min":0.54200000000000004},"min":{"1min":0.51100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77900000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.57499999999999996},"min":{"1min":0.46800000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.94699999999999995,"5min":1.02,"15min":1.02},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.65300000000000002,"15min":0.58499999999999996},"min":{"1min":0.501,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.745}]},{"osd":4,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89700000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.49199999999999999,"15min":0.46500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.67400000000000004,"15min":0.61199999999999999},"min":{"1min":0.57999999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98799999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0309999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.66800000000000004},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83099999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.58099999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.262},"last":0.90300000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.018,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.624,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.748,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.119,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.002}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.75700000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.82999999999999996},"min":{"1min":0.64800000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.069,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0669999999999999}]},{"osd":10,"last update":"Tue Oct 17 01:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81000000000000005,"15min":0.80200000000000005},"min":{"1min":0.41999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.96899999999999997}]},{"osd":11,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.89100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.82099999999999995,"5min":1.01,"15min":1.02},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.53400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.89900000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":5.4029999999999996,"5min":5.4029999999999996,"15min":5.4029999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85899999999999999,"5min":1.117,"15min":1.117},"last":0.66700000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Oct 17 01:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.65400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95499999999999996,"5min":1.115,"15min":1.115},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.60599999999999998},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.87,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72299999999999998,"15min":0.67700000000000005},"min":{"1min":0.56100000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94699999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.754},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.73099999999999998,"15min":0.65900000000000003},"min":{"1min":0.54400000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.746}]},{"osd":6,"last update":"Tue Oct 17 01:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.194,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96799999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89500000000000002,"15min":0.90500000000000003},"min":{"1min":0.66400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0169999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.83899999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80400000000000005,"15min":0.83399999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.175,"5min":1.181,"15min":1.181},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.91900000000000004,"15min":0.95899999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2430000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.96299999999999997,"15min":0.98799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95699999999999996,"5min":1.506,"15min":1.506},"last":0.873},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1419999999999999,"5min":1.149,"15min":1.149},"last":0.85699999999999998}]},{"osd":11,"last update":"Tue Oct 17 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.96799999999999997},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.0529999999999999,"5min":1.373,"15min":1.373},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.90800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":502700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.55900000000000005,"15min":0.51400000000000001},"min":{"1min":0.46100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 17 01:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76500000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Oct 17 01:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":2.6619999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72099999999999997,"15min":0.65900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.696,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66900000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61899999999999999,"15min":0.629},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72499999999999998}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92900000000000005,"5min":1.29,"15min":1.29},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":1.016,"15min":1.016},"last":0.80400000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.97299999999999998,"15min":1.01},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.999,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.746}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":443643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"min":{"1min":0.35899999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.371},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.46500000000000002,"15min":0.41299999999999998},"min":{"1min":0.34399999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39400000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68300000000000005,"15min":0.628},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.056,"5min":1.085,"15min":1.085},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68500000000000005,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63600000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.95099999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.65800000000000003},"min":{"1min":0.54200000000000004,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.73499999999999999},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82599999999999996,"15min":0.80900000000000005},"min":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.85899999999999999}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.874,"15min":0.92000000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.234,"15min":1.234},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86199999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.83399999999999996}]},{"osd":10,"last update":"Tue Oct 17 01:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.874},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.873,"15min":0.88200000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.65800000000000003}]},{"osd":11,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.89000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40799999999999997,"15min":0.35999999999999999},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.379},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505}]},{"osd":2,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76700000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76700000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81799999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68899999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.58799999999999997},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77100000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":1.044,"15min":1.044},"last":0.72199999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.022,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89100000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.79200000000000004,"15min":0.77700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.82299999999999995,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":0.82199999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":1.089},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99199999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.97899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1036763,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.754,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.432,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.52500000000000002},"min":{"1min":0.47499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.74099999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":1.24,"15min":1.24},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.432,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.71999999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.39500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.51700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.45500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.88600000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83999999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.61399999999999999},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88200000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.095,"15min":1.095},"last":0.81899999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.625},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.873},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.35999999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.83099999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.88200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.60699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.89500000000000002,"5min":0.997,"15min":0.997},"last":0.84599999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90300000000000002,"5min":1.01,"15min":1.01},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91300000000000003,"5min":1.081,"15min":1.081},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:14:46.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-17T01:14:47.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 -- 172.21.15.45:0/330185114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8074d90 msgr2=0x7f43f80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 --2- 172.21.15.45:0/330185114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8074d90 0x7f43f80731e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009990 tx=0x7f43ec009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 -- 172.21.15.45:0/330185114 shutdown_connections 2023-10-17T01:14:47.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 --2- 172.21.15.45:0/330185114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f8108c10 0x7f43f810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 --2- 172.21.15.45:0/330185114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8074d90 0x7f43f80731e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 --2- 172.21.15.45:0/330185114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f80737b0 0x7f43f8073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 -- 172.21.15.45:0/330185114 >> 172.21.15.45:0/330185114 conn(0x7f43f80fc280 msgr2=0x7f43f80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 -- 172.21.15.45:0/330185114 shutdown_connections 2023-10-17T01:14:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.173+0000 7f43fedff700 1 -- 172.21.15.45:0/330185114 wait complete. 2023-10-17T01:14:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.174+0000 7f43fedff700 1 Processor -- start 2023-10-17T01:14:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 -- start start 2023-10-17T01:14:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f80737b0 0x7f43f8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8074d90 0x7f43f8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 0x7f43f81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81a0760 con 0x7f43f8108c10 2023-10-17T01:14:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81a08a0 con 0x7f43f80737b0 2023-10-17T01:14:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fedff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81a0a30 con 0x7f43f8074d90 2023-10-17T01:14:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fd39c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 0x7f43f81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fd39c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 0x7f43f81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39108/0 (socket says 172.21.15.45:39108) 2023-10-17T01:14:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fd39c700 1 -- 172.21.15.45:0/2488367736 learned_addr learned my addr 172.21.15.45:0/2488367736 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43f7fff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8074d90 0x7f43f8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.175+0000 7f43fcb9b700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f80737b0 0x7f43f8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fd39c700 1 -- 172.21.15.45:0/2488367736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8074d90 msgr2=0x7f43f8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fd39c700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8074d90 0x7f43f8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fd39c700 1 -- 172.21.15.45:0/2488367736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f80737b0 msgr2=0x7f43f8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fd39c700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f80737b0 0x7f43f8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fd39c700 1 -- 172.21.15.45:0/2488367736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0095f0 con 0x7f43f8108c10 2023-10-17T01:14:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43f7fff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8074d90 0x7f43f8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fd39c700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 0x7f43f81a0190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43f000ef30 tx=0x7f43f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0018070 con 0x7f43f8108c10 2023-10-17T01:14:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f000f040 con 0x7f43f8108c10 2023-10-17T01:14:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000cd00 con 0x7f43f8108c10 2023-10-17T01:14:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f81a0cb0 con 0x7f43f8108c10 2023-10-17T01:14:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.176+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f81a1170 con 0x7f43f8108c10 2023-10-17T01:14:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.177+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f81072c0 con 0x7f43f8108c10 2023-10-17T01:14:47.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.179+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f43f0003a00 con 0x7f43f8108c10 2023-10-17T01:14:47.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.180+0000 7f43f5ffb700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43e80a0bf0 0x7f43e80a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.182+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f43f00cbbf0 con 0x7f43f8108c10 2023-10-17T01:14:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.182+0000 7f43fcb9b700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43e80a0bf0 0x7f43e80a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.182+0000 7f43fcb9b700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43e80a0bf0 0x7f43e80a30b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b680 tx=0x7f43ec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.185+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f43f00955e0 con 0x7f43f8108c10 2023-10-17T01:14:47.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.393+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43f8061190 con 0x7f43e80a0bf0 2023-10-17T01:14:47.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.404+0000 7f43f5ffb700 1 -- 172.21.15.45:0/2488367736 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7f43f8061190 con 0x7f43e80a0bf0 2023-10-17T01:14:47.411 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:47.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43e80a0bf0 msgr2=0x7f43e80a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43e80a0bf0 0x7f43e80a30b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b680 tx=0x7f43ec005fd0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 msgr2=0x7f43f81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 0x7f43f81a0190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43f000ef30 tx=0x7f43f000c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 shutdown_connections 2023-10-17T01:14:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f43e80a0bf0 0x7f43e80a30b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43f8108c10 0x7f43f81a0190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f80737b0 0x7f43f8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 --2- 172.21.15.45:0/2488367736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43f8074d90 0x7f43f8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 >> 172.21.15.45:0/2488367736 conn(0x7f43f80fc280 msgr2=0x7f43f80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:47.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 shutdown_connections 2023-10-17T01:14:47.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.410+0000 7f43fedff700 1 -- 172.21.15.45:0/2488367736 wait complete. 2023-10-17T01:14:47.420 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:14:47.430 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-17T01:14:46.458100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655086,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1122,"num_read_kb":677,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334820,"kb_used_data":11396,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607004,"statfs":{"total":1151940427776,"available":1151597572096,"internally_reserved":0,"allocated":11669504,"data_stored":7279338,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1022,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":63,"num_write":14,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003018"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474988+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:14:32.474988+0000","last_peered":"2023-10-17T01:14:32.474988+0000","last_clean":"2023-10-17T01:14:32.474988+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:14:32.474988+0000","last_undegraded":"2023-10-17T01:14:32.474988+0000","last_fullsized":"2023-10-17T01:14:32.474988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475967+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:14:32.475967+0000","last_peered":"2023-10-17T01:14:32.475967+0000","last_clean":"2023-10-17T01:14:32.475967+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:14:32.475967+0000","last_undegraded":"2023-10-17T01:14:32.475967+0000","last_fullsized":"2023-10-17T01:14:32.475967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479353+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:14:32.479353+0000","last_peered":"2023-10-17T01:14:32.479353+0000","last_clean":"2023-10-17T01:14:32.479353+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:14:32.479353+0000","last_undegraded":"2023-10-17T01:14:32.479353+0000","last_fullsized":"2023-10-17T01:14:32.479353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.462397+0000","last_change":"2023-10-17T01:13:33.514935+0000","last_active":"2023-10-17T01:14:32.462397+0000","last_peered":"2023-10-17T01:14:32.462397+0000","last_clean":"2023-10-17T01:14:32.462397+0000","last_became_active":"2023-10-17T01:13:33.514537+0000","last_became_peered":"2023-10-17T01:13:33.514537+0000","last_unstale":"2023-10-17T01:14:32.462397+0000","last_undegraded":"2023-10-17T01:14:32.462397+0000","last_fullsized":"2023-10-17T01:14:32.462397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:56.247357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490466+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:14:32.490466+0000","last_peered":"2023-10-17T01:14:32.490466+0000","last_clean":"2023-10-17T01:14:32.490466+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:14:32.490466+0000","last_undegraded":"2023-10-17T01:14:32.490466+0000","last_fullsized":"2023-10-17T01:14:32.490466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.466611+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:14:32.466611+0000","last_peered":"2023-10-17T01:14:32.466611+0000","last_clean":"2023-10-17T01:14:32.466611+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:14:32.466611+0000","last_undegraded":"2023-10-17T01:14:32.466611+0000","last_fullsized":"2023-10-17T01:14:32.466611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.457606+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:14:32.457606+0000","last_peered":"2023-10-17T01:14:32.457606+0000","last_clean":"2023-10-17T01:14:32.457606+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:14:32.457606+0000","last_undegraded":"2023-10-17T01:14:32.457606+0000","last_fullsized":"2023-10-17T01:14:32.457606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490514+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:14:32.490514+0000","last_peered":"2023-10-17T01:14:32.490514+0000","last_clean":"2023-10-17T01:14:32.490514+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:14:32.490514+0000","last_undegraded":"2023-10-17T01:14:32.490514+0000","last_fullsized":"2023-10-17T01:14:32.490514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.476373+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:14:32.476373+0000","last_peered":"2023-10-17T01:14:32.476373+0000","last_clean":"2023-10-17T01:14:32.476373+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:14:32.476373+0000","last_undegraded":"2023-10-17T01:14:32.476373+0000","last_fullsized":"2023-10-17T01:14:32.476373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.472094+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:14:32.472094+0000","last_peered":"2023-10-17T01:14:32.472094+0000","last_clean":"2023-10-17T01:14:32.472094+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:14:32.472094+0000","last_undegraded":"2023-10-17T01:14:32.472094+0000","last_fullsized":"2023-10-17T01:14:32.472094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.480952+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:14:32.480952+0000","last_peered":"2023-10-17T01:14:32.480952+0000","last_clean":"2023-10-17T01:14:32.480952+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:14:32.480952+0000","last_undegraded":"2023-10-17T01:14:32.480952+0000","last_fullsized":"2023-10-17T01:14:32.480952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.461642+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:14:32.461642+0000","last_peered":"2023-10-17T01:14:32.461642+0000","last_clean":"2023-10-17T01:14:32.461642+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:14:32.461642+0000","last_undegraded":"2023-10-17T01:14:32.461642+0000","last_fullsized":"2023-10-17T01:14:32.461642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490977+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:14:32.490977+0000","last_peered":"2023-10-17T01:14:32.490977+0000","last_clean":"2023-10-17T01:14:32.490977+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:14:32.490977+0000","last_undegraded":"2023-10-17T01:14:32.490977+0000","last_fullsized":"2023-10-17T01:14:32.490977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474419+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:14:32.474419+0000","last_peered":"2023-10-17T01:14:32.474419+0000","last_clean":"2023-10-17T01:14:32.474419+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:14:32.474419+0000","last_undegraded":"2023-10-17T01:14:32.474419+0000","last_fullsized":"2023-10-17T01:14:32.474419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.477729+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:14:32.477729+0000","last_peered":"2023-10-17T01:14:32.477729+0000","last_clean":"2023-10-17T01:14:32.477729+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:14:32.477729+0000","last_undegraded":"2023-10-17T01:14:32.477729+0000","last_fullsized":"2023-10-17T01:14:32.477729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.478337+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:14:32.478337+0000","last_peered":"2023-10-17T01:14:32.478337+0000","last_clean":"2023-10-17T01:14:32.478337+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:14:32.478337+0000","last_undegraded":"2023-10-17T01:14:32.478337+0000","last_fullsized":"2023-10-17T01:14:32.478337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.478690+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:14:32.478690+0000","last_peered":"2023-10-17T01:14:32.478690+0000","last_clean":"2023-10-17T01:14:32.478690+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:14:32.478690+0000","last_undegraded":"2023-10-17T01:14:32.478690+0000","last_fullsized":"2023-10-17T01:14:32.478690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.477870+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:14:32.477870+0000","last_peered":"2023-10-17T01:14:32.477870+0000","last_clean":"2023-10-17T01:14:32.477870+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:14:32.477870+0000","last_undegraded":"2023-10-17T01:14:32.477870+0000","last_fullsized":"2023-10-17T01:14:32.477870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484919+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:14:32.484919+0000","last_peered":"2023-10-17T01:14:32.484919+0000","last_clean":"2023-10-17T01:14:32.484919+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:14:32.484919+0000","last_undegraded":"2023-10-17T01:14:32.484919+0000","last_fullsized":"2023-10-17T01:14:32.484919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.595468+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:14:43.595468+0000","last_peered":"2023-10-17T01:14:43.595468+0000","last_clean":"2023-10-17T01:14:43.595468+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:14:43.595468+0000","last_undegraded":"2023-10-17T01:14:43.595468+0000","last_fullsized":"2023-10-17T01:14:43.595468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.482277+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:14:32.482277+0000","last_peered":"2023-10-17T01:14:32.482277+0000","last_clean":"2023-10-17T01:14:32.482277+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:14:32.482277+0000","last_undegraded":"2023-10-17T01:14:32.482277+0000","last_fullsized":"2023-10-17T01:14:32.482277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512845+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:32.512845+0000","last_peered":"2023-10-17T01:13:32.512845+0000","last_clean":"2023-10-17T01:13:32.512845+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:32.512845+0000","last_undegraded":"2023-10-17T01:13:32.512845+0000","last_fullsized":"2023-10-17T01:13:32.512845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479641+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:14:32.479641+0000","last_peered":"2023-10-17T01:14:32.479641+0000","last_clean":"2023-10-17T01:14:32.479641+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:14:32.479641+0000","last_undegraded":"2023-10-17T01:14:32.479641+0000","last_fullsized":"2023-10-17T01:14:32.479641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516202+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:32.516202+0000","last_peered":"2023-10-17T01:13:32.516202+0000","last_clean":"2023-10-17T01:13:32.516202+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:32.516202+0000","last_undegraded":"2023-10-17T01:13:32.516202+0000","last_fullsized":"2023-10-17T01:13:32.516202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486971+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:14:32.486971+0000","last_peered":"2023-10-17T01:14:32.486971+0000","last_clean":"2023-10-17T01:14:32.486971+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:14:32.486971+0000","last_undegraded":"2023-10-17T01:14:32.486971+0000","last_fullsized":"2023-10-17T01:14:32.486971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516290+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:32.516290+0000","last_peered":"2023-10-17T01:13:32.516290+0000","last_clean":"2023-10-17T01:13:32.516290+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:32.516290+0000","last_undegraded":"2023-10-17T01:13:32.516290+0000","last_fullsized":"2023-10-17T01:13:32.516290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475047+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:14:32.475047+0000","last_peered":"2023-10-17T01:14:32.475047+0000","last_clean":"2023-10-17T01:14:32.475047+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:14:32.475047+0000","last_undegraded":"2023-10-17T01:14:32.475047+0000","last_fullsized":"2023-10-17T01:14:32.475047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.596973+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:14:43.596973+0000","last_peered":"2023-10-17T01:14:43.596973+0000","last_clean":"2023-10-17T01:14:43.596973+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:14:43.596973+0000","last_undegraded":"2023-10-17T01:14:43.596973+0000","last_fullsized":"2023-10-17T01:14:43.596973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1107,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.460354+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:14:32.460354+0000","last_peered":"2023-10-17T01:14:32.460354+0000","last_clean":"2023-10-17T01:14:32.460354+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:14:32.460354+0000","last_undegraded":"2023-10-17T01:14:32.460354+0000","last_fullsized":"2023-10-17T01:14:32.460354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.029617+0000","last_change":"2023-10-17T01:13:34.029617+0000","last_active":"2023-10-17T01:13:34.029617+0000","last_peered":"2023-10-17T01:13:34.029617+0000","last_clean":"2023-10-17T01:13:34.029617+0000","last_became_active":"2023-10-17T01:13:34.023843+0000","last_became_peered":"2023-10-17T01:13:34.023843+0000","last_unstale":"2023-10-17T01:13:34.029617+0000","last_undegraded":"2023-10-17T01:13:34.029617+0000","last_fullsized":"2023-10-17T01:13:34.029617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:03:37.817216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484628+0000","last_change":"2023-10-17T01:13:34.130813+0000","last_active":"2023-10-17T01:14:32.484628+0000","last_peered":"2023-10-17T01:14:32.484628+0000","last_clean":"2023-10-17T01:14:32.484628+0000","last_became_active":"2023-10-17T01:13:34.130345+0000","last_became_peered":"2023-10-17T01:13:34.130345+0000","last_unstale":"2023-10-17T01:14:32.484628+0000","last_undegraded":"2023-10-17T01:14:32.484628+0000","last_fullsized":"2023-10-17T01:14:32.484628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:56:47.808654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508250+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:32.508250+0000","last_peered":"2023-10-17T01:13:32.508250+0000","last_clean":"2023-10-17T01:13:32.508250+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:32.508250+0000","last_undegraded":"2023-10-17T01:13:32.508250+0000","last_fullsized":"2023-10-17T01:13:32.508250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.491221+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:14:32.491221+0000","last_peered":"2023-10-17T01:14:32.491221+0000","last_clean":"2023-10-17T01:14:32.491221+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:14:32.491221+0000","last_undegraded":"2023-10-17T01:14:32.491221+0000","last_fullsized":"2023-10-17T01:14:32.491221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507782+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:32.507782+0000","last_peered":"2023-10-17T01:13:32.507782+0000","last_clean":"2023-10-17T01:13:32.507782+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:32.507782+0000","last_undegraded":"2023-10-17T01:13:32.507782+0000","last_fullsized":"2023-10-17T01:13:32.507782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484000+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:14:32.484000+0000","last_peered":"2023-10-17T01:14:32.484000+0000","last_clean":"2023-10-17T01:14:32.484000+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:14:32.484000+0000","last_undegraded":"2023-10-17T01:14:32.484000+0000","last_fullsized":"2023-10-17T01:14:32.484000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":208,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:41.593389+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:14:41.593389+0000","last_peered":"2023-10-17T01:14:41.593389+0000","last_clean":"2023-10-17T01:14:41.593389+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:14:41.593389+0000","last_undegraded":"2023-10-17T01:14:41.593389+0000","last_fullsized":"2023-10-17T01:14:41.593389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.476176+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:14:32.476176+0000","last_peered":"2023-10-17T01:14:32.476176+0000","last_clean":"2023-10-17T01:14:32.476176+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:14:32.476176+0000","last_undegraded":"2023-10-17T01:14:32.476176+0000","last_fullsized":"2023-10-17T01:14:32.476176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.471633+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:14:32.471633+0000","last_peered":"2023-10-17T01:14:32.471633+0000","last_clean":"2023-10-17T01:14:32.471633+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:14:32.471633+0000","last_undegraded":"2023-10-17T01:14:32.471633+0000","last_fullsized":"2023-10-17T01:14:32.471633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508333+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:32.508333+0000","last_peered":"2023-10-17T01:13:32.508333+0000","last_clean":"2023-10-17T01:13:32.508333+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:32.508333+0000","last_undegraded":"2023-10-17T01:13:32.508333+0000","last_fullsized":"2023-10-17T01:13:32.508333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479738+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:14:32.479738+0000","last_peered":"2023-10-17T01:14:32.479738+0000","last_clean":"2023-10-17T01:14:32.479738+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:14:32.479738+0000","last_undegraded":"2023-10-17T01:14:32.479738+0000","last_fullsized":"2023-10-17T01:14:32.479738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":243,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:41.593836+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:14:41.593836+0000","last_peered":"2023-10-17T01:14:41.593836+0000","last_clean":"2023-10-17T01:14:41.593836+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:14:41.593836+0000","last_undegraded":"2023-10-17T01:14:41.593836+0000","last_fullsized":"2023-10-17T01:14:41.593836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57393,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":153,"num_write":35,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470653+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:14:32.470653+0000","last_peered":"2023-10-17T01:14:32.470653+0000","last_clean":"2023-10-17T01:14:32.470653+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:14:32.470653+0000","last_undegraded":"2023-10-17T01:14:32.470653+0000","last_fullsized":"2023-10-17T01:14:32.470653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513865+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:32.513865+0000","last_peered":"2023-10-17T01:13:32.513865+0000","last_clean":"2023-10-17T01:13:32.513865+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:32.513865+0000","last_undegraded":"2023-10-17T01:13:32.513865+0000","last_fullsized":"2023-10-17T01:13:32.513865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486907+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:14:32.486907+0000","last_peered":"2023-10-17T01:14:32.486907+0000","last_clean":"2023-10-17T01:14:32.486907+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:14:32.486907+0000","last_undegraded":"2023-10-17T01:14:32.486907+0000","last_fullsized":"2023-10-17T01:14:32.486907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508044+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:32.508044+0000","last_peered":"2023-10-17T01:13:32.508044+0000","last_clean":"2023-10-17T01:13:32.508044+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:32.508044+0000","last_undegraded":"2023-10-17T01:13:32.508044+0000","last_fullsized":"2023-10-17T01:13:32.508044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490807+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:14:32.490807+0000","last_peered":"2023-10-17T01:14:32.490807+0000","last_clean":"2023-10-17T01:14:32.490807+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:14:32.490807+0000","last_undegraded":"2023-10-17T01:14:32.490807+0000","last_fullsized":"2023-10-17T01:14:32.490807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.597131+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:14:43.597131+0000","last_peered":"2023-10-17T01:14:43.597131+0000","last_clean":"2023-10-17T01:14:43.597131+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:14:43.597131+0000","last_undegraded":"2023-10-17T01:14:43.597131+0000","last_fullsized":"2023-10-17T01:14:43.597131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.472490+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:14:32.472490+0000","last_peered":"2023-10-17T01:14:32.472490+0000","last_clean":"2023-10-17T01:14:32.472490+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:14:32.472490+0000","last_undegraded":"2023-10-17T01:14:32.472490+0000","last_fullsized":"2023-10-17T01:14:32.472490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.858283+0000","last_change":"2023-10-17T01:13:33.858283+0000","last_active":"2023-10-17T01:13:33.858283+0000","last_peered":"2023-10-17T01:13:33.858283+0000","last_clean":"2023-10-17T01:13:33.858283+0000","last_became_active":"2023-10-17T01:13:33.844620+0000","last_became_peered":"2023-10-17T01:13:33.844620+0000","last_unstale":"2023-10-17T01:13:33.858283+0000","last_undegraded":"2023-10-17T01:13:33.858283+0000","last_fullsized":"2023-10-17T01:13:33.858283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:48:34.953384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.466233+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:14:32.466233+0000","last_peered":"2023-10-17T01:14:32.466233+0000","last_clean":"2023-10-17T01:14:32.466233+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:14:32.466233+0000","last_undegraded":"2023-10-17T01:14:32.466233+0000","last_fullsized":"2023-10-17T01:14:32.466233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516499+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:32.516499+0000","last_peered":"2023-10-17T01:13:32.516499+0000","last_clean":"2023-10-17T01:13:32.516499+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:32.516499+0000","last_undegraded":"2023-10-17T01:13:32.516499+0000","last_fullsized":"2023-10-17T01:13:32.516499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467302+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:14:32.467302+0000","last_peered":"2023-10-17T01:14:32.467302+0000","last_clean":"2023-10-17T01:14:32.467302+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:14:32.467302+0000","last_undegraded":"2023-10-17T01:14:32.467302+0000","last_fullsized":"2023-10-17T01:14:32.467302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521188+0000","last_change":"2023-10-17T01:13:33.521188+0000","last_active":"2023-10-17T01:13:33.521188+0000","last_peered":"2023-10-17T01:13:33.521188+0000","last_clean":"2023-10-17T01:13:33.521188+0000","last_became_active":"2023-10-17T01:13:33.520645+0000","last_became_peered":"2023-10-17T01:13:33.520645+0000","last_unstale":"2023-10-17T01:13:33.521188+0000","last_undegraded":"2023-10-17T01:13:33.521188+0000","last_fullsized":"2023-10-17T01:13:33.521188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:55:42.028467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485550+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:14:32.485550+0000","last_peered":"2023-10-17T01:14:32.485550+0000","last_clean":"2023-10-17T01:14:32.485550+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:14:32.485550+0000","last_undegraded":"2023-10-17T01:14:32.485550+0000","last_fullsized":"2023-10-17T01:14:32.485550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:42.594670+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:14:42.594670+0000","last_peered":"2023-10-17T01:14:42.594670+0000","last_clean":"2023-10-17T01:14:42.594670+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:14:42.594670+0000","last_undegraded":"2023-10-17T01:14:42.594670+0000","last_fullsized":"2023-10-17T01:14:42.594670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470919+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:14:32.470919+0000","last_peered":"2023-10-17T01:14:32.470919+0000","last_clean":"2023-10-17T01:14:32.470919+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:14:32.470919+0000","last_undegraded":"2023-10-17T01:14:32.470919+0000","last_fullsized":"2023-10-17T01:14:32.470919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467321+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:14:32.467321+0000","last_peered":"2023-10-17T01:14:32.467321+0000","last_clean":"2023-10-17T01:14:32.467321+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:14:32.467321+0000","last_undegraded":"2023-10-17T01:14:32.467321+0000","last_fullsized":"2023-10-17T01:14:32.467321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:44.597987+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:14:44.597987+0000","last_peered":"2023-10-17T01:14:44.597987+0000","last_clean":"2023-10-17T01:14:44.597987+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:14:44.597987+0000","last_undegraded":"2023-10-17T01:14:44.597987+0000","last_fullsized":"2023-10-17T01:14:44.597987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.483880+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:14:32.483880+0000","last_peered":"2023-10-17T01:14:32.483880+0000","last_clean":"2023-10-17T01:14:32.483880+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:14:32.483880+0000","last_undegraded":"2023-10-17T01:14:32.483880+0000","last_fullsized":"2023-10-17T01:14:32.483880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512805+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:32.512805+0000","last_peered":"2023-10-17T01:13:32.512805+0000","last_clean":"2023-10-17T01:13:32.512805+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:32.512805+0000","last_undegraded":"2023-10-17T01:13:32.512805+0000","last_fullsized":"2023-10-17T01:13:32.512805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514380+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:32.514380+0000","last_peered":"2023-10-17T01:13:32.514380+0000","last_clean":"2023-10-17T01:13:32.514380+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:32.514380+0000","last_undegraded":"2023-10-17T01:13:32.514380+0000","last_fullsized":"2023-10-17T01:13:32.514380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.483168+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:14:32.483168+0000","last_peered":"2023-10-17T01:14:32.483168+0000","last_clean":"2023-10-17T01:14:32.483168+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:14:32.483168+0000","last_undegraded":"2023-10-17T01:14:32.483168+0000","last_fullsized":"2023-10-17T01:14:32.483168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521833+0000","last_change":"2023-10-17T01:13:33.521833+0000","last_active":"2023-10-17T01:13:33.521833+0000","last_peered":"2023-10-17T01:13:33.521833+0000","last_clean":"2023-10-17T01:13:33.521833+0000","last_became_active":"2023-10-17T01:13:33.518951+0000","last_became_peered":"2023-10-17T01:13:33.518951+0000","last_unstale":"2023-10-17T01:13:33.521833+0000","last_undegraded":"2023-10-17T01:13:33.521833+0000","last_fullsized":"2023-10-17T01:13:33.521833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:26.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516020+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:32.516020+0000","last_peered":"2023-10-17T01:13:32.516020+0000","last_clean":"2023-10-17T01:13:32.516020+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:32.516020+0000","last_undegraded":"2023-10-17T01:13:32.516020+0000","last_fullsized":"2023-10-17T01:13:32.516020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485317+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:14:32.485317+0000","last_peered":"2023-10-17T01:14:32.485317+0000","last_clean":"2023-10-17T01:14:32.485317+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:14:32.485317+0000","last_undegraded":"2023-10-17T01:14:32.485317+0000","last_fullsized":"2023-10-17T01:14:32.485317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:35.331586+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:14:35.331586+0000","last_peered":"2023-10-17T01:14:35.331586+0000","last_clean":"2023-10-17T01:14:35.331586+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:14:35.331586+0000","last_undegraded":"2023-10-17T01:14:35.331586+0000","last_fullsized":"2023-10-17T01:14:35.331586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486127+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:14:32.486127+0000","last_peered":"2023-10-17T01:14:32.486127+0000","last_clean":"2023-10-17T01:14:32.486127+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:14:32.486127+0000","last_undegraded":"2023-10-17T01:14:32.486127+0000","last_fullsized":"2023-10-17T01:14:32.486127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512549+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:32.512549+0000","last_peered":"2023-10-17T01:13:32.512549+0000","last_clean":"2023-10-17T01:13:32.512549+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:32.512549+0000","last_undegraded":"2023-10-17T01:13:32.512549+0000","last_fullsized":"2023-10-17T01:13:32.512549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484397+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:14:32.484397+0000","last_peered":"2023-10-17T01:14:32.484397+0000","last_clean":"2023-10-17T01:14:32.484397+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:14:32.484397+0000","last_undegraded":"2023-10-17T01:14:32.484397+0000","last_fullsized":"2023-10-17T01:14:32.484397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516068+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:32.516068+0000","last_peered":"2023-10-17T01:13:32.516068+0000","last_clean":"2023-10-17T01:13:32.516068+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:32.516068+0000","last_undegraded":"2023-10-17T01:13:32.516068+0000","last_fullsized":"2023-10-17T01:13:32.516068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.450934+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:14:32.450934+0000","last_peered":"2023-10-17T01:14:32.450934+0000","last_clean":"2023-10-17T01:14:32.450934+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:14:32.450934+0000","last_undegraded":"2023-10-17T01:14:32.450934+0000","last_fullsized":"2023-10-17T01:14:32.450934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513931+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:32.513931+0000","last_peered":"2023-10-17T01:13:32.513931+0000","last_clean":"2023-10-17T01:13:32.513931+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:32.513931+0000","last_undegraded":"2023-10-17T01:13:32.513931+0000","last_fullsized":"2023-10-17T01:13:32.513931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467086+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:14:32.467086+0000","last_peered":"2023-10-17T01:14:32.467086+0000","last_clean":"2023-10-17T01:14:32.467086+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:14:32.467086+0000","last_undegraded":"2023-10-17T01:14:32.467086+0000","last_fullsized":"2023-10-17T01:14:32.467086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.511972+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:33.511972+0000","last_peered":"2023-10-17T01:13:33.511972+0000","last_clean":"2023-10-17T01:13:33.511972+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:33.511972+0000","last_undegraded":"2023-10-17T01:13:33.511972+0000","last_fullsized":"2023-10-17T01:13:33.511972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484207+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:14:32.484207+0000","last_peered":"2023-10-17T01:14:32.484207+0000","last_clean":"2023-10-17T01:14:32.484207+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:14:32.484207+0000","last_undegraded":"2023-10-17T01:14:32.484207+0000","last_fullsized":"2023-10-17T01:14:32.484207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515566+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:32.515566+0000","last_peered":"2023-10-17T01:13:32.515566+0000","last_clean":"2023-10-17T01:13:32.515566+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:32.515566+0000","last_undegraded":"2023-10-17T01:13:32.515566+0000","last_fullsized":"2023-10-17T01:13:32.515566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.487926+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:14:32.487926+0000","last_peered":"2023-10-17T01:14:32.487926+0000","last_clean":"2023-10-17T01:14:32.487926+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:14:32.487926+0000","last_undegraded":"2023-10-17T01:14:32.487926+0000","last_fullsized":"2023-10-17T01:14:32.487926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:42.595441+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:14:42.595441+0000","last_peered":"2023-10-17T01:14:42.595441+0000","last_clean":"2023-10-17T01:14:42.595441+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:14:42.595441+0000","last_undegraded":"2023-10-17T01:14:42.595441+0000","last_fullsized":"2023-10-17T01:14:42.595441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1438,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.481056+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:14:32.481056+0000","last_peered":"2023-10-17T01:14:32.481056+0000","last_clean":"2023-10-17T01:14:32.481056+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:14:32.481056+0000","last_undegraded":"2023-10-17T01:14:32.481056+0000","last_fullsized":"2023-10-17T01:14:32.481056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508206+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:32.508206+0000","last_peered":"2023-10-17T01:13:32.508206+0000","last_clean":"2023-10-17T01:13:32.508206+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:32.508206+0000","last_undegraded":"2023-10-17T01:13:32.508206+0000","last_fullsized":"2023-10-17T01:13:32.508206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.457845+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:14:32.457845+0000","last_peered":"2023-10-17T01:14:32.457845+0000","last_clean":"2023-10-17T01:14:32.457845+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:14:32.457845+0000","last_undegraded":"2023-10-17T01:14:32.457845+0000","last_fullsized":"2023-10-17T01:14:32.457845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508369+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:32.508369+0000","last_peered":"2023-10-17T01:13:32.508369+0000","last_clean":"2023-10-17T01:13:32.508369+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:32.508369+0000","last_undegraded":"2023-10-17T01:13:32.508369+0000","last_fullsized":"2023-10-17T01:13:32.508369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.469788+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:14:32.469788+0000","last_peered":"2023-10-17T01:14:32.469788+0000","last_clean":"2023-10-17T01:14:32.469788+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:14:32.469788+0000","last_undegraded":"2023-10-17T01:14:32.469788+0000","last_fullsized":"2023-10-17T01:14:32.469788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470156+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:14:32.470156+0000","last_peered":"2023-10-17T01:14:32.470156+0000","last_clean":"2023-10-17T01:14:32.470156+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:14:32.470156+0000","last_undegraded":"2023-10-17T01:14:32.470156+0000","last_fullsized":"2023-10-17T01:14:32.470156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.471400+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:14:32.471400+0000","last_peered":"2023-10-17T01:14:32.471400+0000","last_clean":"2023-10-17T01:14:32.471400+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:14:32.471400+0000","last_undegraded":"2023-10-17T01:14:32.471400+0000","last_fullsized":"2023-10-17T01:14:32.471400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.482598+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:14:32.482598+0000","last_peered":"2023-10-17T01:14:32.482598+0000","last_clean":"2023-10-17T01:14:32.482598+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:14:32.482598+0000","last_undegraded":"2023-10-17T01:14:32.482598+0000","last_fullsized":"2023-10-17T01:14:32.482598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490997+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:14:32.490997+0000","last_peered":"2023-10-17T01:14:32.490997+0000","last_clean":"2023-10-17T01:14:32.490997+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:14:32.490997+0000","last_undegraded":"2023-10-17T01:14:32.490997+0000","last_fullsized":"2023-10-17T01:14:32.490997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485901+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:14:32.485901+0000","last_peered":"2023-10-17T01:14:32.485901+0000","last_clean":"2023-10-17T01:14:32.485901+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:14:32.485901+0000","last_undegraded":"2023-10-17T01:14:32.485901+0000","last_fullsized":"2023-10-17T01:14:32.485901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467479+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:14:32.467479+0000","last_peered":"2023-10-17T01:14:32.467479+0000","last_clean":"2023-10-17T01:14:32.467479+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:14:32.467479+0000","last_undegraded":"2023-10-17T01:14:32.467479+0000","last_fullsized":"2023-10-17T01:14:32.467479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474399+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:14:32.474399+0000","last_peered":"2023-10-17T01:14:32.474399+0000","last_clean":"2023-10-17T01:14:32.474399+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:14:32.474399+0000","last_undegraded":"2023-10-17T01:14:32.474399+0000","last_fullsized":"2023-10-17T01:14:32.474399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485736+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:14:32.485736+0000","last_peered":"2023-10-17T01:14:32.485736+0000","last_clean":"2023-10-17T01:14:32.485736+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:14:32.485736+0000","last_undegraded":"2023-10-17T01:14:32.485736+0000","last_fullsized":"2023-10-17T01:14:32.485736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475798+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:14:32.475798+0000","last_peered":"2023-10-17T01:14:32.475798+0000","last_clean":"2023-10-17T01:14:32.475798+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:14:32.475798+0000","last_undegraded":"2023-10-17T01:14:32.475798+0000","last_fullsized":"2023-10-17T01:14:32.475798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485548+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:14:32.485548+0000","last_peered":"2023-10-17T01:14:32.485548+0000","last_clean":"2023-10-17T01:14:32.485548+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:14:32.485548+0000","last_undegraded":"2023-10-17T01:14:32.485548+0000","last_fullsized":"2023-10-17T01:14:32.485548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.488268+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:14:32.488268+0000","last_peered":"2023-10-17T01:14:32.488268+0000","last_clean":"2023-10-17T01:14:32.488268+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:14:32.488268+0000","last_undegraded":"2023-10-17T01:14:32.488268+0000","last_fullsized":"2023-10-17T01:14:32.488268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490569+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:14:32.490569+0000","last_peered":"2023-10-17T01:14:32.490569+0000","last_clean":"2023-10-17T01:14:32.490569+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:14:32.490569+0000","last_undegraded":"2023-10-17T01:14:32.490569+0000","last_fullsized":"2023-10-17T01:14:32.490569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470712+0000","last_change":"2023-10-17T01:13:33.522423+0000","last_active":"2023-10-17T01:14:32.470712+0000","last_peered":"2023-10-17T01:14:32.470712+0000","last_clean":"2023-10-17T01:14:32.470712+0000","last_became_active":"2023-10-17T01:13:33.521927+0000","last_became_peered":"2023-10-17T01:13:33.521927+0000","last_unstale":"2023-10-17T01:14:32.470712+0000","last_undegraded":"2023-10-17T01:14:32.470712+0000","last_fullsized":"2023-10-17T01:14:32.470712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:07:25.868960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64472,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":996,"num_read_kb":568,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":193416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72599999999999998,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.51500000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51000000000000001,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63700000000000001,"5min":1.468,"15min":1.468},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66000000000000003,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.54400000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83999999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76900000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80300000000000005,"5min":1.409,"15min":1.409},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":1.159,"15min":1.159},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.749,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0980000000000001,"5min":1.236,"15min":1.236},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89000000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1259999999999999,"5min":1.359,"15min":1.359},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.68999999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.95699999999999996,"5min":1.21,"15min":1.21},"last":0.872},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.68200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94699999999999995,"5min":1.413,"15min":1.413},"last":0.83599999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":1092223,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.61599999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.91000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67500000000000004,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83899999999999997,"5min":0.87,"15min":0.87},"last":0.59799999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76000000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.621}]},{"osd":5,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.86099999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95299999999999996,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.78500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88700000000000001,"5min":1.117,"15min":1.117},"last":0.874},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.131,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.879},"min":{"1min":0.60099999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.325,"15min":1.325},"last":0.78700000000000003}]},{"osd":11,"last update":"Tue Oct 17 01:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81499999999999995,"15min":0.81599999999999995},"min":{"1min":0.66600000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.96499999999999997,"5min":1.028,"15min":1.028},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84699999999999998,"15min":0.84899999999999998},"min":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.89300000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.51500000000000001,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.32500000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.379}]},{"osd":1,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.66800000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.437},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.751},"min":{"1min":0.48799999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77800000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.216,"15min":1.216},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.71099999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68799999999999994,"15min":0.73399999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76100000000000001,"15min":0.80400000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.877,"15min":0.86599999999999999},"min":{"1min":0.51500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90400000000000003,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.83899999999999997,"15min":0.88100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.198,"15min":1.198},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.84099999999999997,"15min":0.89500000000000002},"min":{"1min":0.59499999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.94299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":6.9379999999999997,"15min":6.9379999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97399999999999998,"5min":1.27,"15min":1.27},"last":0.66200000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.94499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.27,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92300000000000004,"15min":0.96699999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.996,"5min":1.242,"15min":1.242},"last":0.79700000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.93799999999999994},"min":{"1min":0.627,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.077,"5min":1.218,"15min":1.218},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0189999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Tue Oct 17 01:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.749},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86499999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55000000000000004}]},{"osd":1,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.81499999999999995,"5min":0.874,"15min":0.874},"last":0.626},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.57899999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45100000000000001},"min":{"1min":0.39100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.436},{"interface":"front","average":{"1min":0.504,"5min":0.47799999999999998,"15min":0.46700000000000003},"min":{"1min":0.40100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.61599999999999999,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.498,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.50900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94799999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.61899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.67400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0009999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.755,"15min":0.753},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93799999999999994,"5min":1.51,"15min":1.51},"last":0.77500000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.53200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79700000000000004,"15min":0.77600000000000002},"min":{"1min":0.67200000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92400000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.755,"15min":0.748},"min":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.67700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98799999999999999,"5min":0.996,"15min":0.996},"last":0.748}]},{"osd":11,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.89300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":1036658,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64700000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":1.373,"15min":1.373},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85499999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.74099999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83199999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81499999999999995,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84299999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99199999999999999,"5min":1.879,"15min":1.879},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0660000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.64200000000000002}]},{"osd":8,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77300000000000002,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.014,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95399999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65900000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.86499999999999999,"15min":0.89200000000000002},"min":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85899999999999999,"15min":0.89500000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.76200000000000001}]},{"osd":10,"last update":"Tue Oct 17 01:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.83799999999999997,"15min":0.84999999999999998},"min":{"1min":0.50800000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.02,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.56000000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":1.149,"15min":1.149},"last":0.872}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.443},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68899999999999995,"5min":1.339,"15min":1.339},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":1.458,"15min":1.458},"last":0.748},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.752,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.80700000000000005}]},{"osd":3,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67800000000000005,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90200000000000002,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73999999999999999,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0640000000000001,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.61399999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1319999999999999,"5min":1.474,"15min":1.474},"last":0.70799999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95599999999999996,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.57999999999999996}]},{"osd":8,"last update":"Tue Oct 17 01:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.91100000000000003,"15min":0.90900000000000003},"min":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3859999999999999,"5min":1.417,"15min":1.417},"last":0.77300000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.63,"5min":0.625,"15min":0.625},"max":{"1min":1.2909999999999999,"5min":1.657,"15min":1.657},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2270000000000001,"5min":1.532,"15min":1.532},"last":0.82099999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:13:57 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.754,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.73399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.41099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67200000000000004,"5min":0.83499999999999996,"15min":0.85899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.52000000000000002},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.60899999999999999,"15min":0.54500000000000004},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.59799999999999998,"15min":0.54200000000000004},"min":{"1min":0.51100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77900000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.57499999999999996},"min":{"1min":0.46800000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.94699999999999995,"5min":1.02,"15min":1.02},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.65300000000000002,"15min":0.58499999999999996},"min":{"1min":0.501,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.745}]},{"osd":4,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89700000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.49199999999999999,"15min":0.46500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.67400000000000004,"15min":0.61199999999999999},"min":{"1min":0.57999999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98799999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0309999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.66800000000000004},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83099999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.58099999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.262},"last":0.90300000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.018,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.624,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.748,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.119,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.002}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.75700000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.82999999999999996},"min":{"1min":0.64800000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.069,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0669999999999999}]},{"osd":10,"last update":"Tue Oct 17 01:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81000000000000005,"15min":0.80200000000000005},"min":{"1min":0.41999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.96899999999999997}]},{"osd":11,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.89100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.82099999999999995,"5min":1.01,"15min":1.02},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.53400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.89900000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":5.4029999999999996,"5min":5.4029999999999996,"15min":5.4029999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85899999999999999,"5min":1.117,"15min":1.117},"last":0.66700000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Oct 17 01:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.65400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95499999999999996,"5min":1.115,"15min":1.115},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.60599999999999998},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.87,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72299999999999998,"15min":0.67700000000000005},"min":{"1min":0.56100000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94699999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.754},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.73099999999999998,"15min":0.65900000000000003},"min":{"1min":0.54400000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.746}]},{"osd":6,"last update":"Tue Oct 17 01:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.194,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96799999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89500000000000002,"15min":0.90500000000000003},"min":{"1min":0.66400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0169999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.83899999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80400000000000005,"15min":0.83399999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.175,"5min":1.181,"15min":1.181},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.91900000000000004,"15min":0.95899999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2430000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.96299999999999997,"15min":0.98799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95699999999999996,"5min":1.506,"15min":1.506},"last":0.873},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1419999999999999,"5min":1.149,"15min":1.149},"last":0.85699999999999998}]},{"osd":11,"last update":"Tue Oct 17 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.96799999999999997},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.0529999999999999,"5min":1.373,"15min":1.373},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.90800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":502700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.55900000000000005,"15min":0.51400000000000001},"min":{"1min":0.46100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 17 01:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76500000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Oct 17 01:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":2.6619999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72099999999999997,"15min":0.65900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.696,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66900000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61899999999999999,"15min":0.629},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72499999999999998}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92900000000000005,"5min":1.29,"15min":1.29},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":1.016,"15min":1.016},"last":0.80400000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.97299999999999998,"15min":1.01},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.999,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.746}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":443643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"min":{"1min":0.35899999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.371},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.46500000000000002,"15min":0.41299999999999998},"min":{"1min":0.34399999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39400000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68300000000000005,"15min":0.628},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.056,"5min":1.085,"15min":1.085},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68500000000000005,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63600000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.95099999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.65800000000000003},"min":{"1min":0.54200000000000004,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.73499999999999999},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82599999999999996,"15min":0.80900000000000005},"min":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.85899999999999999}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.874,"15min":0.92000000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.234,"15min":1.234},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86199999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.83399999999999996}]},{"osd":10,"last update":"Tue Oct 17 01:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.874},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.873,"15min":0.88200000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.65800000000000003}]},{"osd":11,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.89000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40799999999999997,"15min":0.35999999999999999},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.379},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505}]},{"osd":2,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76700000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76700000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81799999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68899999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.58799999999999997},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77100000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":1.044,"15min":1.044},"last":0.72199999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.022,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89100000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.79200000000000004,"15min":0.77700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.82299999999999995,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":0.82199999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":1.089},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99199999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.97899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1036763,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.754,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.432,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.52500000000000002},"min":{"1min":0.47499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.74099999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":1.24,"15min":1.24},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.432,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.71999999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.39500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.51700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.45500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.88600000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83999999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.61399999999999999},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88200000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.095,"15min":1.095},"last":0.81899999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.625},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.873},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.35999999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.83099999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.88200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.60699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.89500000000000002,"5min":0.997,"15min":0.997},"last":0.84599999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90300000000000002,"5min":1.01,"15min":1.01},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91300000000000003,"5min":1.081,"15min":1.081},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:14:47.437 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-17T01:14:47.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-17T01:14:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1628560460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a4074d90 msgr2=0x7f17a40731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1628560460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a4074d90 0x7f17a40731e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009990 tx=0x7f17a0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1628560460 shutdown_connections 2023-10-17T01:14:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1628560460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a4108c10 0x7f17a410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1628560460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a4074d90 0x7f17a40731e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1628560460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a40737b0 0x7f17a4073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1628560460 >> 172.21.15.45:0/1628560460 conn(0x7f17a40fc280 msgr2=0x7f17a40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1628560460 shutdown_connections 2023-10-17T01:14:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.675+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1628560460 wait complete. 2023-10-17T01:14:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 Processor -- start 2023-10-17T01:14:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 -- start start 2023-10-17T01:14:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 0x7f17a4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a4074d90 0x7f17a4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a4108c10 0x7f17a41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a41a0790 con 0x7f17a40737b0 2023-10-17T01:14:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a41a0900 con 0x7f17a4074d90 2023-10-17T01:14:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.677+0000 7f17ac1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a41a0ac0 con 0x7f17a4108c10 2023-10-17T01:14:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 0x7f17a4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a979a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a4074d90 0x7f17a4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 0x7f17a4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39126/0 (socket says 172.21.15.45:39126) 2023-10-17T01:14:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 -- 172.21.15.45:0/1199927089 learned_addr learned my addr 172.21.15.45:0/1199927089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17aa79c700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a4108c10 0x7f17a41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 -- 172.21.15.45:0/1199927089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a4108c10 msgr2=0x7f17a41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a4108c10 0x7f17a41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 -- 172.21.15.45:0/1199927089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a4074d90 msgr2=0x7f17a4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a4074d90 0x7f17a4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 -- 172.21.15.45:0/1199927089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00095f0 con 0x7f17a40737b0 2023-10-17T01:14:47.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.678+0000 7f17a9f9b700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 0x7f17a4198c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f17a0004740 tx=0x7f17a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:47.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.679+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a001d070 con 0x7f17a40737b0 2023-10-17T01:14:47.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.679+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a0004830 con 0x7f17a40737b0 2023-10-17T01:14:47.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.679+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a41a0d40 con 0x7f17a40737b0 2023-10-17T01:14:47.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.679+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00053c0 con 0x7f17a40737b0 2023-10-17T01:14:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.679+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a41a1160 con 0x7f17a40737b0 2023-10-17T01:14:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.680+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a41072c0 con 0x7f17a40737b0 2023-10-17T01:14:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.682+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f17a0005580 con 0x7f17a40737b0 2023-10-17T01:14:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.683+0000 7f179affd700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17900a0bf0 0x7f17900a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.683+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f17a00cd270 con 0x7f17a40737b0 2023-10-17T01:14:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.683+0000 7f17a979a700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17900a0bf0 0x7f17900a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.684+0000 7f17a979a700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17900a0bf0 0x7f17900a30b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f17a4074ab0 tx=0x7f1794009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.687+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f17a005b0b0 con 0x7f17a40737b0 2023-10-17T01:14:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:47 smithi045 ceph-mon[98474]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-17T01:14:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:47 smithi060 ceph-mon[134136]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-17T01:14:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.894+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 --> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17a4061190 con 0x7f17900a0bf0 2023-10-17T01:14:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.905+0000 7f179affd700 1 -- 172.21.15.45:0/1199927089 <== mgr.14150 v2:172.21.15.45:6800/869456926 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7f17a4061190 con 0x7f17900a0bf0 2023-10-17T01:14:47.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:47.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:47 smithi181 ceph-mon[134520]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-17T01:14:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17900a0bf0 msgr2=0x7f17900a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17900a0bf0 0x7f17900a30b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f17a4074ab0 tx=0x7f1794009260 comp rx=0 tx=0).stop 2023-10-17T01:14:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 msgr2=0x7f17a4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 0x7f17a4198c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f17a0004740 tx=0x7f17a0009580 comp rx=0 tx=0).stop 2023-10-17T01:14:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 shutdown_connections 2023-10-17T01:14:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f17900a0bf0 0x7f17900a30b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a4108c10 0x7f17a41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a40737b0 0x7f17a4198c50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 --2- 172.21.15.45:0/1199927089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a4074d90 0x7f17a4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 >> 172.21.15.45:0/1199927089 conn(0x7f17a40fc280 msgr2=0x7f17a40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.909+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 shutdown_connections 2023-10-17T01:14:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:47.910+0000 7f17ac1ff700 1 -- 172.21.15.45:0/1199927089 wait complete. 2023-10-17T01:14:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-17T01:14:47.928 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-17T01:14:46.458100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655086,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1122,"num_read_kb":677,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":851,"ondisk_log_size":851,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334820,"kb_used_data":11396,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607004,"statfs":{"total":1151940427776,"available":1151597572096,"internally_reserved":0,"allocated":11669504,"data_stored":7279338,"data_compressed":107733,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1022,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":63,"num_write":14,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003018"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474988+0000","last_change":"2023-10-17T01:13:01.060354+0000","last_active":"2023-10-17T01:14:32.474988+0000","last_peered":"2023-10-17T01:14:32.474988+0000","last_clean":"2023-10-17T01:14:32.474988+0000","last_became_active":"2023-10-17T01:13:01.059721+0000","last_became_peered":"2023-10-17T01:13:01.059721+0000","last_unstale":"2023-10-17T01:14:32.474988+0000","last_undegraded":"2023-10-17T01:14:32.474988+0000","last_fullsized":"2023-10-17T01:14:32.474988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:45:50.238967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475967+0000","last_change":"2023-10-17T01:13:01.083408+0000","last_active":"2023-10-17T01:14:32.475967+0000","last_peered":"2023-10-17T01:14:32.475967+0000","last_clean":"2023-10-17T01:14:32.475967+0000","last_became_active":"2023-10-17T01:13:01.082981+0000","last_became_peered":"2023-10-17T01:13:01.082981+0000","last_unstale":"2023-10-17T01:14:32.475967+0000","last_undegraded":"2023-10-17T01:14:32.475967+0000","last_fullsized":"2023-10-17T01:14:32.475967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:56:21.430509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479353+0000","last_change":"2023-10-17T01:13:01.069445+0000","last_active":"2023-10-17T01:14:32.479353+0000","last_peered":"2023-10-17T01:14:32.479353+0000","last_clean":"2023-10-17T01:14:32.479353+0000","last_became_active":"2023-10-17T01:13:01.067583+0000","last_became_peered":"2023-10-17T01:13:01.067583+0000","last_unstale":"2023-10-17T01:14:32.479353+0000","last_undegraded":"2023-10-17T01:14:32.479353+0000","last_fullsized":"2023-10-17T01:14:32.479353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:23:12.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.462397+0000","last_change":"2023-10-17T01:13:33.514935+0000","last_active":"2023-10-17T01:14:32.462397+0000","last_peered":"2023-10-17T01:14:32.462397+0000","last_clean":"2023-10-17T01:14:32.462397+0000","last_became_active":"2023-10-17T01:13:33.514537+0000","last_became_peered":"2023-10-17T01:13:33.514537+0000","last_unstale":"2023-10-17T01:14:32.462397+0000","last_undegraded":"2023-10-17T01:14:32.462397+0000","last_fullsized":"2023-10-17T01:14:32.462397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:41:56.247357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490466+0000","last_change":"2023-10-17T01:13:01.073412+0000","last_active":"2023-10-17T01:14:32.490466+0000","last_peered":"2023-10-17T01:14:32.490466+0000","last_clean":"2023-10-17T01:14:32.490466+0000","last_became_active":"2023-10-17T01:13:01.073110+0000","last_became_peered":"2023-10-17T01:13:01.073110+0000","last_unstale":"2023-10-17T01:14:32.490466+0000","last_undegraded":"2023-10-17T01:14:32.490466+0000","last_fullsized":"2023-10-17T01:14:32.490466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:50:28.539025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.466611+0000","last_change":"2023-10-17T01:13:01.066338+0000","last_active":"2023-10-17T01:14:32.466611+0000","last_peered":"2023-10-17T01:14:32.466611+0000","last_clean":"2023-10-17T01:14:32.466611+0000","last_became_active":"2023-10-17T01:13:01.065944+0000","last_became_peered":"2023-10-17T01:13:01.065944+0000","last_unstale":"2023-10-17T01:14:32.466611+0000","last_undegraded":"2023-10-17T01:14:32.466611+0000","last_fullsized":"2023-10-17T01:14:32.466611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:07:37.039598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.457606+0000","last_change":"2023-10-17T01:13:01.066721+0000","last_active":"2023-10-17T01:14:32.457606+0000","last_peered":"2023-10-17T01:14:32.457606+0000","last_clean":"2023-10-17T01:14:32.457606+0000","last_became_active":"2023-10-17T01:13:01.066375+0000","last_became_peered":"2023-10-17T01:13:01.066375+0000","last_unstale":"2023-10-17T01:14:32.457606+0000","last_undegraded":"2023-10-17T01:14:32.457606+0000","last_fullsized":"2023-10-17T01:14:32.457606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:57:05.850558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490514+0000","last_change":"2023-10-17T01:13:01.064299+0000","last_active":"2023-10-17T01:14:32.490514+0000","last_peered":"2023-10-17T01:14:32.490514+0000","last_clean":"2023-10-17T01:14:32.490514+0000","last_became_active":"2023-10-17T01:13:01.062462+0000","last_became_peered":"2023-10-17T01:13:01.062462+0000","last_unstale":"2023-10-17T01:14:32.490514+0000","last_undegraded":"2023-10-17T01:14:32.490514+0000","last_fullsized":"2023-10-17T01:14:32.490514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:54:48.475827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.476373+0000","last_change":"2023-10-17T01:13:01.060962+0000","last_active":"2023-10-17T01:14:32.476373+0000","last_peered":"2023-10-17T01:14:32.476373+0000","last_clean":"2023-10-17T01:14:32.476373+0000","last_became_active":"2023-10-17T01:13:01.060703+0000","last_became_peered":"2023-10-17T01:13:01.060703+0000","last_unstale":"2023-10-17T01:14:32.476373+0000","last_undegraded":"2023-10-17T01:14:32.476373+0000","last_fullsized":"2023-10-17T01:14:32.476373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:05.249129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.472094+0000","last_change":"2023-10-17T01:13:01.070863+0000","last_active":"2023-10-17T01:14:32.472094+0000","last_peered":"2023-10-17T01:14:32.472094+0000","last_clean":"2023-10-17T01:14:32.472094+0000","last_became_active":"2023-10-17T01:13:01.070600+0000","last_became_peered":"2023-10-17T01:13:01.070600+0000","last_unstale":"2023-10-17T01:14:32.472094+0000","last_undegraded":"2023-10-17T01:14:32.472094+0000","last_fullsized":"2023-10-17T01:14:32.472094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:51:13.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.480952+0000","last_change":"2023-10-17T01:13:01.073335+0000","last_active":"2023-10-17T01:14:32.480952+0000","last_peered":"2023-10-17T01:14:32.480952+0000","last_clean":"2023-10-17T01:14:32.480952+0000","last_became_active":"2023-10-17T01:13:01.073014+0000","last_became_peered":"2023-10-17T01:13:01.073014+0000","last_unstale":"2023-10-17T01:14:32.480952+0000","last_undegraded":"2023-10-17T01:14:32.480952+0000","last_fullsized":"2023-10-17T01:14:32.480952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:15:24.389178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.461642+0000","last_change":"2023-10-17T01:13:01.071567+0000","last_active":"2023-10-17T01:14:32.461642+0000","last_peered":"2023-10-17T01:14:32.461642+0000","last_clean":"2023-10-17T01:14:32.461642+0000","last_became_active":"2023-10-17T01:13:01.071219+0000","last_became_peered":"2023-10-17T01:13:01.071219+0000","last_unstale":"2023-10-17T01:14:32.461642+0000","last_undegraded":"2023-10-17T01:14:32.461642+0000","last_fullsized":"2023-10-17T01:14:32.461642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:20:35.869051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490977+0000","last_change":"2023-10-17T01:13:01.070367+0000","last_active":"2023-10-17T01:14:32.490977+0000","last_peered":"2023-10-17T01:14:32.490977+0000","last_clean":"2023-10-17T01:14:32.490977+0000","last_became_active":"2023-10-17T01:13:01.069849+0000","last_became_peered":"2023-10-17T01:13:01.069849+0000","last_unstale":"2023-10-17T01:14:32.490977+0000","last_undegraded":"2023-10-17T01:14:32.490977+0000","last_fullsized":"2023-10-17T01:14:32.490977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:32:41.719090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474419+0000","last_change":"2023-10-17T01:13:01.079546+0000","last_active":"2023-10-17T01:14:32.474419+0000","last_peered":"2023-10-17T01:14:32.474419+0000","last_clean":"2023-10-17T01:14:32.474419+0000","last_became_active":"2023-10-17T01:13:01.077531+0000","last_became_peered":"2023-10-17T01:13:01.077531+0000","last_unstale":"2023-10-17T01:14:32.474419+0000","last_undegraded":"2023-10-17T01:14:32.474419+0000","last_fullsized":"2023-10-17T01:14:32.474419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:08.895866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.477729+0000","last_change":"2023-10-17T01:13:01.079631+0000","last_active":"2023-10-17T01:14:32.477729+0000","last_peered":"2023-10-17T01:14:32.477729+0000","last_clean":"2023-10-17T01:14:32.477729+0000","last_became_active":"2023-10-17T01:13:01.078803+0000","last_became_peered":"2023-10-17T01:13:01.078803+0000","last_unstale":"2023-10-17T01:14:32.477729+0000","last_undegraded":"2023-10-17T01:14:32.477729+0000","last_fullsized":"2023-10-17T01:14:32.477729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:17:50.962954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.478337+0000","last_change":"2023-10-17T01:13:01.067280+0000","last_active":"2023-10-17T01:14:32.478337+0000","last_peered":"2023-10-17T01:14:32.478337+0000","last_clean":"2023-10-17T01:14:32.478337+0000","last_became_active":"2023-10-17T01:13:01.066943+0000","last_became_peered":"2023-10-17T01:13:01.066943+0000","last_unstale":"2023-10-17T01:14:32.478337+0000","last_undegraded":"2023-10-17T01:14:32.478337+0000","last_fullsized":"2023-10-17T01:14:32.478337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:57.861426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.478690+0000","last_change":"2023-10-17T01:13:01.060529+0000","last_active":"2023-10-17T01:14:32.478690+0000","last_peered":"2023-10-17T01:14:32.478690+0000","last_clean":"2023-10-17T01:14:32.478690+0000","last_became_active":"2023-10-17T01:13:01.060089+0000","last_became_peered":"2023-10-17T01:13:01.060089+0000","last_unstale":"2023-10-17T01:14:32.478690+0000","last_undegraded":"2023-10-17T01:14:32.478690+0000","last_fullsized":"2023-10-17T01:14:32.478690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:06:35.397685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.477870+0000","last_change":"2023-10-17T01:13:01.058266+0000","last_active":"2023-10-17T01:14:32.477870+0000","last_peered":"2023-10-17T01:14:32.477870+0000","last_clean":"2023-10-17T01:14:32.477870+0000","last_became_active":"2023-10-17T01:13:01.057966+0000","last_became_peered":"2023-10-17T01:13:01.057966+0000","last_unstale":"2023-10-17T01:14:32.477870+0000","last_undegraded":"2023-10-17T01:14:32.477870+0000","last_fullsized":"2023-10-17T01:14:32.477870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:27:02.136105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484919+0000","last_change":"2023-10-17T01:13:01.067920+0000","last_active":"2023-10-17T01:14:32.484919+0000","last_peered":"2023-10-17T01:14:32.484919+0000","last_clean":"2023-10-17T01:14:32.484919+0000","last_became_active":"2023-10-17T01:13:01.067619+0000","last_became_peered":"2023-10-17T01:13:01.067619+0000","last_unstale":"2023-10-17T01:14:32.484919+0000","last_undegraded":"2023-10-17T01:14:32.484919+0000","last_fullsized":"2023-10-17T01:14:32.484919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:13:51.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.595468+0000","last_change":"2023-10-17T01:13:00.488928+0000","last_active":"2023-10-17T01:14:43.595468+0000","last_peered":"2023-10-17T01:14:43.595468+0000","last_clean":"2023-10-17T01:14:43.595468+0000","last_became_active":"2023-10-17T01:13:00.487936+0000","last_became_peered":"2023-10-17T01:13:00.487936+0000","last_unstale":"2023-10-17T01:14:43.595468+0000","last_undegraded":"2023-10-17T01:14:43.595468+0000","last_fullsized":"2023-10-17T01:14:43.595468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:47:37.240387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.482277+0000","last_change":"2023-10-17T01:13:01.058765+0000","last_active":"2023-10-17T01:14:32.482277+0000","last_peered":"2023-10-17T01:14:32.482277+0000","last_clean":"2023-10-17T01:14:32.482277+0000","last_became_active":"2023-10-17T01:13:01.058339+0000","last_became_peered":"2023-10-17T01:13:01.058339+0000","last_unstale":"2023-10-17T01:14:32.482277+0000","last_undegraded":"2023-10-17T01:14:32.482277+0000","last_fullsized":"2023-10-17T01:14:32.482277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:47:46.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512845+0000","last_change":"2023-10-17T01:13:00.498947+0000","last_active":"2023-10-17T01:13:32.512845+0000","last_peered":"2023-10-17T01:13:32.512845+0000","last_clean":"2023-10-17T01:13:32.512845+0000","last_became_active":"2023-10-17T01:13:00.498172+0000","last_became_peered":"2023-10-17T01:13:00.498172+0000","last_unstale":"2023-10-17T01:13:32.512845+0000","last_undegraded":"2023-10-17T01:13:32.512845+0000","last_fullsized":"2023-10-17T01:13:32.512845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:02:37.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479641+0000","last_change":"2023-10-17T01:13:01.076347+0000","last_active":"2023-10-17T01:14:32.479641+0000","last_peered":"2023-10-17T01:14:32.479641+0000","last_clean":"2023-10-17T01:14:32.479641+0000","last_became_active":"2023-10-17T01:13:01.074364+0000","last_became_peered":"2023-10-17T01:13:01.074364+0000","last_unstale":"2023-10-17T01:14:32.479641+0000","last_undegraded":"2023-10-17T01:14:32.479641+0000","last_fullsized":"2023-10-17T01:14:32.479641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:09:41.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516202+0000","last_change":"2023-10-17T01:13:00.491024+0000","last_active":"2023-10-17T01:13:32.516202+0000","last_peered":"2023-10-17T01:13:32.516202+0000","last_clean":"2023-10-17T01:13:32.516202+0000","last_became_active":"2023-10-17T01:13:00.490690+0000","last_became_peered":"2023-10-17T01:13:00.490690+0000","last_unstale":"2023-10-17T01:13:32.516202+0000","last_undegraded":"2023-10-17T01:13:32.516202+0000","last_fullsized":"2023-10-17T01:13:32.516202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:25:50.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486971+0000","last_change":"2023-10-17T01:13:01.065100+0000","last_active":"2023-10-17T01:14:32.486971+0000","last_peered":"2023-10-17T01:14:32.486971+0000","last_clean":"2023-10-17T01:14:32.486971+0000","last_became_active":"2023-10-17T01:13:01.064775+0000","last_became_peered":"2023-10-17T01:13:01.064775+0000","last_unstale":"2023-10-17T01:14:32.486971+0000","last_undegraded":"2023-10-17T01:14:32.486971+0000","last_fullsized":"2023-10-17T01:14:32.486971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:34:00.837705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516290+0000","last_change":"2023-10-17T01:13:00.488241+0000","last_active":"2023-10-17T01:13:32.516290+0000","last_peered":"2023-10-17T01:13:32.516290+0000","last_clean":"2023-10-17T01:13:32.516290+0000","last_became_active":"2023-10-17T01:13:00.487955+0000","last_became_peered":"2023-10-17T01:13:00.487955+0000","last_unstale":"2023-10-17T01:13:32.516290+0000","last_undegraded":"2023-10-17T01:13:32.516290+0000","last_fullsized":"2023-10-17T01:13:32.516290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:45:45.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475047+0000","last_change":"2023-10-17T01:13:01.062099+0000","last_active":"2023-10-17T01:14:32.475047+0000","last_peered":"2023-10-17T01:14:32.475047+0000","last_clean":"2023-10-17T01:14:32.475047+0000","last_became_active":"2023-10-17T01:13:01.061816+0000","last_became_peered":"2023-10-17T01:13:01.061816+0000","last_unstale":"2023-10-17T01:14:32.475047+0000","last_undegraded":"2023-10-17T01:14:32.475047+0000","last_fullsized":"2023-10-17T01:14:32.475047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:09:25.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.596973+0000","last_change":"2023-10-17T01:13:00.486049+0000","last_active":"2023-10-17T01:14:43.596973+0000","last_peered":"2023-10-17T01:14:43.596973+0000","last_clean":"2023-10-17T01:14:43.596973+0000","last_became_active":"2023-10-17T01:13:00.485753+0000","last_became_peered":"2023-10-17T01:13:00.485753+0000","last_unstale":"2023-10-17T01:14:43.596973+0000","last_undegraded":"2023-10-17T01:14:43.596973+0000","last_fullsized":"2023-10-17T01:14:43.596973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:56:35.592452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1107,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.460354+0000","last_change":"2023-10-17T01:13:01.065815+0000","last_active":"2023-10-17T01:14:32.460354+0000","last_peered":"2023-10-17T01:14:32.460354+0000","last_clean":"2023-10-17T01:14:32.460354+0000","last_became_active":"2023-10-17T01:13:01.065527+0000","last_became_peered":"2023-10-17T01:13:01.065527+0000","last_unstale":"2023-10-17T01:14:32.460354+0000","last_undegraded":"2023-10-17T01:14:32.460354+0000","last_fullsized":"2023-10-17T01:14:32.460354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:37:54.679396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:34.029617+0000","last_change":"2023-10-17T01:13:34.029617+0000","last_active":"2023-10-17T01:13:34.029617+0000","last_peered":"2023-10-17T01:13:34.029617+0000","last_clean":"2023-10-17T01:13:34.029617+0000","last_became_active":"2023-10-17T01:13:34.023843+0000","last_became_peered":"2023-10-17T01:13:34.023843+0000","last_unstale":"2023-10-17T01:13:34.029617+0000","last_undegraded":"2023-10-17T01:13:34.029617+0000","last_fullsized":"2023-10-17T01:13:34.029617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:03:37.817216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484628+0000","last_change":"2023-10-17T01:13:34.130813+0000","last_active":"2023-10-17T01:14:32.484628+0000","last_peered":"2023-10-17T01:14:32.484628+0000","last_clean":"2023-10-17T01:14:32.484628+0000","last_became_active":"2023-10-17T01:13:34.130345+0000","last_became_peered":"2023-10-17T01:13:34.130345+0000","last_unstale":"2023-10-17T01:14:32.484628+0000","last_undegraded":"2023-10-17T01:14:32.484628+0000","last_fullsized":"2023-10-17T01:14:32.484628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:56:47.808654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508250+0000","last_change":"2023-10-17T01:13:00.490672+0000","last_active":"2023-10-17T01:13:32.508250+0000","last_peered":"2023-10-17T01:13:32.508250+0000","last_clean":"2023-10-17T01:13:32.508250+0000","last_became_active":"2023-10-17T01:13:00.488284+0000","last_became_peered":"2023-10-17T01:13:00.488284+0000","last_unstale":"2023-10-17T01:13:32.508250+0000","last_undegraded":"2023-10-17T01:13:32.508250+0000","last_fullsized":"2023-10-17T01:13:32.508250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:14:08.382771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.491221+0000","last_change":"2023-10-17T01:13:01.068262+0000","last_active":"2023-10-17T01:14:32.491221+0000","last_peered":"2023-10-17T01:14:32.491221+0000","last_clean":"2023-10-17T01:14:32.491221+0000","last_became_active":"2023-10-17T01:13:01.067646+0000","last_became_peered":"2023-10-17T01:13:01.067646+0000","last_unstale":"2023-10-17T01:14:32.491221+0000","last_undegraded":"2023-10-17T01:14:32.491221+0000","last_fullsized":"2023-10-17T01:14:32.491221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:36:48.054338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.507782+0000","last_change":"2023-10-17T01:13:00.488251+0000","last_active":"2023-10-17T01:13:32.507782+0000","last_peered":"2023-10-17T01:13:32.507782+0000","last_clean":"2023-10-17T01:13:32.507782+0000","last_became_active":"2023-10-17T01:13:00.487905+0000","last_became_peered":"2023-10-17T01:13:00.487905+0000","last_unstale":"2023-10-17T01:13:32.507782+0000","last_undegraded":"2023-10-17T01:13:32.507782+0000","last_fullsized":"2023-10-17T01:13:32.507782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:00:45.688186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484000+0000","last_change":"2023-10-17T01:13:01.065204+0000","last_active":"2023-10-17T01:14:32.484000+0000","last_peered":"2023-10-17T01:14:32.484000+0000","last_clean":"2023-10-17T01:14:32.484000+0000","last_became_active":"2023-10-17T01:13:01.064379+0000","last_became_peered":"2023-10-17T01:13:01.064379+0000","last_unstale":"2023-10-17T01:14:32.484000+0000","last_undegraded":"2023-10-17T01:14:32.484000+0000","last_fullsized":"2023-10-17T01:14:32.484000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:00:01.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":208,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:41.593389+0000","last_change":"2023-10-17T01:13:00.484672+0000","last_active":"2023-10-17T01:14:41.593389+0000","last_peered":"2023-10-17T01:14:41.593389+0000","last_clean":"2023-10-17T01:14:41.593389+0000","last_became_active":"2023-10-17T01:13:00.484327+0000","last_became_peered":"2023-10-17T01:13:00.484327+0000","last_unstale":"2023-10-17T01:14:41.593389+0000","last_undegraded":"2023-10-17T01:14:41.593389+0000","last_fullsized":"2023-10-17T01:14:41.593389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:28:28.995382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.476176+0000","last_change":"2023-10-17T01:13:01.058430+0000","last_active":"2023-10-17T01:14:32.476176+0000","last_peered":"2023-10-17T01:14:32.476176+0000","last_clean":"2023-10-17T01:14:32.476176+0000","last_became_active":"2023-10-17T01:13:01.058097+0000","last_became_peered":"2023-10-17T01:13:01.058097+0000","last_unstale":"2023-10-17T01:14:32.476176+0000","last_undegraded":"2023-10-17T01:14:32.476176+0000","last_fullsized":"2023-10-17T01:14:32.476176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:50:45.925629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513474+0000","last_change":"2023-10-17T01:13:00.487178+0000","last_active":"2023-10-17T01:13:32.513474+0000","last_peered":"2023-10-17T01:13:32.513474+0000","last_clean":"2023-10-17T01:13:32.513474+0000","last_became_active":"2023-10-17T01:13:00.486866+0000","last_became_peered":"2023-10-17T01:13:00.486866+0000","last_unstale":"2023-10-17T01:13:32.513474+0000","last_undegraded":"2023-10-17T01:13:32.513474+0000","last_fullsized":"2023-10-17T01:13:32.513474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:15:43.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.471633+0000","last_change":"2023-10-17T01:13:01.083353+0000","last_active":"2023-10-17T01:14:32.471633+0000","last_peered":"2023-10-17T01:14:32.471633+0000","last_clean":"2023-10-17T01:14:32.471633+0000","last_became_active":"2023-10-17T01:13:01.077920+0000","last_became_peered":"2023-10-17T01:13:01.077920+0000","last_unstale":"2023-10-17T01:14:32.471633+0000","last_undegraded":"2023-10-17T01:14:32.471633+0000","last_fullsized":"2023-10-17T01:14:32.471633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:20:33.885951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508333+0000","last_change":"2023-10-17T01:13:00.503472+0000","last_active":"2023-10-17T01:13:32.508333+0000","last_peered":"2023-10-17T01:13:32.508333+0000","last_clean":"2023-10-17T01:13:32.508333+0000","last_became_active":"2023-10-17T01:13:00.503277+0000","last_became_peered":"2023-10-17T01:13:00.503277+0000","last_unstale":"2023-10-17T01:13:32.508333+0000","last_undegraded":"2023-10-17T01:13:32.508333+0000","last_fullsized":"2023-10-17T01:13:32.508333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:30:22.722944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.479738+0000","last_change":"2023-10-17T01:13:01.063033+0000","last_active":"2023-10-17T01:14:32.479738+0000","last_peered":"2023-10-17T01:14:32.479738+0000","last_clean":"2023-10-17T01:14:32.479738+0000","last_became_active":"2023-10-17T01:13:01.062672+0000","last_became_peered":"2023-10-17T01:13:01.062672+0000","last_unstale":"2023-10-17T01:14:32.479738+0000","last_undegraded":"2023-10-17T01:14:32.479738+0000","last_fullsized":"2023-10-17T01:14:32.479738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:27:17.266361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":243,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:41.593836+0000","last_change":"2023-10-17T01:13:00.485766+0000","last_active":"2023-10-17T01:14:41.593836+0000","last_peered":"2023-10-17T01:14:41.593836+0000","last_clean":"2023-10-17T01:14:41.593836+0000","last_became_active":"2023-10-17T01:13:00.485424+0000","last_became_peered":"2023-10-17T01:13:00.485424+0000","last_unstale":"2023-10-17T01:14:41.593836+0000","last_undegraded":"2023-10-17T01:14:41.593836+0000","last_fullsized":"2023-10-17T01:14:41.593836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:35:01.511910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57393,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":153,"num_write":35,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470653+0000","last_change":"2023-10-17T01:13:01.073203+0000","last_active":"2023-10-17T01:14:32.470653+0000","last_peered":"2023-10-17T01:14:32.470653+0000","last_clean":"2023-10-17T01:14:32.470653+0000","last_became_active":"2023-10-17T01:13:01.072821+0000","last_became_peered":"2023-10-17T01:13:01.072821+0000","last_unstale":"2023-10-17T01:14:32.470653+0000","last_undegraded":"2023-10-17T01:14:32.470653+0000","last_fullsized":"2023-10-17T01:14:32.470653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:21:16.828944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513865+0000","last_change":"2023-10-17T01:13:00.482711+0000","last_active":"2023-10-17T01:13:32.513865+0000","last_peered":"2023-10-17T01:13:32.513865+0000","last_clean":"2023-10-17T01:13:32.513865+0000","last_became_active":"2023-10-17T01:13:00.482405+0000","last_became_peered":"2023-10-17T01:13:00.482405+0000","last_unstale":"2023-10-17T01:13:32.513865+0000","last_undegraded":"2023-10-17T01:13:32.513865+0000","last_fullsized":"2023-10-17T01:13:32.513865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:42:51.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486907+0000","last_change":"2023-10-17T01:13:01.072052+0000","last_active":"2023-10-17T01:14:32.486907+0000","last_peered":"2023-10-17T01:14:32.486907+0000","last_clean":"2023-10-17T01:14:32.486907+0000","last_became_active":"2023-10-17T01:13:01.071640+0000","last_became_peered":"2023-10-17T01:13:01.071640+0000","last_unstale":"2023-10-17T01:14:32.486907+0000","last_undegraded":"2023-10-17T01:14:32.486907+0000","last_fullsized":"2023-10-17T01:14:32.486907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:07:36.428835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508044+0000","last_change":"2023-10-17T01:13:00.493154+0000","last_active":"2023-10-17T01:13:32.508044+0000","last_peered":"2023-10-17T01:13:32.508044+0000","last_clean":"2023-10-17T01:13:32.508044+0000","last_became_active":"2023-10-17T01:13:00.492874+0000","last_became_peered":"2023-10-17T01:13:00.492874+0000","last_unstale":"2023-10-17T01:13:32.508044+0000","last_undegraded":"2023-10-17T01:13:32.508044+0000","last_fullsized":"2023-10-17T01:13:32.508044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:15.007356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490807+0000","last_change":"2023-10-17T01:13:01.073519+0000","last_active":"2023-10-17T01:14:32.490807+0000","last_peered":"2023-10-17T01:14:32.490807+0000","last_clean":"2023-10-17T01:14:32.490807+0000","last_became_active":"2023-10-17T01:13:01.073225+0000","last_became_peered":"2023-10-17T01:13:01.073225+0000","last_unstale":"2023-10-17T01:14:32.490807+0000","last_undegraded":"2023-10-17T01:14:32.490807+0000","last_fullsized":"2023-10-17T01:14:32.490807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:26:25.943366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:43.597131+0000","last_change":"2023-10-17T01:13:00.485548+0000","last_active":"2023-10-17T01:14:43.597131+0000","last_peered":"2023-10-17T01:14:43.597131+0000","last_clean":"2023-10-17T01:14:43.597131+0000","last_became_active":"2023-10-17T01:13:00.485274+0000","last_became_peered":"2023-10-17T01:13:00.485274+0000","last_unstale":"2023-10-17T01:14:43.597131+0000","last_undegraded":"2023-10-17T01:14:43.597131+0000","last_fullsized":"2023-10-17T01:14:43.597131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:26:27.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.472490+0000","last_change":"2023-10-17T01:13:01.065606+0000","last_active":"2023-10-17T01:14:32.472490+0000","last_peered":"2023-10-17T01:14:32.472490+0000","last_clean":"2023-10-17T01:14:32.472490+0000","last_became_active":"2023-10-17T01:13:01.065296+0000","last_became_peered":"2023-10-17T01:13:01.065296+0000","last_unstale":"2023-10-17T01:14:32.472490+0000","last_undegraded":"2023-10-17T01:14:32.472490+0000","last_fullsized":"2023-10-17T01:14:32.472490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:37:38.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.858283+0000","last_change":"2023-10-17T01:13:33.858283+0000","last_active":"2023-10-17T01:13:33.858283+0000","last_peered":"2023-10-17T01:13:33.858283+0000","last_clean":"2023-10-17T01:13:33.858283+0000","last_became_active":"2023-10-17T01:13:33.844620+0000","last_became_peered":"2023-10-17T01:13:33.844620+0000","last_unstale":"2023-10-17T01:13:33.858283+0000","last_undegraded":"2023-10-17T01:13:33.858283+0000","last_fullsized":"2023-10-17T01:13:33.858283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:48:34.953384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.466233+0000","last_change":"2023-10-17T01:13:01.064967+0000","last_active":"2023-10-17T01:14:32.466233+0000","last_peered":"2023-10-17T01:14:32.466233+0000","last_clean":"2023-10-17T01:14:32.466233+0000","last_became_active":"2023-10-17T01:13:01.064680+0000","last_became_peered":"2023-10-17T01:13:01.064680+0000","last_unstale":"2023-10-17T01:14:32.466233+0000","last_undegraded":"2023-10-17T01:14:32.466233+0000","last_fullsized":"2023-10-17T01:14:32.466233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:51:27.451349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516499+0000","last_change":"2023-10-17T01:13:00.487754+0000","last_active":"2023-10-17T01:13:32.516499+0000","last_peered":"2023-10-17T01:13:32.516499+0000","last_clean":"2023-10-17T01:13:32.516499+0000","last_became_active":"2023-10-17T01:13:00.487424+0000","last_became_peered":"2023-10-17T01:13:00.487424+0000","last_unstale":"2023-10-17T01:13:32.516499+0000","last_undegraded":"2023-10-17T01:13:32.516499+0000","last_fullsized":"2023-10-17T01:13:32.516499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:00:41.618311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467302+0000","last_change":"2023-10-17T01:13:01.066523+0000","last_active":"2023-10-17T01:14:32.467302+0000","last_peered":"2023-10-17T01:14:32.467302+0000","last_clean":"2023-10-17T01:14:32.467302+0000","last_became_active":"2023-10-17T01:13:01.066226+0000","last_became_peered":"2023-10-17T01:13:01.066226+0000","last_unstale":"2023-10-17T01:14:32.467302+0000","last_undegraded":"2023-10-17T01:14:32.467302+0000","last_fullsized":"2023-10-17T01:14:32.467302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:17:36.794207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521188+0000","last_change":"2023-10-17T01:13:33.521188+0000","last_active":"2023-10-17T01:13:33.521188+0000","last_peered":"2023-10-17T01:13:33.521188+0000","last_clean":"2023-10-17T01:13:33.521188+0000","last_became_active":"2023-10-17T01:13:33.520645+0000","last_became_peered":"2023-10-17T01:13:33.520645+0000","last_unstale":"2023-10-17T01:13:33.521188+0000","last_undegraded":"2023-10-17T01:13:33.521188+0000","last_fullsized":"2023-10-17T01:13:33.521188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:55:42.028467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485550+0000","last_change":"2023-10-17T01:13:01.067901+0000","last_active":"2023-10-17T01:14:32.485550+0000","last_peered":"2023-10-17T01:14:32.485550+0000","last_clean":"2023-10-17T01:14:32.485550+0000","last_became_active":"2023-10-17T01:13:01.067567+0000","last_became_peered":"2023-10-17T01:13:01.067567+0000","last_unstale":"2023-10-17T01:14:32.485550+0000","last_undegraded":"2023-10-17T01:14:32.485550+0000","last_fullsized":"2023-10-17T01:14:32.485550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:30:30.925835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:42.594670+0000","last_change":"2023-10-17T01:13:00.496087+0000","last_active":"2023-10-17T01:14:42.594670+0000","last_peered":"2023-10-17T01:14:42.594670+0000","last_clean":"2023-10-17T01:14:42.594670+0000","last_became_active":"2023-10-17T01:13:00.495786+0000","last_became_peered":"2023-10-17T01:13:00.495786+0000","last_unstale":"2023-10-17T01:14:42.594670+0000","last_undegraded":"2023-10-17T01:14:42.594670+0000","last_fullsized":"2023-10-17T01:14:42.594670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:31:56.896318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470919+0000","last_change":"2023-10-17T01:13:01.069796+0000","last_active":"2023-10-17T01:14:32.470919+0000","last_peered":"2023-10-17T01:14:32.470919+0000","last_clean":"2023-10-17T01:14:32.470919+0000","last_became_active":"2023-10-17T01:13:01.069467+0000","last_became_peered":"2023-10-17T01:13:01.069467+0000","last_unstale":"2023-10-17T01:14:32.470919+0000","last_undegraded":"2023-10-17T01:14:32.470919+0000","last_fullsized":"2023-10-17T01:14:32.470919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:02:49.172993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467321+0000","last_change":"2023-10-17T01:13:01.061697+0000","last_active":"2023-10-17T01:14:32.467321+0000","last_peered":"2023-10-17T01:14:32.467321+0000","last_clean":"2023-10-17T01:14:32.467321+0000","last_became_active":"2023-10-17T01:13:01.061434+0000","last_became_peered":"2023-10-17T01:13:01.061434+0000","last_unstale":"2023-10-17T01:14:32.467321+0000","last_undegraded":"2023-10-17T01:14:32.467321+0000","last_fullsized":"2023-10-17T01:14:32.467321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:59:37.119060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:44.597987+0000","last_change":"2023-10-17T01:13:00.501898+0000","last_active":"2023-10-17T01:14:44.597987+0000","last_peered":"2023-10-17T01:14:44.597987+0000","last_clean":"2023-10-17T01:14:44.597987+0000","last_became_active":"2023-10-17T01:13:00.501378+0000","last_became_peered":"2023-10-17T01:13:00.501378+0000","last_unstale":"2023-10-17T01:14:44.597987+0000","last_undegraded":"2023-10-17T01:14:44.597987+0000","last_fullsized":"2023-10-17T01:14:44.597987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:52:09.874860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.483880+0000","last_change":"2023-10-17T01:13:01.054135+0000","last_active":"2023-10-17T01:14:32.483880+0000","last_peered":"2023-10-17T01:14:32.483880+0000","last_clean":"2023-10-17T01:14:32.483880+0000","last_became_active":"2023-10-17T01:13:01.051873+0000","last_became_peered":"2023-10-17T01:13:01.051873+0000","last_unstale":"2023-10-17T01:14:32.483880+0000","last_undegraded":"2023-10-17T01:14:32.483880+0000","last_fullsized":"2023-10-17T01:14:32.483880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:24:32.614588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512805+0000","last_change":"2023-10-17T01:13:00.483338+0000","last_active":"2023-10-17T01:13:32.512805+0000","last_peered":"2023-10-17T01:13:32.512805+0000","last_clean":"2023-10-17T01:13:32.512805+0000","last_became_active":"2023-10-17T01:13:00.482989+0000","last_became_peered":"2023-10-17T01:13:00.482989+0000","last_unstale":"2023-10-17T01:13:32.512805+0000","last_undegraded":"2023-10-17T01:13:32.512805+0000","last_fullsized":"2023-10-17T01:13:32.512805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:30:21.291417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.514380+0000","last_change":"2023-10-17T01:11:54.999941+0000","last_active":"2023-10-17T01:13:32.514380+0000","last_peered":"2023-10-17T01:13:32.514380+0000","last_clean":"2023-10-17T01:13:32.514380+0000","last_became_active":"2023-10-17T01:11:54.999476+0000","last_became_peered":"2023-10-17T01:11:54.999476+0000","last_unstale":"2023-10-17T01:13:32.514380+0000","last_undegraded":"2023-10-17T01:13:32.514380+0000","last_fullsized":"2023-10-17T01:13:32.514380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:08:49.310669+0000","last_clean_scrub_stamp":"2023-10-17T01:08:49.310669+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:24:36.340425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.483168+0000","last_change":"2023-10-17T01:13:01.066599+0000","last_active":"2023-10-17T01:14:32.483168+0000","last_peered":"2023-10-17T01:14:32.483168+0000","last_clean":"2023-10-17T01:14:32.483168+0000","last_became_active":"2023-10-17T01:13:01.066259+0000","last_became_peered":"2023-10-17T01:13:01.066259+0000","last_unstale":"2023-10-17T01:14:32.483168+0000","last_undegraded":"2023-10-17T01:14:32.483168+0000","last_fullsized":"2023-10-17T01:14:32.483168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:06:37.667449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.521833+0000","last_change":"2023-10-17T01:13:33.521833+0000","last_active":"2023-10-17T01:13:33.521833+0000","last_peered":"2023-10-17T01:13:33.521833+0000","last_clean":"2023-10-17T01:13:33.521833+0000","last_became_active":"2023-10-17T01:13:33.518951+0000","last_became_peered":"2023-10-17T01:13:33.518951+0000","last_unstale":"2023-10-17T01:13:33.521833+0000","last_undegraded":"2023-10-17T01:13:33.521833+0000","last_fullsized":"2023-10-17T01:13:33.521833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:59:26.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516020+0000","last_change":"2023-10-17T01:13:00.504912+0000","last_active":"2023-10-17T01:13:32.516020+0000","last_peered":"2023-10-17T01:13:32.516020+0000","last_clean":"2023-10-17T01:13:32.516020+0000","last_became_active":"2023-10-17T01:13:00.504704+0000","last_became_peered":"2023-10-17T01:13:00.504704+0000","last_unstale":"2023-10-17T01:13:32.516020+0000","last_undegraded":"2023-10-17T01:13:32.516020+0000","last_fullsized":"2023-10-17T01:13:32.516020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:20:28.721076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485317+0000","last_change":"2023-10-17T01:13:01.081459+0000","last_active":"2023-10-17T01:14:32.485317+0000","last_peered":"2023-10-17T01:14:32.485317+0000","last_clean":"2023-10-17T01:14:32.485317+0000","last_became_active":"2023-10-17T01:13:01.075277+0000","last_became_peered":"2023-10-17T01:13:01.075277+0000","last_unstale":"2023-10-17T01:14:32.485317+0000","last_undegraded":"2023-10-17T01:14:32.485317+0000","last_fullsized":"2023-10-17T01:14:32.485317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:37:55.113349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:35.331586+0000","last_change":"2023-10-17T01:13:00.487749+0000","last_active":"2023-10-17T01:14:35.331586+0000","last_peered":"2023-10-17T01:14:35.331586+0000","last_clean":"2023-10-17T01:14:35.331586+0000","last_became_active":"2023-10-17T01:13:00.487417+0000","last_became_peered":"2023-10-17T01:13:00.487417+0000","last_unstale":"2023-10-17T01:14:35.331586+0000","last_undegraded":"2023-10-17T01:14:35.331586+0000","last_fullsized":"2023-10-17T01:14:35.331586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:56:02.890243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.486127+0000","last_change":"2023-10-17T01:13:01.077883+0000","last_active":"2023-10-17T01:14:32.486127+0000","last_peered":"2023-10-17T01:14:32.486127+0000","last_clean":"2023-10-17T01:14:32.486127+0000","last_became_active":"2023-10-17T01:13:01.074577+0000","last_became_peered":"2023-10-17T01:13:01.074577+0000","last_unstale":"2023-10-17T01:14:32.486127+0000","last_undegraded":"2023-10-17T01:14:32.486127+0000","last_fullsized":"2023-10-17T01:14:32.486127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:34:06.813549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.512549+0000","last_change":"2023-10-17T01:13:00.487482+0000","last_active":"2023-10-17T01:13:32.512549+0000","last_peered":"2023-10-17T01:13:32.512549+0000","last_clean":"2023-10-17T01:13:32.512549+0000","last_became_active":"2023-10-17T01:13:00.487127+0000","last_became_peered":"2023-10-17T01:13:00.487127+0000","last_unstale":"2023-10-17T01:13:32.512549+0000","last_undegraded":"2023-10-17T01:13:32.512549+0000","last_fullsized":"2023-10-17T01:13:32.512549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:52:18.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484397+0000","last_change":"2023-10-17T01:13:01.075597+0000","last_active":"2023-10-17T01:14:32.484397+0000","last_peered":"2023-10-17T01:14:32.484397+0000","last_clean":"2023-10-17T01:14:32.484397+0000","last_became_active":"2023-10-17T01:13:01.075209+0000","last_became_peered":"2023-10-17T01:13:01.075209+0000","last_unstale":"2023-10-17T01:14:32.484397+0000","last_undegraded":"2023-10-17T01:14:32.484397+0000","last_fullsized":"2023-10-17T01:14:32.484397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:22:14.154087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.516068+0000","last_change":"2023-10-17T01:13:00.500897+0000","last_active":"2023-10-17T01:13:32.516068+0000","last_peered":"2023-10-17T01:13:32.516068+0000","last_clean":"2023-10-17T01:13:32.516068+0000","last_became_active":"2023-10-17T01:13:00.500626+0000","last_became_peered":"2023-10-17T01:13:00.500626+0000","last_unstale":"2023-10-17T01:13:32.516068+0000","last_undegraded":"2023-10-17T01:13:32.516068+0000","last_fullsized":"2023-10-17T01:13:32.516068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:39:12.405418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.450934+0000","last_change":"2023-10-17T01:13:01.069006+0000","last_active":"2023-10-17T01:14:32.450934+0000","last_peered":"2023-10-17T01:14:32.450934+0000","last_clean":"2023-10-17T01:14:32.450934+0000","last_became_active":"2023-10-17T01:13:01.068613+0000","last_became_peered":"2023-10-17T01:13:01.068613+0000","last_unstale":"2023-10-17T01:14:32.450934+0000","last_undegraded":"2023-10-17T01:14:32.450934+0000","last_fullsized":"2023-10-17T01:14:32.450934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:51:14.012167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.513931+0000","last_change":"2023-10-17T01:13:00.497247+0000","last_active":"2023-10-17T01:13:32.513931+0000","last_peered":"2023-10-17T01:13:32.513931+0000","last_clean":"2023-10-17T01:13:32.513931+0000","last_became_active":"2023-10-17T01:13:00.496894+0000","last_became_peered":"2023-10-17T01:13:00.496894+0000","last_unstale":"2023-10-17T01:13:32.513931+0000","last_undegraded":"2023-10-17T01:13:32.513931+0000","last_fullsized":"2023-10-17T01:13:32.513931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T04:46:45.975770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467086+0000","last_change":"2023-10-17T01:13:01.059390+0000","last_active":"2023-10-17T01:14:32.467086+0000","last_peered":"2023-10-17T01:14:32.467086+0000","last_clean":"2023-10-17T01:14:32.467086+0000","last_became_active":"2023-10-17T01:13:01.059088+0000","last_became_peered":"2023-10-17T01:13:01.059088+0000","last_unstale":"2023-10-17T01:14:32.467086+0000","last_undegraded":"2023-10-17T01:14:32.467086+0000","last_fullsized":"2023-10-17T01:14:32.467086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:34:37.046051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:13:33.511972+0000","last_change":"2023-10-17T01:13:00.486020+0000","last_active":"2023-10-17T01:13:33.511972+0000","last_peered":"2023-10-17T01:13:33.511972+0000","last_clean":"2023-10-17T01:13:33.511972+0000","last_became_active":"2023-10-17T01:13:00.485725+0000","last_became_peered":"2023-10-17T01:13:00.485725+0000","last_unstale":"2023-10-17T01:13:33.511972+0000","last_undegraded":"2023-10-17T01:13:33.511972+0000","last_fullsized":"2023-10-17T01:13:33.511972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T07:53:29.837958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.484207+0000","last_change":"2023-10-17T01:13:01.062751+0000","last_active":"2023-10-17T01:14:32.484207+0000","last_peered":"2023-10-17T01:14:32.484207+0000","last_clean":"2023-10-17T01:14:32.484207+0000","last_became_active":"2023-10-17T01:13:01.062405+0000","last_became_peered":"2023-10-17T01:13:01.062405+0000","last_unstale":"2023-10-17T01:14:32.484207+0000","last_undegraded":"2023-10-17T01:14:32.484207+0000","last_fullsized":"2023-10-17T01:14:32.484207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:13:04.779668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.515566+0000","last_change":"2023-10-17T01:13:00.485448+0000","last_active":"2023-10-17T01:13:32.515566+0000","last_peered":"2023-10-17T01:13:32.515566+0000","last_clean":"2023-10-17T01:13:32.515566+0000","last_became_active":"2023-10-17T01:13:00.485118+0000","last_became_peered":"2023-10-17T01:13:00.485118+0000","last_unstale":"2023-10-17T01:13:32.515566+0000","last_undegraded":"2023-10-17T01:13:32.515566+0000","last_fullsized":"2023-10-17T01:13:32.515566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:31:10.121845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.487926+0000","last_change":"2023-10-17T01:13:01.067155+0000","last_active":"2023-10-17T01:14:32.487926+0000","last_peered":"2023-10-17T01:14:32.487926+0000","last_clean":"2023-10-17T01:14:32.487926+0000","last_became_active":"2023-10-17T01:13:01.066856+0000","last_became_peered":"2023-10-17T01:13:01.066856+0000","last_unstale":"2023-10-17T01:14:32.487926+0000","last_undegraded":"2023-10-17T01:14:32.487926+0000","last_fullsized":"2023-10-17T01:14:32.487926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:11:25.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:42.595441+0000","last_change":"2023-10-17T01:13:00.500473+0000","last_active":"2023-10-17T01:14:42.595441+0000","last_peered":"2023-10-17T01:14:42.595441+0000","last_clean":"2023-10-17T01:14:42.595441+0000","last_became_active":"2023-10-17T01:13:00.500151+0000","last_became_peered":"2023-10-17T01:13:00.500151+0000","last_unstale":"2023-10-17T01:14:42.595441+0000","last_undegraded":"2023-10-17T01:14:42.595441+0000","last_fullsized":"2023-10-17T01:14:42.595441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T06:11:18.588532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1438,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.481056+0000","last_change":"2023-10-17T01:13:01.067973+0000","last_active":"2023-10-17T01:14:32.481056+0000","last_peered":"2023-10-17T01:14:32.481056+0000","last_clean":"2023-10-17T01:14:32.481056+0000","last_became_active":"2023-10-17T01:13:01.067525+0000","last_became_peered":"2023-10-17T01:13:01.067525+0000","last_unstale":"2023-10-17T01:14:32.481056+0000","last_undegraded":"2023-10-17T01:14:32.481056+0000","last_fullsized":"2023-10-17T01:14:32.481056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T12:08:26.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508206+0000","last_change":"2023-10-17T01:13:00.493161+0000","last_active":"2023-10-17T01:13:32.508206+0000","last_peered":"2023-10-17T01:13:32.508206+0000","last_clean":"2023-10-17T01:13:32.508206+0000","last_became_active":"2023-10-17T01:13:00.492865+0000","last_became_peered":"2023-10-17T01:13:00.492865+0000","last_unstale":"2023-10-17T01:13:32.508206+0000","last_undegraded":"2023-10-17T01:13:32.508206+0000","last_fullsized":"2023-10-17T01:13:32.508206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T05:49:29.015630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.457845+0000","last_change":"2023-10-17T01:13:01.067579+0000","last_active":"2023-10-17T01:14:32.457845+0000","last_peered":"2023-10-17T01:14:32.457845+0000","last_clean":"2023-10-17T01:14:32.457845+0000","last_became_active":"2023-10-17T01:13:01.067244+0000","last_became_peered":"2023-10-17T01:13:01.067244+0000","last_unstale":"2023-10-17T01:14:32.457845+0000","last_undegraded":"2023-10-17T01:14:32.457845+0000","last_fullsized":"2023-10-17T01:14:32.457845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:09:59.462276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-17T01:13:32.508369+0000","last_change":"2023-10-17T01:13:00.494266+0000","last_active":"2023-10-17T01:13:32.508369+0000","last_peered":"2023-10-17T01:13:32.508369+0000","last_clean":"2023-10-17T01:13:32.508369+0000","last_became_active":"2023-10-17T01:13:00.493950+0000","last_became_peered":"2023-10-17T01:13:00.493950+0000","last_unstale":"2023-10-17T01:13:32.508369+0000","last_undegraded":"2023-10-17T01:13:32.508369+0000","last_fullsized":"2023-10-17T01:13:32.508369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:12:59.432300+0000","last_clean_scrub_stamp":"2023-10-17T01:12:59.432300+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:44:24.531115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.469788+0000","last_change":"2023-10-17T01:13:01.066033+0000","last_active":"2023-10-17T01:14:32.469788+0000","last_peered":"2023-10-17T01:14:32.469788+0000","last_clean":"2023-10-17T01:14:32.469788+0000","last_became_active":"2023-10-17T01:13:01.065683+0000","last_became_peered":"2023-10-17T01:13:01.065683+0000","last_unstale":"2023-10-17T01:14:32.469788+0000","last_undegraded":"2023-10-17T01:14:32.469788+0000","last_fullsized":"2023-10-17T01:14:32.469788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:57:30.491360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470156+0000","last_change":"2023-10-17T01:13:01.061388+0000","last_active":"2023-10-17T01:14:32.470156+0000","last_peered":"2023-10-17T01:14:32.470156+0000","last_clean":"2023-10-17T01:14:32.470156+0000","last_became_active":"2023-10-17T01:13:01.061116+0000","last_became_peered":"2023-10-17T01:13:01.061116+0000","last_unstale":"2023-10-17T01:14:32.470156+0000","last_undegraded":"2023-10-17T01:14:32.470156+0000","last_fullsized":"2023-10-17T01:14:32.470156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:17:48.071250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.471400+0000","last_change":"2023-10-17T01:13:01.070369+0000","last_active":"2023-10-17T01:14:32.471400+0000","last_peered":"2023-10-17T01:14:32.471400+0000","last_clean":"2023-10-17T01:14:32.471400+0000","last_became_active":"2023-10-17T01:13:01.070177+0000","last_became_peered":"2023-10-17T01:13:01.070177+0000","last_unstale":"2023-10-17T01:14:32.471400+0000","last_undegraded":"2023-10-17T01:14:32.471400+0000","last_fullsized":"2023-10-17T01:14:32.471400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:24:42.579223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.482598+0000","last_change":"2023-10-17T01:13:01.071127+0000","last_active":"2023-10-17T01:14:32.482598+0000","last_peered":"2023-10-17T01:14:32.482598+0000","last_clean":"2023-10-17T01:14:32.482598+0000","last_became_active":"2023-10-17T01:13:01.070865+0000","last_became_peered":"2023-10-17T01:13:01.070865+0000","last_unstale":"2023-10-17T01:14:32.482598+0000","last_undegraded":"2023-10-17T01:14:32.482598+0000","last_fullsized":"2023-10-17T01:14:32.482598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T09:17:00.792278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490997+0000","last_change":"2023-10-17T01:13:01.079442+0000","last_active":"2023-10-17T01:14:32.490997+0000","last_peered":"2023-10-17T01:14:32.490997+0000","last_clean":"2023-10-17T01:14:32.490997+0000","last_became_active":"2023-10-17T01:13:01.079156+0000","last_became_peered":"2023-10-17T01:13:01.079156+0000","last_unstale":"2023-10-17T01:14:32.490997+0000","last_undegraded":"2023-10-17T01:14:32.490997+0000","last_fullsized":"2023-10-17T01:14:32.490997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:25:32.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485901+0000","last_change":"2023-10-17T01:13:01.081716+0000","last_active":"2023-10-17T01:14:32.485901+0000","last_peered":"2023-10-17T01:14:32.485901+0000","last_clean":"2023-10-17T01:14:32.485901+0000","last_became_active":"2023-10-17T01:13:01.081012+0000","last_became_peered":"2023-10-17T01:13:01.081012+0000","last_unstale":"2023-10-17T01:14:32.485901+0000","last_undegraded":"2023-10-17T01:14:32.485901+0000","last_fullsized":"2023-10-17T01:14:32.485901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T08:58:46.852184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.467479+0000","last_change":"2023-10-17T01:13:01.056073+0000","last_active":"2023-10-17T01:14:32.467479+0000","last_peered":"2023-10-17T01:14:32.467479+0000","last_clean":"2023-10-17T01:14:32.467479+0000","last_became_active":"2023-10-17T01:13:01.055751+0000","last_became_peered":"2023-10-17T01:13:01.055751+0000","last_unstale":"2023-10-17T01:14:32.467479+0000","last_undegraded":"2023-10-17T01:14:32.467479+0000","last_fullsized":"2023-10-17T01:14:32.467479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:45:28.833739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.474399+0000","last_change":"2023-10-17T01:13:01.061045+0000","last_active":"2023-10-17T01:14:32.474399+0000","last_peered":"2023-10-17T01:14:32.474399+0000","last_clean":"2023-10-17T01:14:32.474399+0000","last_became_active":"2023-10-17T01:13:01.060754+0000","last_became_peered":"2023-10-17T01:13:01.060754+0000","last_unstale":"2023-10-17T01:14:32.474399+0000","last_undegraded":"2023-10-17T01:14:32.474399+0000","last_fullsized":"2023-10-17T01:14:32.474399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:08:52.360094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485736+0000","last_change":"2023-10-17T01:13:01.070327+0000","last_active":"2023-10-17T01:14:32.485736+0000","last_peered":"2023-10-17T01:14:32.485736+0000","last_clean":"2023-10-17T01:14:32.485736+0000","last_became_active":"2023-10-17T01:13:01.069276+0000","last_became_peered":"2023-10-17T01:13:01.069276+0000","last_unstale":"2023-10-17T01:14:32.485736+0000","last_undegraded":"2023-10-17T01:14:32.485736+0000","last_fullsized":"2023-10-17T01:14:32.485736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T10:30:43.504843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.475798+0000","last_change":"2023-10-17T01:13:01.063579+0000","last_active":"2023-10-17T01:14:32.475798+0000","last_peered":"2023-10-17T01:14:32.475798+0000","last_clean":"2023-10-17T01:14:32.475798+0000","last_became_active":"2023-10-17T01:13:01.063312+0000","last_became_peered":"2023-10-17T01:13:01.063312+0000","last_unstale":"2023-10-17T01:14:32.475798+0000","last_undegraded":"2023-10-17T01:14:32.475798+0000","last_fullsized":"2023-10-17T01:14:32.475798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:02:27.131956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.485548+0000","last_change":"2023-10-17T01:13:01.053850+0000","last_active":"2023-10-17T01:14:32.485548+0000","last_peered":"2023-10-17T01:14:32.485548+0000","last_clean":"2023-10-17T01:14:32.485548+0000","last_became_active":"2023-10-17T01:13:01.052714+0000","last_became_peered":"2023-10-17T01:13:01.052714+0000","last_unstale":"2023-10-17T01:14:32.485548+0000","last_undegraded":"2023-10-17T01:14:32.485548+0000","last_fullsized":"2023-10-17T01:14:32.485548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T03:03:24.967101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.488268+0000","last_change":"2023-10-17T01:13:01.054298+0000","last_active":"2023-10-17T01:14:32.488268+0000","last_peered":"2023-10-17T01:14:32.488268+0000","last_clean":"2023-10-17T01:14:32.488268+0000","last_became_active":"2023-10-17T01:13:01.053923+0000","last_became_peered":"2023-10-17T01:13:01.053923+0000","last_unstale":"2023-10-17T01:14:32.488268+0000","last_undegraded":"2023-10-17T01:14:32.488268+0000","last_fullsized":"2023-10-17T01:14:32.488268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:06:09.654504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.490569+0000","last_change":"2023-10-17T01:13:01.080731+0000","last_active":"2023-10-17T01:14:32.490569+0000","last_peered":"2023-10-17T01:14:32.490569+0000","last_clean":"2023-10-17T01:14:32.490569+0000","last_became_active":"2023-10-17T01:13:01.077363+0000","last_became_peered":"2023-10-17T01:13:01.077363+0000","last_unstale":"2023-10-17T01:14:32.490569+0000","last_undegraded":"2023-10-17T01:14:32.490569+0000","last_fullsized":"2023-10-17T01:14:32.490569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T13:03:29.117132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-17T01:14:32.470712+0000","last_change":"2023-10-17T01:13:33.522423+0000","last_active":"2023-10-17T01:14:32.470712+0000","last_peered":"2023-10-17T01:14:32.470712+0000","last_clean":"2023-10-17T01:14:32.470712+0000","last_became_active":"2023-10-17T01:13:33.521927+0000","last_became_peered":"2023-10-17T01:13:33.521927+0000","last_unstale":"2023-10-17T01:14:32.470712+0000","last_undegraded":"2023-10-17T01:14:32.470712+0000","last_fullsized":"2023-10-17T01:14:32.470712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-17T01:13:00.463460+0000","last_clean_scrub_stamp":"2023-10-17T01:13:00.463460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T11:07:25.868960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282362,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64472,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":996,"num_read_kb":568,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":193416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72599999999999998,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.51500000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78100000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51000000000000001,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63700000000000001,"5min":1.468,"15min":1.468},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.66000000000000003,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.54400000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83999999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76900000000000002,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80300000000000005,"5min":1.409,"15min":1.409},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":1.159,"15min":1.159},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.749,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0980000000000001,"5min":1.236,"15min":1.236},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Tue Oct 17 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89000000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1259999999999999,"5min":1.359,"15min":1.359},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.68999999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.95699999999999996,"5min":1.21,"15min":1.21},"last":0.872},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.68200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94699999999999995,"5min":1.413,"15min":1.413},"last":0.83599999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":1092223,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.61599999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.91000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67500000000000004,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83899999999999997,"5min":0.87,"15min":0.87},"last":0.59799999999999998}]},{"osd":3,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76000000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.621}]},{"osd":5,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.86099999999999999}]},{"osd":6,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95299999999999996,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.78500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Oct 17 01:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88700000000000001,"5min":1.117,"15min":1.117},"last":0.874},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.131,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.879},"min":{"1min":0.60099999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.325,"15min":1.325},"last":0.78700000000000003}]},{"osd":11,"last update":"Tue Oct 17 01:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81499999999999995,"15min":0.81599999999999995},"min":{"1min":0.66600000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.96499999999999997,"5min":1.028,"15min":1.028},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84699999999999998,"15min":0.84899999999999998},"min":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.89300000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.51500000000000001,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.32500000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.379}]},{"osd":1,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.66800000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79300000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.437},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.751},"min":{"1min":0.48799999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70699999999999996,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77800000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.216,"15min":1.216},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.67500000000000004,"15min":0.71099999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68799999999999994,"15min":0.73399999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.76100000000000001,"15min":0.80400000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.177,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.877,"15min":0.86599999999999999},"min":{"1min":0.51500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90400000000000003,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.83899999999999997,"15min":0.88100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.198,"15min":1.198},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.84099999999999997,"15min":0.89500000000000002},"min":{"1min":0.59499999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:12:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.94299999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":6.9379999999999997,"15min":6.9379999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97399999999999998,"5min":1.27,"15min":1.27},"last":0.66200000000000003}]},{"osd":8,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.94499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.27,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92300000000000004,"15min":0.96699999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.996,"5min":1.242,"15min":1.242},"last":0.79700000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.93799999999999994},"min":{"1min":0.627,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.077,"5min":1.218,"15min":1.218},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.89900000000000002,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0189999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Tue Oct 17 01:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.749},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86499999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55000000000000004}]},{"osd":1,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.81499999999999995,"5min":0.874,"15min":0.874},"last":0.626},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.57899999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001}]},{"osd":2,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45100000000000001},"min":{"1min":0.39100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.436},{"interface":"front","average":{"1min":0.504,"5min":0.47799999999999998,"15min":0.46700000000000003},"min":{"1min":0.40100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.61599999999999999,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.498,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.50900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94799999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.61899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.67400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0009999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.755,"15min":0.753},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93799999999999994,"5min":1.51,"15min":1.51},"last":0.77500000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.53200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79700000000000004,"15min":0.77600000000000002},"min":{"1min":0.67200000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92400000000000004}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.755,"15min":0.748},"min":{"1min":0.58999999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.67700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98799999999999999,"5min":0.996,"15min":0.996},"last":0.748}]},{"osd":11,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.89300000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":1036658,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.373}]},{"osd":1,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64700000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":1.373,"15min":1.373},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85499999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.74099999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83199999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81499999999999995,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84299999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.46300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99199999999999999,"5min":1.879,"15min":1.879},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0660000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.64200000000000002}]},{"osd":8,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77300000000000002,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.014,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95399999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65900000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.86499999999999999,"15min":0.89200000000000002},"min":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85899999999999999,"15min":0.89500000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.76200000000000001}]},{"osd":10,"last update":"Tue Oct 17 01:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.83799999999999997,"15min":0.84999999999999998},"min":{"1min":0.50800000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.02,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.56000000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":1.149,"15min":1.149},"last":0.872}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68500000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.443},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.496},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68899999999999995,"5min":1.339,"15min":1.339},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.47799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80500000000000005,"5min":1.458,"15min":1.458},"last":0.748},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.752,"15min":0.73099999999999998},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.80700000000000005}]},{"osd":3,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67800000000000005,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90200000000000002,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.46899999999999997}]},{"osd":4,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73999999999999999,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0640000000000001,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.61399999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1319999999999999,"5min":1.474,"15min":1.474},"last":0.70799999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:14:30 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95599999999999996,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.54400000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.57999999999999996}]},{"osd":8,"last update":"Tue Oct 17 01:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.91100000000000003,"15min":0.90900000000000003},"min":{"1min":0.70599999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3859999999999999,"5min":1.417,"15min":1.417},"last":0.77300000000000002}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.63,"5min":0.625,"15min":0.625},"max":{"1min":1.2909999999999999,"5min":1.657,"15min":1.657},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2270000000000001,"5min":1.532,"15min":1.532},"last":0.82099999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:13:57 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.754,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.73399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.41099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67200000000000004,"5min":0.83499999999999996,"15min":0.85899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.52000000000000002},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.60899999999999999,"15min":0.54500000000000004},"min":{"1min":0.54700000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.59799999999999998,"15min":0.54200000000000004},"min":{"1min":0.51100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77900000000000003}]},{"osd":3,"last update":"Tue Oct 17 01:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.57499999999999996},"min":{"1min":0.46800000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.94699999999999995,"5min":1.02,"15min":1.02},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.65300000000000002,"15min":0.58499999999999996},"min":{"1min":0.501,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":0.745}]},{"osd":4,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47699999999999998,"15min":0.44},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89700000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.49199999999999999,"15min":0.46500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.67400000000000004,"15min":0.61199999999999999},"min":{"1min":0.57999999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98799999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.68700000000000006,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0309999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Oct 17 01:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.66800000000000004},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83099999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.58099999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.262},"last":0.90300000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.626},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.018,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81399999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.624,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.748,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.119,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.002}]},{"osd":9,"last update":"Tue Oct 17 01:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.75700000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84399999999999997,"15min":0.82999999999999996},"min":{"1min":0.64800000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.069,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0669999999999999}]},{"osd":10,"last update":"Tue Oct 17 01:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81000000000000005,"15min":0.80200000000000005},"min":{"1min":0.41999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.96899999999999997}]},{"osd":11,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.89100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 17 01:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.82099999999999995,"5min":1.01,"15min":1.02},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.53400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.89900000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":5.4029999999999996,"5min":5.4029999999999996,"15min":5.4029999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85899999999999999,"5min":1.117,"15min":1.117},"last":0.66700000000000004}]},{"osd":3,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Oct 17 01:14:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.65400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95499999999999996,"5min":1.115,"15min":1.115},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.60599999999999998},"min":{"1min":0.50900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.87,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72299999999999998,"15min":0.67700000000000005},"min":{"1min":0.56100000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94699999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.754},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.73099999999999998,"15min":0.65900000000000003},"min":{"1min":0.54400000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.746}]},{"osd":6,"last update":"Tue Oct 17 01:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.67300000000000004},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.194,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.84499999999999997,"15min":0.82899999999999996},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79100000000000004}]},{"osd":8,"last update":"Tue Oct 17 01:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96799999999999997,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89500000000000002,"15min":0.90500000000000003},"min":{"1min":0.66400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0169999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.83899999999999997}]},{"osd":9,"last update":"Tue Oct 17 01:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80400000000000005,"15min":0.83399999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.175,"5min":1.181,"15min":1.181},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.91900000000000004,"15min":0.95899999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2430000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Tue Oct 17 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.96299999999999997,"15min":0.98799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95699999999999996,"5min":1.506,"15min":1.506},"last":0.873},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1419999999999999,"5min":1.149,"15min":1.149},"last":0.85699999999999998}]},{"osd":11,"last update":"Tue Oct 17 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.96799999999999997},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.0529999999999999,"5min":1.373,"15min":1.373},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60899999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.90800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":502700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.39100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48499999999999999}]},{"osd":1,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.55900000000000005,"15min":0.51400000000000001},"min":{"1min":0.46100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 17 01:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76500000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Oct 17 01:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":2.6619999999999999,"5min":3.9950000000000001,"15min":3.9950000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72099999999999997,"15min":0.65900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.696,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Oct 17 01:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.054,"15min":1.054},"last":0.50900000000000001}]},{"osd":8,"last update":"Tue Oct 17 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66900000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61899999999999999,"15min":0.629},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72499999999999998}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92900000000000005,"5min":1.29,"15min":1.29},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":1.016,"15min":1.016},"last":0.80400000000000005}]},{"osd":10,"last update":"Tue Oct 17 01:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.97299999999999998,"15min":1.01},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.999,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.746}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":443643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.41499999999999998},"min":{"1min":0.35899999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.371},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.46500000000000002,"15min":0.41299999999999998},"min":{"1min":0.34399999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39400000000000002}]},{"osd":1,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68300000000000005,"15min":0.628},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.056,"5min":1.085,"15min":1.085},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68500000000000005,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Oct 17 01:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63600000000000001,"15min":0.59799999999999998},"min":{"1min":0.495,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.95099999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Oct 17 01:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Oct 17 01:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69299999999999995,"15min":0.65800000000000003},"min":{"1min":0.54200000000000004,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Oct 17 01:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.73499999999999999},"min":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82599999999999996,"15min":0.80900000000000005},"min":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.85899999999999999}]},{"osd":8,"last update":"Tue Oct 17 01:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.874,"15min":0.92000000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.234,"15min":1.234},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.83699999999999997,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86199999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.83399999999999996}]},{"osd":10,"last update":"Tue Oct 17 01:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0780000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.874},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.873,"15min":0.88200000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.65800000000000003}]},{"osd":11,"last update":"Tue Oct 17 01:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.89000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.52700000000000002,"15min":0.505},"min":{"1min":0.51000000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.47699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40799999999999997,"15min":0.35999999999999999},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.379},"min":{"1min":0.30599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505}]},{"osd":2,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76700000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76700000000000002}]},{"osd":3,"last update":"Tue Oct 17 01:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81799999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68899999999999995}]},{"osd":5,"last update":"Tue Oct 17 01:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.58799999999999997},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90400000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Oct 17 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Oct 17 01:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77100000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":1.044,"15min":1.044},"last":0.72199999999999998}]},{"osd":8,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.022,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89100000000000001}]},{"osd":9,"last update":"Tue Oct 17 01:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83199999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.79200000000000004,"15min":0.77700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.82299999999999995,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":0.82199999999999995}]},{"osd":10,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":1.089,"15min":1.089},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99199999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Tue Oct 17 01:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.97899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":1036763,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.754,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.432,"5min":0.309,"15min":0.309},"max":{"1min":0.81999999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.52500000000000002},"min":{"1min":0.47499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.74099999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":1.24,"15min":1.24},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.432,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.71999999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.39500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.51700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.45500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.88600000000000001}]},{"osd":6,"last update":"Tue Oct 17 01:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83999999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.61399999999999999},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Tue Oct 17 01:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88200000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.095,"15min":1.095},"last":0.81899999999999995}]},{"osd":8,"last update":"Tue Oct 17 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63,"15min":0.625},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.873},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.35999999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.83099999999999996}]},{"osd":9,"last update":"Tue Oct 17 01:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.88200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.60699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":0.89500000000000002,"5min":0.997,"15min":0.997},"last":0.84599999999999997}]},{"osd":10,"last update":"Tue Oct 17 01:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76600000000000001,"15min":0.76900000000000002},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90300000000000002,"5min":1.01,"15min":1.01},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.623,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.91300000000000003,"5min":1.081,"15min":1.081},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34269,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35911,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-17T01:14:47.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 -- 172.21.15.45:0/2499324921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 msgr2=0x7f25301064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 --2- 172.21.15.45:0/2499324921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f25301064d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2524009990 tx=0x7f2524009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 -- 172.21.15.45:0/2499324921 shutdown_connections 2023-10-17T01:14:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 --2- 172.21.15.45:0/2499324921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 0x7f2530108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 --2- 172.21.15.45:0/2499324921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25301017b0 0x7f2530103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 --2- 172.21.15.45:0/2499324921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f25301064d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 -- 172.21.15.45:0/2499324921 >> 172.21.15.45:0/2499324921 conn(0x7f25300fb180 msgr2=0x7f25300fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 -- 172.21.15.45:0/2499324921 shutdown_connections 2023-10-17T01:14:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.175+0000 7f25355ff700 1 -- 172.21.15.45:0/2499324921 wait complete. 2023-10-17T01:14:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.177+0000 7f25355ff700 1 Processor -- start 2023-10-17T01:14:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.177+0000 7f25355ff700 1 -- start start 2023-10-17T01:14:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.177+0000 7f25355ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25301017b0 0x7f2530198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.177+0000 7f25355ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f2530199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f25355ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 0x7f25301a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f25355ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25301a0780 con 0x7f2530106a10 2023-10-17T01:14:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f25355ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25301a08f0 con 0x7f25301040e0 2023-10-17T01:14:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f25355ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25301a0ab0 con 0x7f25301017b0 2023-10-17T01:14:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f2530199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f2530199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50026/0 (socket says 172.21.15.45:50026) 2023-10-17T01:14:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 -- 172.21.15.45:0/1339614174 learned_addr learned my addr 172.21.15.45:0/1339614174 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 0x7f25301a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252effd700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25301017b0 0x7f2530198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 -- 172.21.15.45:0/1339614174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25301017b0 msgr2=0x7f2530198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25301017b0 0x7f2530198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 -- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 msgr2=0x7f25301a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 0x7f25301a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 -- 172.21.15.45:0/1339614174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2528009c40 con 0x7f25301040e0 2023-10-17T01:14:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252f7fe700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 0x7f25301a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.178+0000 7f252e7fc700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f2530199180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2524004740 tx=0x7f2524009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.179+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252401d070 con 0x7f25301040e0 2023-10-17T01:14:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.179+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2524004830 con 0x7f25301040e0 2023-10-17T01:14:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.179+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240053c0 con 0x7f25301040e0 2023-10-17T01:14:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.179+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25240095f0 con 0x7f25301040e0 2023-10-17T01:14:48.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.179+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25301a0f10 con 0x7f25301040e0 2023-10-17T01:14:48.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.180+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25301085e0 con 0x7f25301040e0 2023-10-17T01:14:48.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.182+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2524005540 con 0x7f25301040e0 2023-10-17T01:14:48.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.183+0000 7f2517fff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25180a0bf0 0x7f25180a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.184+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f25240cd710 con 0x7f25301040e0 2023-10-17T01:14:48.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.184+0000 7f252effd700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25180a0bf0 0x7f25180a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.184+0000 7f252effd700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25180a0bf0 0x7f25180a30b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f253019a110 tx=0x7f2520009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.186+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2524097100 con 0x7f25301040e0 2023-10-17T01:14:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.404+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f253002ce40 con 0x7f25301040e0 2023-10-17T01:14:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.406+0000 7f2517fff700 1 -- 172.21.15.45:0/1339614174 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f252402f090 con 0x7f25301040e0 2023-10-17T01:14:48.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:48.407 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25180a0bf0 msgr2=0x7f25180a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25180a0bf0 0x7f25180a30b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f253019a110 tx=0x7f2520009260 comp rx=0 tx=0).stop 2023-10-17T01:14:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 msgr2=0x7f2530199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f2530199180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2524004740 tx=0x7f2524009580 comp rx=0 tx=0).stop 2023-10-17T01:14:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 shutdown_connections 2023-10-17T01:14:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25180a0bf0 0x7f25180a30b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2530106a10 0x7f25301a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25301017b0 0x7f2530198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 --2- 172.21.15.45:0/1339614174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25301040e0 0x7f2530199180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 >> 172.21.15.45:0/1339614174 conn(0x7f25300fb180 msgr2=0x7f25300ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 shutdown_connections 2023-10-17T01:14:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.409+0000 7f25355ff700 1 -- 172.21.15.45:0/1339614174 wait complete. 2023-10-17T01:14:48.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:14:48.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:14:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 -- 172.21.15.45:0/2220028331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 msgr2=0x7f9808101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 --2- 172.21.15.45:0/2220028331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808101160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f98040099c0 tx=0x7f9804009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 -- 172.21.15.45:0/2220028331 shutdown_connections 2023-10-17T01:14:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 --2- 172.21.15.45:0/2220028331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98081028d0 0x7f9808109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 --2- 172.21.15.45:0/2220028331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808101160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 --2- 172.21.15.45:0/2220028331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9808101f30 0x7f9808102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 -- 172.21.15.45:0/2220028331 >> 172.21.15.45:0/2220028331 conn(0x7f98080fc290 msgr2=0x7f98080fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.639+0000 7f980fdff700 1 -- 172.21.15.45:0/2220028331 shutdown_connections 2023-10-17T01:14:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.640+0000 7f980fdff700 1 -- 172.21.15.45:0/2220028331 wait complete. 2023-10-17T01:14:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.641+0000 7f980fdff700 1 Processor -- start 2023-10-17T01:14:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.641+0000 7f980fdff700 1 -- start start 2023-10-17T01:14:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980fdff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980fdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9808101f30 0x7f9808196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980fdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98081028d0 0x7f980819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980fdff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819e5e0 con 0x7f9808100d40 2023-10-17T01:14:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980fdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819e720 con 0x7f98081028d0 2023-10-17T01:14:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980fdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819e8b0 con 0x7f9808101f30 2023-10-17T01:14:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53606/0 (socket says 172.21.15.45:53606) 2023-10-17T01:14:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 -- 172.21.15.45:0/1608212634 learned_addr learned my addr 172.21.15.45:0/1608212634 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980d39a700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9808101f30 0x7f9808196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980e39c700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98081028d0 0x7f980819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 -- 172.21.15.45:0/1608212634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9808101f30 msgr2=0x7f9808196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9808101f30 0x7f9808196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 -- 172.21.15.45:0/1608212634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98081028d0 msgr2=0x7f980819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98081028d0 0x7f980819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980db9b700 1 -- 172.21.15.45:0/1608212634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040095f0 con 0x7f9808100d40 2023-10-17T01:14:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980d39a700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9808101f30 0x7f9808196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:14:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.642+0000 7f980e39c700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98081028d0 0x7f980819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:14:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.643+0000 7f980db9b700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808196a60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f98040098e0 tx=0x7f980400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.643+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980401d070 con 0x7f9808100d40 2023-10-17T01:14:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.643+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9804005510 con 0x7f9808100d40 2023-10-17T01:14:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.643+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980819eb30 con 0x7f9808100d40 2023-10-17T01:14:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.643+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804004d00 con 0x7f9808100d40 2023-10-17T01:14:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.643+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980819efb0 con 0x7f9808100d40 2023-10-17T01:14:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.644+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980810c5a0 con 0x7f9808100d40 2023-10-17T01:14:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.646+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f980400fc30 con 0x7f9808100d40 2023-10-17T01:14:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.648+0000 7f97feffd700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97f40a0ba0 0x7f97f40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.648+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f98040cd030 con 0x7f9808100d40 2023-10-17T01:14:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.648+0000 7f980d39a700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97f40a0ba0 0x7f97f40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.648+0000 7f980d39a700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97f40a0ba0 0x7f97f40a3060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009be0 tx=0x7f97f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.651+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f9804096a80 con 0x7f9808100d40 2023-10-17T01:14:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:48 smithi045 ceph-mon[98474]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:48 smithi045 ceph-mon[98474]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:48.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:48 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1339614174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:48 smithi060 ceph-mon[134136]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:48 smithi060 ceph-mon[134136]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:48.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:48 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1339614174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.864+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f980819f560 con 0x7f9808100d40 2023-10-17T01:14:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.865+0000 7f97feffd700 1 -- 172.21.15.45:0/1608212634 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f980409a0a0 con 0x7f9808100d40 2023-10-17T01:14:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:14:48.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97f40a0ba0 msgr2=0x7f97f40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97f40a0ba0 0x7f97f40a3060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009be0 tx=0x7f97f8009260 comp rx=0 tx=0).stop 2023-10-17T01:14:48.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 msgr2=0x7f9808196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:48.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808196a60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f98040098e0 tx=0x7f980400f740 comp rx=0 tx=0).stop 2023-10-17T01:14:48.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 shutdown_connections 2023-10-17T01:14:48.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97f40a0ba0 0x7f97f40a3060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98081028d0 0x7f980819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.867+0000 7f980fdff700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9808100d40 0x7f9808196a60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.868+0000 7f980fdff700 1 --2- 172.21.15.45:0/1608212634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9808101f30 0x7f9808196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:48.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.868+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 >> 172.21.15.45:0/1608212634 conn(0x7f98080fc290 msgr2=0x7f98080fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:48.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.868+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 shutdown_connections 2023-10-17T01:14:48.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:48.868+0000 7f980fdff700 1 -- 172.21.15.45:0/1608212634 wait complete. 2023-10-17T01:14:48.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:14:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:48 smithi181 ceph-mon[134520]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:48 smithi181 ceph-mon[134520]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:48.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:48 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1339614174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.095+0000 7f8fd87ff700 1 -- 172.21.15.45:0/764344993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101f30 msgr2=0x7f8fd0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.095+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/764344993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101f30 0x7f8fd0102390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00099c0 tx=0x7f8fc0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 -- 172.21.15.45:0/764344993 shutdown_connections 2023-10-17T01:14:49.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/764344993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd01028d0 0x7f8fd0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/764344993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd0100d40 0x7f8fd0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/764344993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0101f30 0x7f8fd0102390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 -- 172.21.15.45:0/764344993 >> 172.21.15.45:0/764344993 conn(0x7f8fd00fc290 msgr2=0x7f8fd00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:49.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 -- 172.21.15.45:0/764344993 shutdown_connections 2023-10-17T01:14:49.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.096+0000 7f8fd87ff700 1 -- 172.21.15.45:0/764344993 wait complete. 2023-10-17T01:14:49.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 Processor -- start 2023-10-17T01:14:49.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 -- start start 2023-10-17T01:14:49.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 0x7f8fd0074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0101f30 0x7f8fd0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd01028d0 0x7f8fd0073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0073e20 con 0x7f8fd0100d40 2023-10-17T01:14:49.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0073f90 con 0x7f8fd0101f30 2023-10-17T01:14:49.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd87ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01a0af0 con 0x7f8fd01028d0 2023-10-17T01:14:49.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd659b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 0x7f8fd0074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd659b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 0x7f8fd0074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53630/0 (socket says 172.21.15.45:53630) 2023-10-17T01:14:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd659b700 1 -- 172.21.15.45:0/2561335477 learned_addr learned my addr 172.21.15.45:0/2561335477 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd6d9c700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd01028d0 0x7f8fd0073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.097+0000 7f8fd5d9a700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0101f30 0x7f8fd0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd659b700 1 -- 172.21.15.45:0/2561335477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd01028d0 msgr2=0x7f8fd0073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd659b700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd01028d0 0x7f8fd0073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd659b700 1 -- 172.21.15.45:0/2561335477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0101f30 msgr2=0x7f8fd0073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd659b700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0101f30 0x7f8fd0073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd659b700 1 -- 172.21.15.45:0/2561335477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00095f0 con 0x7f8fd0100d40 2023-10-17T01:14:49.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd659b700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 0x7f8fd0074b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00e970 tx=0x7f8fcc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc009790 con 0x7f8fd0100d40 2023-10-17T01:14:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd01a0dd0 con 0x7f8fd0100d40 2023-10-17T01:14:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fcc00c7e0 con 0x7f8fd0100d40 2023-10-17T01:14:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.098+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc003c70 con 0x7f8fd0100d40 2023-10-17T01:14:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.099+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01a1220 con 0x7f8fd0100d40 2023-10-17T01:14:49.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.099+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd010c5a0 con 0x7f8fd0100d40 2023-10-17T01:14:49.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.101+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8fcc00cc50 con 0x7f8fd0100d40 2023-10-17T01:14:49.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.103+0000 7f8fc77fe700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0b20 0x7f8fbc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.103+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8fcc014070 con 0x7f8fd0100d40 2023-10-17T01:14:49.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.103+0000 7f8fd5d9a700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0b20 0x7f8fbc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.104+0000 7f8fd5d9a700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0b20 0x7f8fbc0a2fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b580 tx=0x7f8fc0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.106+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f8fcc095ce0 con 0x7f8fd0100d40 2023-10-17T01:14:49.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.318+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fd0107cc0 con 0x7f8fd0100d40 2023-10-17T01:14:49.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.318+0000 7f8fc77fe700 1 -- 172.21.15.45:0/2561335477 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fcc003dd0 con 0x7f8fd0100d40 2023-10-17T01:14:49.319 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0b20 msgr2=0x7f8fbc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0b20 0x7f8fbc0a2fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b580 tx=0x7f8fc0005fb0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 msgr2=0x7f8fd0074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 0x7f8fd0074b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00e970 tx=0x7f8fcc00ec80 comp rx=0 tx=0).stop 2023-10-17T01:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 shutdown_connections 2023-10-17T01:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8fbc0a0b20 0x7f8fbc0a2fe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd01028d0 0x7f8fd0073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0100d40 0x7f8fd0074b00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 --2- 172.21.15.45:0/2561335477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fd0101f30 0x7f8fd0073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 >> 172.21.15.45:0/2561335477 conn(0x7f8fd00fc290 msgr2=0x7f8fd00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:49.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 shutdown_connections 2023-10-17T01:14:49.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.321+0000 7f8fd87ff700 1 -- 172.21.15.45:0/2561335477 wait complete. 2023-10-17T01:14:49.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:14:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 -- 172.21.15.45:0/3616399605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 msgr2=0x7f2f10101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 --2- 172.21.15.45:0/3616399605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10101160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009920 tx=0x7f2f0c009c30 comp rx=0 tx=0).stop 2023-10-17T01:14:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 -- 172.21.15.45:0/3616399605 shutdown_connections 2023-10-17T01:14:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 --2- 172.21.15.45:0/3616399605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f101028d0 0x7f2f10109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 --2- 172.21.15.45:0/3616399605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10101160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 --2- 172.21.15.45:0/3616399605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f10101f30 0x7f2f10102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 -- 172.21.15.45:0/3616399605 >> 172.21.15.45:0/3616399605 conn(0x7f2f100fc290 msgr2=0x7f2f100fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 -- 172.21.15.45:0/3616399605 shutdown_connections 2023-10-17T01:14:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.551+0000 7f2f17fff700 1 -- 172.21.15.45:0/3616399605 wait complete. 2023-10-17T01:14:49.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 Processor -- start 2023-10-17T01:14:49.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 -- start start 2023-10-17T01:14:49.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f10101f30 0x7f2f101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f101028d0 0x7f2f101a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101a08e0 con 0x7f2f10101f30 2023-10-17T01:14:49.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101a0a20 con 0x7f2f10100d40 2023-10-17T01:14:49.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.552+0000 7f2f17fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101a0bb0 con 0x7f2f101028d0 2023-10-17T01:14:49.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50078/0 (socket says 172.21.15.45:50078) 2023-10-17T01:14:49.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 -- 172.21.15.45:0/356402130 learned_addr learned my addr 172.21.15.45:0/356402130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:49.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f1559a700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f10101f30 0x7f2f101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f1659c700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f101028d0 0x7f2f101a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 -- 172.21.15.45:0/356402130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f101028d0 msgr2=0x7f2f101a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f101028d0 0x7f2f101a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 -- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f10101f30 msgr2=0x7f2f101992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f10101f30 0x7f2f101992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 -- 172.21.15.45:0/356402130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08009d00 con 0x7f2f10100d40 2023-10-17T01:14:49.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f1559a700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f10101f30 0x7f2f101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:49.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.553+0000 7f2f15d9b700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10198d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009230 tx=0x7f2f0c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:49.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.554+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c01c070 con 0x7f2f10100d40 2023-10-17T01:14:49.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.554+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f0c00f7d0 con 0x7f2f10100d40 2023-10-17T01:14:49.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.554+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c009520 con 0x7f2f10100d40 2023-10-17T01:14:49.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.554+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004930 con 0x7f2f10100d40 2023-10-17T01:14:49.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.554+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f101a10c0 con 0x7f2f10100d40 2023-10-17T01:14:49.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.555+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f10193390 con 0x7f2f10100d40 2023-10-17T01:14:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.557+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2f0c020070 con 0x7f2f10100d40 2023-10-17T01:14:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.558+0000 7f2f06ffd700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2efc0a0ba0 0x7f2efc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.558+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2f0c0cd500 con 0x7f2f10100d40 2023-10-17T01:14:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.558+0000 7f2f1559a700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2efc0a0ba0 0x7f2efc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.559+0000 7f2f1559a700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2efc0a0ba0 0x7f2efc0a3060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f00011490 tx=0x7f2f00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.560+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f2f0c096ef0 con 0x7f2f10100d40 2023-10-17T01:14:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.771+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f1002cdc0 con 0x7f2f10100d40 2023-10-17T01:14:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.772+0000 7f2f06ffd700 1 -- 172.21.15.45:0/356402130 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f0c09a510 con 0x7f2f10100d40 2023-10-17T01:14:49.773 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:14:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2efc0a0ba0 msgr2=0x7f2efc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2efc0a0ba0 0x7f2efc0a3060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f00011490 tx=0x7f2f00009260 comp rx=0 tx=0).stop 2023-10-17T01:14:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 msgr2=0x7f2f10198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10198d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009230 tx=0x7f2f0c00b790 comp rx=0 tx=0).stop 2023-10-17T01:14:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 shutdown_connections 2023-10-17T01:14:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2efc0a0ba0 0x7f2efc0a3060 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f101028d0 0x7f2f101a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f10100d40 0x7f2f10198d60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 --2- 172.21.15.45:0/356402130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f10101f30 0x7f2f101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 >> 172.21.15.45:0/356402130 conn(0x7f2f100fc290 msgr2=0x7f2f10107950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 shutdown_connections 2023-10-17T01:14:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.775+0000 7f2f17fff700 1 -- 172.21.15.45:0/356402130 wait complete. 2023-10-17T01:14:49.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:49 smithi045 ceph-mon[98474]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:49.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:49 smithi045 ceph-mon[98474]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-10-17T01:14:49.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:49 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1608212634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:49.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:49 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2561335477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:49.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:14:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:49 smithi060 ceph-mon[134136]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:49 smithi060 ceph-mon[134136]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-10-17T01:14:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:49 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1608212634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:49.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:49 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2561335477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:49 smithi181 ceph-mon[134520]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-17T01:14:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:49 smithi181 ceph-mon[134520]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-10-17T01:14:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:49 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1608212634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:14:49.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:49 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2561335477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:14:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.999+0000 7f97e93ff700 1 -- 172.21.15.45:0/1683172632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4100d50 msgr2=0x7f97e4101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:49.999+0000 7f97e93ff700 1 --2- 172.21.15.45:0/1683172632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4100d50 0x7f97e4101170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97d40098f0 tx=0x7f97d4009c00 comp rx=0 tx=0).stop 2023-10-17T01:14:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 -- 172.21.15.45:0/1683172632 shutdown_connections 2023-10-17T01:14:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 --2- 172.21.15.45:0/1683172632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e4109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 --2- 172.21.15.45:0/1683172632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4100d50 0x7f97e4101170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 --2- 172.21.15.45:0/1683172632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97e4101f40 0x7f97e41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 -- 172.21.15.45:0/1683172632 >> 172.21.15.45:0/1683172632 conn(0x7f97e40fc280 msgr2=0x7f97e40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 -- 172.21.15.45:0/1683172632 shutdown_connections 2023-10-17T01:14:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.000+0000 7f97e93ff700 1 -- 172.21.15.45:0/1683172632 wait complete. 2023-10-17T01:14:50.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 Processor -- start 2023-10-17T01:14:50.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 -- start start 2023-10-17T01:14:50.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97e4100d50 0x7f97e4198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4101f40 0x7f97e4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e41a0830 con 0x7f97e41028e0 2023-10-17T01:14:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e41a09a0 con 0x7f97e4100d50 2023-10-17T01:14:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e93ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e41a0b60 con 0x7f97e4101f40 2023-10-17T01:14:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e41a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53664/0 (socket says 172.21.15.45:53664) 2023-10-17T01:14:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.001+0000 7f97e37fe700 1 -- 172.21.15.45:0/2818434101 learned_addr learned my addr 172.21.15.45:0/2818434101 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e27fc700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4101f40 0x7f97e4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e2ffd700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97e4100d50 0x7f97e4198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e37fe700 1 -- 172.21.15.45:0/2818434101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4101f40 msgr2=0x7f97e4199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e37fe700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4101f40 0x7f97e4199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e37fe700 1 -- 172.21.15.45:0/2818434101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97e4100d50 msgr2=0x7f97e4198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e37fe700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97e4100d50 0x7f97e4198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e37fe700 1 -- 172.21.15.45:0/2818434101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d4009520 con 0x7f97e41028e0 2023-10-17T01:14:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e37fe700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e41a0260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00ef30 tx=0x7f97dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc018070 con 0x7f97e41028e0 2023-10-17T01:14:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e41a0de0 con 0x7f97e41028e0 2023-10-17T01:14:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.002+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc00f040 con 0x7f97e41028e0 2023-10-17T01:14:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.003+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00ccf0 con 0x7f97e41028e0 2023-10-17T01:14:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.003+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e41a1260 con 0x7f97e41028e0 2023-10-17T01:14:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.003+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e410c5b0 con 0x7f97e41028e0 2023-10-17T01:14:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.006+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f97dc003a00 con 0x7f97e41028e0 2023-10-17T01:14:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.007+0000 7f97cbfff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97cc0a0bf0 0x7f97cc0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.007+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f97dc0cc620 con 0x7f97e41028e0 2023-10-17T01:14:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.007+0000 7f97e2ffd700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97cc0a0bf0 0x7f97cc0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.008+0000 7f97e2ffd700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97cc0a0bf0 0x7f97cc0a30b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009fd0 tx=0x7f97d4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.009+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f97dc096010 con 0x7f97e41028e0 2023-10-17T01:14:50.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.266+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97e41a18a0 con 0x7f97e41028e0 2023-10-17T01:14:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.266+0000 7f97cbfff700 1 -- 172.21.15.45:0/2818434101 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f97dc099630 con 0x7f97e41028e0 2023-10-17T01:14:50.267 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:50.267 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:13:08.449854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:14:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97cc0a0bf0 msgr2=0x7f97cc0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97cc0a0bf0 0x7f97cc0a30b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009fd0 tx=0x7f97d4019040 comp rx=0 tx=0).stop 2023-10-17T01:14:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 msgr2=0x7f97e41a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e41a0260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00ef30 tx=0x7f97dc00c3a0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 shutdown_connections 2023-10-17T01:14:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97cc0a0bf0 0x7f97cc0a30b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97e41028e0 0x7f97e41a0260 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97e4100d50 0x7f97e4198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 --2- 172.21.15.45:0/2818434101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e4101f40 0x7f97e4199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 >> 172.21.15.45:0/2818434101 conn(0x7f97e40fc280 msgr2=0x7f97e40fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 shutdown_connections 2023-10-17T01:14:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.269+0000 7f97e93ff700 1 -- 172.21.15.45:0/2818434101 wait complete. 2023-10-17T01:14:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-17T01:14:50.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:14:50.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 -- 172.21.15.45:0/1375731013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c0737b0 msgr2=0x7f353c073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 --2- 172.21.15.45:0/1375731013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c0737b0 0x7f353c073c30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3530009990 tx=0x7f3530009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 -- 172.21.15.45:0/1375731013 shutdown_connections 2023-10-17T01:14:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 --2- 172.21.15.45:0/1375731013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f353c108c10 0x7f353c10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 --2- 172.21.15.45:0/1375731013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f353c074d90 0x7f353c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 --2- 172.21.15.45:0/1375731013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c0737b0 0x7f353c073c30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 -- 172.21.15.45:0/1375731013 >> 172.21.15.45:0/1375731013 conn(0x7f353c0fc280 msgr2=0x7f353c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 -- 172.21.15.45:0/1375731013 shutdown_connections 2023-10-17T01:14:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.494+0000 7f35419ff700 1 -- 172.21.15.45:0/1375731013 wait complete. 2023-10-17T01:14:50.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.495+0000 7f35419ff700 1 Processor -- start 2023-10-17T01:14:50.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 -- start start 2023-10-17T01:14:50.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f353c0737b0 0x7f353c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 0x7f353c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f353c108c10 0x7f353c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c1a0770 con 0x7f353c074d90 2023-10-17T01:14:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c1a08e0 con 0x7f353c0737b0 2023-10-17T01:14:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f35419ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c1a0aa0 con 0x7f353c108c10 2023-10-17T01:14:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f353a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 0x7f353c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f353a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 0x7f353c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53672/0 (socket says 172.21.15.45:53672) 2023-10-17T01:14:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f353a7fc700 1 -- 172.21.15.45:0/3898288379 learned_addr learned my addr 172.21.15.45:0/3898288379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f353affd700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f353c0737b0 0x7f353c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.496+0000 7f353b7fe700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f353c108c10 0x7f353c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f353a7fc700 1 -- 172.21.15.45:0/3898288379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f353c108c10 msgr2=0x7f353c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f353a7fc700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f353c108c10 0x7f353c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f353a7fc700 1 -- 172.21.15.45:0/3898288379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f353c0737b0 msgr2=0x7f353c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f353a7fc700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f353c0737b0 0x7f353c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f353a7fc700 1 -- 172.21.15.45:0/3898288379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534009c40 con 0x7f353c074d90 2023-10-17T01:14:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f353a7fc700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 0x7f353c199170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3530004710 tx=0x7f3530009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353001d070 con 0x7f353c074d90 2023-10-17T01:14:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3530021410 con 0x7f353c074d90 2023-10-17T01:14:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35300095f0 con 0x7f353c074d90 2023-10-17T01:14:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530017650 con 0x7f353c074d90 2023-10-17T01:14:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.497+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353c1a1010 con 0x7f353c074d90 2023-10-17T01:14:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.498+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353c1072c0 con 0x7f353c074d90 2023-10-17T01:14:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.500+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f3530021580 con 0x7f353c074d90 2023-10-17T01:14:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.502+0000 7f35409fd700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35240a0bf0 0x7f35240a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.502+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f35300cdb70 con 0x7f353c074d90 2023-10-17T01:14:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.502+0000 7f353affd700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35240a0bf0 0x7f35240a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.503+0000 7f353affd700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35240a0bf0 0x7f35240a30b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f353c19a100 tx=0x7f352c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.504+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3530097560 con 0x7f353c074d90 2023-10-17T01:14:50.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.720+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f353c10e870 con 0x7f353c074d90 2023-10-17T01:14:50.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.721+0000 7f35409fd700 1 -- 172.21.15.45:0/3898288379 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f353009ab80 con 0x7f353c074d90 2023-10-17T01:14:50.722 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:50.722 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:14:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35240a0bf0 msgr2=0x7f35240a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35240a0bf0 0x7f35240a30b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f353c19a100 tx=0x7f352c009260 comp rx=0 tx=0).stop 2023-10-17T01:14:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 msgr2=0x7f353c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 0x7f353c199170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3530004710 tx=0x7f3530009580 comp rx=0 tx=0).stop 2023-10-17T01:14:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 shutdown_connections 2023-10-17T01:14:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35240a0bf0 0x7f35240a30b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f353c108c10 0x7f353c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f353c0737b0 0x7f353c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 --2- 172.21.15.45:0/3898288379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353c074d90 0x7f353c199170 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 >> 172.21.15.45:0/3898288379 conn(0x7f353c0fc280 msgr2=0x7f353c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 shutdown_connections 2023-10-17T01:14:50.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.724+0000 7f35419ff700 1 -- 172.21.15.45:0/3898288379 wait complete. 2023-10-17T01:14:50.743 DEBUG:tasks.fwd_scrub:Yielding 2023-10-17T01:14:50.744 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-17T01:14:50.754 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:14:50.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-17T01:14:50.755 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-17T01:14:50.755 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-17T01:14:50.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-17T01:14:50.760 INFO:teuthology.run_tasks:Running task exec... 2023-10-17T01:14:50.769 INFO:teuthology.task.exec:Executing custom commands... 2023-10-17T01:14:50.770 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:14:50.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-10-17T01:14:50.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:50 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/356402130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:50.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:50 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2818434101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:50.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:50 smithi045 ceph-mon[98474]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-17T01:14:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:50 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/356402130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:50 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2818434101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:50 smithi060 ceph-mon[134136]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-17T01:14:50.873 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:50.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:50 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/356402130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:14:50.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:50 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2818434101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:14:50.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:50 smithi181 ceph-mon[134520]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-17T01:14:50.921 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem Size Used Avail Use% Mounted on 2023-10-17T01:14:50.921 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:3300,172.21.15.60:3300,172.21.15.181:3300:/volumes/_nogroup/sv_1/c18bfcb3-8f74-497f-901a-c6d13f726359 340G 0 340G 0% /tmp/cephfs 2023-10-17T01:14:50.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-10-17T01:14:50.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 -- 172.21.15.45:0/2510325269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 msgr2=0x7fa82c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 --2- 172.21.15.45:0/2510325269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c101160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009990 tx=0x7fa81c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 -- 172.21.15.45:0/2510325269 shutdown_connections 2023-10-17T01:14:50.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 --2- 172.21.15.45:0/2510325269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c1028d0 0x7fa82c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 --2- 172.21.15.45:0/2510325269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c101160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 --2- 172.21.15.45:0/2510325269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c101f30 0x7fa82c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 -- 172.21.15.45:0/2510325269 >> 172.21.15.45:0/2510325269 conn(0x7fa82c0fc290 msgr2=0x7fa82c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:50.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 -- 172.21.15.45:0/2510325269 shutdown_connections 2023-10-17T01:14:50.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.979+0000 7fa8313ff700 1 -- 172.21.15.45:0/2510325269 wait complete. 2023-10-17T01:14:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.980+0000 7fa8313ff700 1 Processor -- start 2023-10-17T01:14:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.980+0000 7fa8313ff700 1 -- start start 2023-10-17T01:14:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 0x7fa82c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1028d0 0x7fa82c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa8313ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c19e640 con 0x7fa82c101f30 2023-10-17T01:14:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa8313ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c19e780 con 0x7fa82c1028d0 2023-10-17T01:14:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa8313ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c19e910 con 0x7fa82c100d40 2023-10-17T01:14:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 0x7fa82c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 0x7fa82c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53688/0 (socket says 172.21.15.45:53688) 2023-10-17T01:14:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 -- 172.21.15.45:0/3071465311 learned_addr learned my addr 172.21.15.45:0/3071465311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82affd700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1028d0 0x7fa82c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 -- 172.21.15.45:0/3071465311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 msgr2=0x7fa82c196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 -- 172.21.15.45:0/3071465311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1028d0 msgr2=0x7fa82c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.981+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1028d0 0x7fa82c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa82a7fc700 1 -- 172.21.15.45:0/3071465311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0095f0 con 0x7fa82c101f30 2023-10-17T01:14:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa82affd700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 0x7fa82c197000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa82000e9a0 tx=0x7fa82000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa813fff700 1 -- 172.21.15.45:0/3071465311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820009790 con 0x7fa82c101f30 2023-10-17T01:14:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c19ebf0 con 0x7fa82c101f30 2023-10-17T01:14:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa813fff700 1 -- 172.21.15.45:0/3071465311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa820010430 con 0x7fa82c101f30 2023-10-17T01:14:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c19f150 con 0x7fa82c101f30 2023-10-17T01:14:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.982+0000 7fa813fff700 1 -- 172.21.15.45:0/3071465311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820018620 con 0x7fa82c101f30 2023-10-17T01:14:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.987+0000 7fa813fff700 1 -- 172.21.15.45:0/3071465311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa8200105a0 con 0x7fa82c101f30 2023-10-17T01:14:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.988+0000 7fa844da3b80 1 Processor -- start 2023-10-17T01:14:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.988+0000 7fa813fff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa8140a0b20 0x7fa8140a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.988+0000 7fa844da3b80 1 -- start start 2023-10-17T01:14:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.988+0000 7fa813fff700 1 -- 172.21.15.45:0/3071465311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa820014070 con 0x7fa82c101f30 2023-10-17T01:14:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa8140a0b20 0x7fa8140a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x561342734040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x561342734da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x5613427357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa844da3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134268ba90 con 0x561342734580 2023-10-17T01:14:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa844da3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561342689450 con 0x561342735330 2023-10-17T01:14:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa844da3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561342684d40 con 0x561342733c20 2023-10-17T01:14:50.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa82affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x561342734040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa82affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x561342734040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34348/0 (socket says 172.21.15.45:34348) 2023-10-17T01:14:50.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa82affd700 1 -- 172.21.15.45:0/3626341209 learned_addr learned my addr 172.21.15.45:0/3626341209 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:50.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x5613427357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.989+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x561342734da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa8140a0b20 0x7fa8140a2fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa82400f540 tx=0x7fa824009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82affd700 1 -- 172.21.15.45:0/3626341209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 msgr2=0x5613427357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82affd700 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x5613427357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82affd700 1 -- 172.21.15.45:0/3626341209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 msgr2=0x561342734da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82affd700 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x561342734da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82affd700 1 -- 172.21.15.45:0/3626341209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613424b2e60 con 0x561342733c20 2023-10-17T01:14:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa82affd700 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x561342734040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa82c101d90 tx=0x7fa81c005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b603b3b5820a9adb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa8117fa700 1 -- 172.21.15.45:0/3626341209 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c01d070 con 0x561342733c20 2023-10-17T01:14:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa8117fa700 1 -- 172.21.15.45:0/3626341209 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c01e040 con 0x561342733c20 2023-10-17T01:14:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.990+0000 7fa8117fa700 1 -- 172.21.15.45:0/3626341209 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004e00 con 0x561342733c20 2023-10-17T01:14:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 -- 172.21.15.45:0/3626341209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 msgr2=0x561342734040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x561342734040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa82c101d90 tx=0x7fa81c005ee0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 -- 172.21.15.45:0/3626341209 shutdown_connections 2023-10-17T01:14:50.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x5613427357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x561342734040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3626341209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x561342734da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:50.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 -- 172.21.15.45:0/3626341209 >> 172.21.15.45:0/3626341209 conn(0x561342693150 msgr2=0x56134272e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:50.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 -- 172.21.15.45:0/3626341209 shutdown_connections 2023-10-17T01:14:51.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.991+0000 7fa844da3b80 1 -- 172.21.15.45:0/3626341209 wait complete. 2023-10-17T01:14:51.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 -- start start 2023-10-17T01:14:51.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 Processor -- start 2023-10-17T01:14:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342734580 0x5613427421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x561342749210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561342621590 con 0x561342733c20 2023-10-17T01:14:51.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561342621230 con 0x561342735330 2023-10-17T01:14:51.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.992+0000 7fa844da3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613425089a0 con 0x561342734580 2023-10-17T01:14:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa844da3b80 10 client.0 _ll_register_callbacks cb 0x56134256eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53708/0 (socket says 172.21.15.45:53708) 2023-10-17T01:14:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 -- 172.21.15.45:0/3324964955 learned_addr learned my addr 172.21.15.45:0/3324964955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342734580 0x5613427421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x561342749210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 -- 172.21.15.45:0/3324964955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342734580 msgr2=0x5613427421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342734580 0x5613427421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 -- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 msgr2=0x561342749210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x561342749210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82affd700 1 -- 172.21.15.45:0/3324964955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c109ad0 con 0x561342733c20 2023-10-17T01:14:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.993+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342734580 0x5613427421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa82affd700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741c90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009990 tx=0x7fa81c022b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8223ee7f81708064 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa80eff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa80eff5700 1 -- 172.21.15.45:0/3324964955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c01d030 con 0x561342733c20 2023-10-17T01:14:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa80eff5700 1 -- 172.21.15.45:0/3324964955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa81c004500 con 0x561342733c20 2023-10-17T01:14:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613426afd90 con 0x561342733c20 2023-10-17T01:14:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa80eff5700 1 -- 172.21.15.45:0/3324964955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c004800 con 0x561342733c20 2023-10-17T01:14:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa844da3b80 10 client.15354 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.994+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56134249bce0 con 0x561342733c20 2023-10-17T01:14:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa80eff5700 1 -- 172.21.15.45:0/3324964955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613426afd90 con 0x561342733c20 2023-10-17T01:14:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa80eff5700 1 -- 172.21.15.45:0/3324964955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa81c02c660 con 0x561342733c20 2023-10-17T01:14:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa844da3b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa844da3b80 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:14:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa844da3b80 20 client.15354 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134273fa10 0x56134273fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa844da3b80 4 client.15354 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561342702db0 con 0x56134273fa10 2023-10-17T01:14:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.995+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134273fa10 0x56134273fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.996+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134273fa10 0x56134273fe20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e442b0d01f5156 server_cookie=b4fb77e07f0b4f3a in_seq=0 out_seq=0 2023-10-17T01:14:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.996+0000 7fa80eff5700 10 client.15354 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.996+0000 7fa80eff5700 1 -- 172.21.15.45:0/3324964955 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x561342702db0 con 0x56134273fa10 2023-10-17T01:14:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa80eff5700 10 client.15354 handle_command_reply: tid=0 2023-10-17T01:14:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 client.15354 shutdown 2023-10-17T01:14:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 msgr2=0x561342741c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741c90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009990 tx=0x7fa81c022b00 comp rx=0 tx=0).stop 2023-10-17T01:14:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 shutdown_connections 2023-10-17T01:14:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134273fa10 0x56134273fe20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x561342749210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741c90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3324964955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342734580 0x5613427421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa80eff5700 0 client.15354 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 >> 172.21.15.45:0/3324964955 conn(0x561342693150 msgr2=0x56134272e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 shutdown_connections 2023-10-17T01:14:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 1 -- 172.21.15.45:0/3324964955 wait complete. 2023-10-17T01:14:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:50.997+0000 7fa844da3b80 20 client.15354 trim_cache size 0 max 16384 2023-10-17T01:14:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.015+0000 7fa844da3b80 1 Processor -- start 2023-10-17T01:14:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.015+0000 7fa844da3b80 1 -- start start 2023-10-17T01:14:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.015+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342734580 0x561342741f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342735330 0x56134272e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa844da3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134272b4a0 con 0x561342733c20 2023-10-17T01:14:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa844da3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613427300c0 con 0x561342734580 2023-10-17T01:14:51.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa844da3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134273cd00 con 0x561342735330 2023-10-17T01:14:51.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342735330 0x56134272e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53722/0 (socket says 172.21.15.45:53722) 2023-10-17T01:14:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 -- 172.21.15.45:0/3958141335 learned_addr learned my addr 172.21.15.45:0/3958141335 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342734580 0x561342741f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 -- 172.21.15.45:0/3958141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342735330 msgr2=0x56134272e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342735330 0x56134272e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 -- 172.21.15.45:0/3958141335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342734580 msgr2=0x561342741f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342734580 0x561342741f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82affd700 1 -- 172.21.15.45:0/3958141335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613424d92d0 con 0x561342733c20 2023-10-17T01:14:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.016+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342735330 0x56134272e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.017+0000 7fa82affd700 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000c00 tx=0x7fa81c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df18f387c1d872a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.017+0000 7fa811ffb700 1 -- 172.21.15.45:0/3958141335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c01d070 con 0x561342733c20 2023-10-17T01:14:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.017+0000 7fa811ffb700 1 -- 172.21.15.45:0/3958141335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c022c20 con 0x561342733c20 2023-10-17T01:14:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.017+0000 7fa811ffb700 1 -- 172.21.15.45:0/3958141335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00e640 con 0x561342733c20 2023-10-17T01:14:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 -- 172.21.15.45:0/3958141335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 msgr2=0x561342741a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000c00 tx=0x7fa81c009310 comp rx=0 tx=0).stop 2023-10-17T01:14:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 -- 172.21.15.45:0/3958141335 shutdown_connections 2023-10-17T01:14:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342735330 0x56134272e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342733c20 0x561342741a40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 --2- 172.21.15.45:0/3958141335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342734580 0x561342741f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 -- 172.21.15.45:0/3958141335 >> 172.21.15.45:0/3958141335 conn(0x561342693150 msgr2=0x5613427381a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 -- 172.21.15.45:0/3958141335 shutdown_connections 2023-10-17T01:14:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 -- 172.21.15.45:0/3958141335 wait complete. 2023-10-17T01:14:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.018+0000 7fa844da3b80 1 -- start start 2023-10-17T01:14:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 Processor -- start 2023-10-17T01:14:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x56134274cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x56134274d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x56134274e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134274e560 con 0x561342734580 2023-10-17T01:14:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134274e6d0 con 0x561342735330 2023-10-17T01:14:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134274e890 con 0x561342733c20 2023-10-17T01:14:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa844da3b80 10 client.0 _ll_register_callbacks cb 0x7fa820095ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa82a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x56134274d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa82a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x56134274d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53726/0 (socket says 172.21.15.45:53726) 2023-10-17T01:14:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa82a7fc700 1 -- 172.21.15.45:0/986420810 learned_addr learned my addr 172.21.15.45:0/986420810 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa82affd700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x56134274cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.019+0000 7fa82b7fe700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x56134274e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa82a7fc700 1 -- 172.21.15.45:0/986420810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 msgr2=0x56134274cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x56134274cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa82a7fc700 1 -- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 msgr2=0x56134274e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x56134274e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa82a7fc700 1 -- 172.21.15.45:0/986420810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa824027b80 con 0x561342734580 2023-10-17T01:14:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa82a7fc700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x56134274d010 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa820096340 tx=0x7fa8200b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c83d531b436a28b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa80eff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa80eff5700 1 -- 172.21.15.45:0/986420810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200a1990 con 0x561342734580 2023-10-17T01:14:51.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa80eff5700 1 -- 172.21.15.45:0/986420810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa8200b90b0 con 0x561342734580 2023-10-17T01:14:51.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa81c01e040 con 0x561342734580 2023-10-17T01:14:51.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa80eff5700 1 -- 172.21.15.45:0/986420810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa820010880 con 0x561342734580 2023-10-17T01:14:51.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa844da3b80 10 client.15360 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:51.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.020+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56134249bce0 con 0x561342734580 2023-10-17T01:14:51.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa80eff5700 1 -- 172.21.15.45:0/986420810 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82007c020 con 0x561342734580 2023-10-17T01:14:51.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa80eff5700 1 -- 172.21.15.45:0/986420810 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa820098880 con 0x561342734580 2023-10-17T01:14:51.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa844da3b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:51.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa844da3b80 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:14:51.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa844da3b80 20 client.15360 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:51.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa844da3b80 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134279ca50 0x56134279ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa844da3b80 4 client.15360 mds_command: new command op to 24587 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-17T01:14:51.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56134267ddc0 con 0x56134279ca50 2023-10-17T01:14:51.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.021+0000 7fa82affd700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134279ca50 0x56134279ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.022+0000 7fa82affd700 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134279ca50 0x56134279ee70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0bbfe6e89c2100 server_cookie=cacdf48083a9cf29 in_seq=0 out_seq=0 2023-10-17T01:14:51.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.022+0000 7fa80eff5700 10 client.15360 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.023+0000 7fa80eff5700 1 -- 172.21.15.45:0/986420810 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56134267ddc0 con 0x56134279ca50 2023-10-17T01:14:51.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.023+0000 7fa80eff5700 10 client.15360 handle_command_reply: tid=0 2023-10-17T01:14:51.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.023+0000 7fa844da3b80 1 client.15360 shutdown 2023-10-17T01:14:51.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 msgr2=0x56134274d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x56134274d010 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa820096340 tx=0x7fa8200b9040 comp rx=0 tx=0).stop 2023-10-17T01:14:51.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 shutdown_connections 2023-10-17T01:14:51.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56134279ca50 0x56134279ee70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561342735330 0x56134274e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561342733c20 0x56134274cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 --2- 172.21.15.45:0/986420810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561342734580 0x56134274d010 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa820096340 tx=0x7fa8200b9040 comp rx=0 tx=0).stop 2023-10-17T01:14:51.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 >> 172.21.15.45:0/986420810 conn(0x561342693150 msgr2=0x561342741a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa80eff5700 0 client.15360 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 shutdown_connections 2023-10-17T01:14:51.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 1 -- 172.21.15.45:0/986420810 wait complete. 2023-10-17T01:14:51.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.024+0000 7fa844da3b80 20 client.15360 trim_cache size 0 max 16384 2023-10-17T01:14:51.040 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:51.040 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2023-10-17T01:14:51.040 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "e7200ecc-f164-40e1-a484-d767d4a8042a", 2023-10-17T01:14:51.040 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2023-10-17T01:14:51.040 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa8140a0b20 msgr2=0x7fa8140a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa8140a0b20 0x7fa8140a2fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa82400f540 tx=0x7fa824009920 comp rx=0 tx=0).stop 2023-10-17T01:14:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 msgr2=0x7fa82c197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 0x7fa82c197000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa82000e9a0 tx=0x7fa82000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 shutdown_connections 2023-10-17T01:14:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa8140a0b20 0x7fa8140a2fe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1028d0 0x7fa82c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c100d40 0x7fa82c196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 --2- 172.21.15.45:0/3071465311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c101f30 0x7fa82c197000 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 >> 172.21.15.45:0/3071465311 conn(0x7fa82c0fc290 msgr2=0x7fa82c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 shutdown_connections 2023-10-17T01:14:51.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.025+0000 7fa8313ff700 1 -- 172.21.15.45:0/3071465311 wait complete. 2023-10-17T01:14:51.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:14:51.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-10-17T01:14:51.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-10-17T01:14:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 -- 172.21.15.45:0/16017485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 msgr2=0x7fce38102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 --2- 172.21.15.45:0/16017485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce38102390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0099c0 tx=0x7fce2c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 -- 172.21.15.45:0/16017485 shutdown_connections 2023-10-17T01:14:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 --2- 172.21.15.45:0/16017485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce381028d0 0x7fce38109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 --2- 172.21.15.45:0/16017485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce38100d40 0x7fce38101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 --2- 172.21.15.45:0/16017485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce38102390 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 -- 172.21.15.45:0/16017485 >> 172.21.15.45:0/16017485 conn(0x7fce380fc290 msgr2=0x7fce380fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 -- 172.21.15.45:0/16017485 shutdown_connections 2023-10-17T01:14:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.257+0000 7fce3dfff700 1 -- 172.21.15.45:0/16017485 wait complete. 2023-10-17T01:14:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.258+0000 7fce3dfff700 1 Processor -- start 2023-10-17T01:14:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 -- start start 2023-10-17T01:14:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce38100d40 0x7fce38198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce381991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce381028d0 0x7fce381a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce381a0800 con 0x7fce38101f30 2023-10-17T01:14:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce381a0970 con 0x7fce381028d0 2023-10-17T01:14:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce3dfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce381a0b30 con 0x7fce38100d40 2023-10-17T01:14:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce36ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce381991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce36ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce381991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53742/0 (socket says 172.21.15.45:53742) 2023-10-17T01:14:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce36ffd700 1 -- 172.21.15.45:0/720920154 learned_addr learned my addr 172.21.15.45:0/720920154 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.259+0000 7fce377fe700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce38100d40 0x7fce38198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce37fff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce381028d0 0x7fce381a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce36ffd700 1 -- 172.21.15.45:0/720920154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce38100d40 msgr2=0x7fce38198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce36ffd700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce38100d40 0x7fce38198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce36ffd700 1 -- 172.21.15.45:0/720920154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce381028d0 msgr2=0x7fce381a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce36ffd700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce381028d0 0x7fce381a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce36ffd700 1 -- 172.21.15.45:0/720920154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0095f0 con 0x7fce38101f30 2023-10-17T01:14:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce37fff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce381028d0 0x7fce381a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce36ffd700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce381991d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009fd0 tx=0x7fce2c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce34ff9700 1 -- 172.21.15.45:0/720920154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c01d070 con 0x7fce38101f30 2023-10-17T01:14:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce34ff9700 1 -- 172.21.15.45:0/720920154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce2c005510 con 0x7fce38101f30 2023-10-17T01:14:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce381a0db0 con 0x7fce38101f30 2023-10-17T01:14:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.260+0000 7fce34ff9700 1 -- 172.21.15.45:0/720920154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c004ae0 con 0x7fce38101f30 2023-10-17T01:14:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.261+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce381a11d0 con 0x7fce38101f30 2023-10-17T01:14:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.264+0000 7fce34ff9700 1 -- 172.21.15.45:0/720920154 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fce2c004cc0 con 0x7fce38101f30 2023-10-17T01:14:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 Processor -- start 2023-10-17T01:14:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 -- start start 2023-10-17T01:14:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 0x561aed9cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9ce500 0x561aed9ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cf2b0 0x561aed9cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed925a10 con 0x561aed9cf2b0 2023-10-17T01:14:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed9233d0 con 0x561aed9cdba0 2023-10-17T01:14:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.265+0000 7fce51895b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed91ecc0 con 0x561aed9ce500 2023-10-17T01:14:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 0x561aed9cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 0x561aed9cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50208/0 (socket says 172.21.15.45:50208) 2023-10-17T01:14:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 -- 172.21.15.45:0/2896963297 learned_addr learned my addr 172.21.15.45:0/2896963297 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce377fe700 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9ce500 0x561aed9ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce36ffd700 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cf2b0 0x561aed9cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 -- 172.21.15.45:0/2896963297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9ce500 msgr2=0x561aed9ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9ce500 0x561aed9ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 -- 172.21.15.45:0/2896963297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cf2b0 msgr2=0x561aed9cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cf2b0 0x561aed9cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 -- 172.21.15.45:0/2896963297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aed74cde0 con 0x561aed9cdba0 2023-10-17T01:14:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce377fe700 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9ce500 0x561aed9ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:14:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.266+0000 7fce37fff700 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 0x561aed9cdfc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce30009440 tx=0x7fce30009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec4b248ac1962ea server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.267+0000 7fce34ff9700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce200a0b20 0x7fce200a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.267+0000 7fce1dffb700 1 -- 172.21.15.45:0/2896963297 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000e070 con 0x561aed9cdba0 2023-10-17T01:14:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.267+0000 7fce1dffb700 1 -- 172.21.15.45:0/2896963297 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3000f040 con 0x561aed9cdba0 2023-10-17T01:14:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.267+0000 7fce1dffb700 1 -- 172.21.15.45:0/2896963297 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30014a70 con 0x561aed9cdba0 2023-10-17T01:14:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.267+0000 7fce34ff9700 1 -- 172.21.15.45:0/720920154 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce2c0ccfe0 con 0x7fce38101f30 2023-10-17T01:14:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.267+0000 7fce377fe700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce200a0b20 0x7fce200a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 -- 172.21.15.45:0/2896963297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 msgr2=0x561aed9cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce377fe700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce200a0b20 0x7fce200a2fe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x561aed9ce900 tx=0x7fce28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 0x561aed9cdfc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce30009440 tx=0x7fce30009920 comp rx=0 tx=0).stop 2023-10-17T01:14:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 -- 172.21.15.45:0/2896963297 shutdown_connections 2023-10-17T01:14:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cf2b0 0x561aed9cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9cdba0 0x561aed9cdfc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 --2- 172.21.15.45:0/2896963297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9ce500 0x561aed9ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 -- 172.21.15.45:0/2896963297 >> 172.21.15.45:0/2896963297 conn(0x561aed92d0d0 msgr2=0x561aed9c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 -- 172.21.15.45:0/2896963297 shutdown_connections 2023-10-17T01:14:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 -- 172.21.15.45:0/2896963297 wait complete. 2023-10-17T01:14:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.268+0000 7fce51895b80 1 -- start start 2023-10-17T01:14:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 Processor -- start 2023-10-17T01:14:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9dee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed8bb510 con 0x561aed9cdba0 2023-10-17T01:14:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed8bb1b0 con 0x561aed9ce500 2023-10-17T01:14:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed7a2920 con 0x561aed9cf2b0 2023-10-17T01:14:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce51895b80 10 client.0 _ll_register_callbacks cb 0x561aed808a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53754/0 (socket says 172.21.15.45:53754) 2023-10-17T01:14:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce377fe700 1 -- 172.21.15.45:0/274090824 learned_addr learned my addr 172.21.15.45:0/274090824 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.269+0000 7fce36ffd700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce37fff700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9dee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce36ffd700 1 -- 172.21.15.45:0/274090824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 msgr2=0x561aed9dee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce36ffd700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9dee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce36ffd700 1 -- 172.21.15.45:0/274090824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 msgr2=0x561aed9d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce36ffd700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce36ffd700 1 -- 172.21.15.45:0/274090824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300090f0 con 0x561aed9ce500 2023-10-17T01:14:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce37fff700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9dee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce36ffd700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9d7e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0cab60 tx=0x7fce2c00fe60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a79fd283bceedda4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.270+0000 7fce16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:14:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce16ffd700 1 -- 172.21.15.45:0/274090824 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0cf310 con 0x561aed9ce500 2023-10-17T01:14:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aed949d10 con 0x561aed9ce500 2023-10-17T01:14:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce16ffd700 1 -- 172.21.15.45:0/274090824 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce2c005680 con 0x561aed9ce500 2023-10-17T01:14:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce16ffd700 1 -- 172.21.15.45:0/274090824 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce2c09a230 con 0x561aed9ce500 2023-10-17T01:14:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce51895b80 10 client.25388 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561aed735c60 con 0x561aed9ce500 2023-10-17T01:14:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce16ffd700 1 -- 172.21.15.45:0/274090824 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561aed949d10 con 0x561aed9ce500 2023-10-17T01:14:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce16ffd700 1 -- 172.21.15.45:0/274090824 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fce2c09e150 con 0x561aed9ce500 2023-10-17T01:14:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce51895b80 10 client.25388 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce51895b80 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:14:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.271+0000 7fce51895b80 20 client.25388 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.272+0000 7fce51895b80 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aed9e3320 0x561aed9e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.272+0000 7fce51895b80 4 client.25388 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:14:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.272+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aed99cd30 con 0x561aed9e3320 2023-10-17T01:14:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.272+0000 7fce377fe700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aed9e3320 0x561aed9e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.272+0000 7fce377fe700 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aed9e3320 0x561aed9e5710 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbccbf504f18f0b8 server_cookie=fd89f2d761284eff in_seq=0 out_seq=0 2023-10-17T01:14:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.272+0000 7fce16ffd700 10 client.25388 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.273+0000 7fce16ffd700 1 -- 172.21.15.45:0/274090824 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x561aed99cd30 con 0x561aed9e3320 2023-10-17T01:14:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.273+0000 7fce16ffd700 10 client.25388 handle_command_reply: tid=0 2023-10-17T01:14:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.273+0000 7fce51895b80 1 client.25388 shutdown 2023-10-17T01:14:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.273+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 msgr2=0x561aed9d7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.273+0000 7fce51895b80 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9d7e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0cab60 tx=0x7fce2c00fe60 comp rx=0 tx=0).stop 2023-10-17T01:14:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 shutdown_connections 2023-10-17T01:14:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9dee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aed9e3320 0x561aed9e5710 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 --2- 172.21.15.45:0/274090824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9d7e50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 >> 172.21.15.45:0/274090824 conn(0x561aed92d0d0 msgr2=0x561aed9c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce16ffd700 0 client.25388 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 shutdown_connections 2023-10-17T01:14:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 1 -- 172.21.15.45:0/274090824 wait complete. 2023-10-17T01:14:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.274+0000 7fce51895b80 20 client.25388 trim_cache size 0 max 16384 2023-10-17T01:14:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.292+0000 7fce51895b80 1 Processor -- start 2023-10-17T01:14:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 -- start start 2023-10-17T01:14:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed9c06a0 con 0x561aed9cdba0 2023-10-17T01:14:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed9d5550 con 0x561aed9ce500 2023-10-17T01:14:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce51895b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aed9d6ad0 con 0x561aed9cf2b0 2023-10-17T01:14:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53758/0 (socket says 172.21.15.45:53758) 2023-10-17T01:14:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce377fe700 1 -- 172.21.15.45:0/1719592466 learned_addr learned my addr 172.21.15.45:0/1719592466 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce36ffd700 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.293+0000 7fce37fff700 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce36ffd700 1 -- 172.21.15.45:0/1719592466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 msgr2=0x561aed9d57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce36ffd700 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9d57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce36ffd700 1 -- 172.21.15.45:0/1719592466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 msgr2=0x561aed9d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce36ffd700 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce36ffd700 1 -- 172.21.15.45:0/1719592466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aed773250 con 0x561aed9ce500 2023-10-17T01:14:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce377fe700 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:14:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce36ffd700 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9c8020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00bf40 tx=0x7fce2c0cd330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3404cce87e3eed5d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce1e7fc700 1 -- 172.21.15.45:0/1719592466 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c09a230 con 0x561aed9ce500 2023-10-17T01:14:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce1e7fc700 1 -- 172.21.15.45:0/1719592466 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce2c09a050 con 0x561aed9ce500 2023-10-17T01:14:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.294+0000 7fce1e7fc700 1 -- 172.21.15.45:0/1719592466 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c099c30 con 0x561aed9ce500 2023-10-17T01:14:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 -- 172.21.15.45:0/1719592466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 msgr2=0x561aed9c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9c8020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00bf40 tx=0x7fce2c0cd330 comp rx=0 tx=0).stop 2023-10-17T01:14:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 -- 172.21.15.45:0/1719592466 shutdown_connections 2023-10-17T01:14:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aed9d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aed9d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 --2- 172.21.15.45:0/1719592466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aed9c8020 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 -- 172.21.15.45:0/1719592466 >> 172.21.15.45:0/1719592466 conn(0x561aed92d0d0 msgr2=0x561aed9c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 -- 172.21.15.45:0/1719592466 shutdown_connections 2023-10-17T01:14:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.295+0000 7fce51895b80 1 -- 172.21.15.45:0/1719592466 wait complete. 2023-10-17T01:14:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 -- start start 2023-10-17T01:14:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 Processor -- start 2023-10-17T01:14:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aeda2d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aeda2d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aeda34720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeda34c60 con 0x561aed9cdba0 2023-10-17T01:14:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeda34da0 con 0x561aed9ce500 2023-10-17T01:14:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeda34f30 con 0x561aed9cf2b0 2023-10-17T01:14:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.296+0000 7fce51895b80 10 client.0 _ll_register_callbacks cb 0x7fce2c0d0590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:14:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aeda2d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aeda2d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53762/0 (socket says 172.21.15.45:53762) 2023-10-17T01:14:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 -- 172.21.15.45:0/3731483079 learned_addr learned my addr 172.21.15.45:0/3731483079 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:14:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce36ffd700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aeda2d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce37fff700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aeda34720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 -- 172.21.15.45:0/3731483079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 msgr2=0x561aeda34720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aeda34720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 -- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 msgr2=0x561aeda2d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:14:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aeda2d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 -- 172.21.15.45:0/3731483079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce28012e00 con 0x561aed9cdba0 2023-10-17T01:14:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce377fe700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aeda2d1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fce2802c960 tx=0x7fce28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92186ccabaea959 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:14:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.297+0000 7fce16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:14:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce16ffd700 1 -- 172.21.15.45:0/3731483079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28022040 con 0x561aed9cdba0 2023-10-17T01:14:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce3000f040 con 0x561aed9cdba0 2023-10-17T01:14:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce16ffd700 1 -- 172.21.15.45:0/3731483079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce28004830 con 0x561aed9cdba0 2023-10-17T01:14:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce16ffd700 1 -- 172.21.15.45:0/3731483079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce28017070 con 0x561aed9cdba0 2023-10-17T01:14:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce51895b80 10 client.15378 fetch_fsmap learned FSMap version 26 2023-10-17T01:14:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561aed735c60 con 0x561aed9cdba0 2023-10-17T01:14:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce16ffd700 1 -- 172.21.15.45:0/3731483079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3000f040 con 0x561aed9cdba0 2023-10-17T01:14:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce16ffd700 1 -- 172.21.15.45:0/3731483079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fce2801d550 con 0x561aed9cdba0 2023-10-17T01:14:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.298+0000 7fce51895b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:14:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce51895b80 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:14:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce51895b80 20 client.15378 populate_metadata read hostname 'smithi045' 2023-10-17T01:14:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce51895b80 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aeda369d0 0x561aeda38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:14:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce51895b80 4 client.15378 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:14:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561aed917d40 con 0x561aeda369d0 2023-10-17T01:14:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce36ffd700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aeda369d0 0x561aeda38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:14:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce36ffd700 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aeda369d0 0x561aeda38df0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2927537f0518cd39 server_cookie=94a9a10fc67622a9 in_seq=0 out_seq=0 2023-10-17T01:14:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.299+0000 7fce16ffd700 10 client.15378 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce16ffd700 1 -- 172.21.15.45:0/3731483079 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561aed917d40 con 0x561aeda369d0 2023-10-17T01:14:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce16ffd700 10 client.15378 handle_command_reply: tid=0 2023-10-17T01:14:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 client.15378 shutdown 2023-10-17T01:14:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 msgr2=0x561aeda2d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aeda2d1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fce2802c960 tx=0x7fce28017000 comp rx=0 tx=0).stop 2023-10-17T01:14:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 shutdown_connections 2023-10-17T01:14:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aed9cf2b0 0x561aeda34720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aed9cdba0 0x561aeda2d1a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aeda369d0 0x561aeda38df0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 --2- 172.21.15.45:0/3731483079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aed9ce500 0x561aeda2d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce16ffd700 0 client.15378 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:14:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 >> 172.21.15.45:0/3731483079 conn(0x561aed92d0d0 msgr2=0x561aed9d2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 shutdown_connections 2023-10-17T01:14:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 1 -- 172.21.15.45:0/3731483079 wait complete. 2023-10-17T01:14:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.300+0000 7fce51895b80 20 client.15378 trim_cache size 0 max 16384 2023-10-17T01:14:51.314 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:14:51.314 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:14:51.314 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:14:51.314 INFO:teuthology.orchestra.run.smithi045.stdout: "e7200ecc-f164-40e1-a484-d767d4a8042a": { 2023-10-17T01:14:51.314 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:14:51.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "e7200ecc-f164-40e1-a484-d767d4a8042a", 2023-10-17T01:14:51.315 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:14:51.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:14:51.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:14:51.316 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:14:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce200a0b20 msgr2=0x7fce200a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce200a0b20 0x7fce200a2fe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x561aed9ce900 tx=0x7fce28009260 comp rx=0 tx=0).stop 2023-10-17T01:14:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 msgr2=0x7fce381991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:14:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce381991d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009fd0 tx=0x7fce2c00f690 comp rx=0 tx=0).stop 2023-10-17T01:14:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 shutdown_connections 2023-10-17T01:14:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce200a0b20 0x7fce200a2fe0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce381028d0 0x7fce381a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce38100d40 0x7fce38198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 --2- 172.21.15.45:0/720920154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce38101f30 0x7fce381991d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:14:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 >> 172.21.15.45:0/720920154 conn(0x7fce380fc290 msgr2=0x7fce380fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:14:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 shutdown_connections 2023-10-17T01:14:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:14:51.301+0000 7fce3dfff700 1 -- 172.21.15.45:0/720920154 wait complete. 2023-10-17T01:14:51.320 INFO:tasks.cephfs.filesystem:scrub status for tag:e7200ecc-f164-40e1-a484-d767d4a8042a - {'path': '/', 'tag': 'e7200ecc-f164-40e1-a484-d767d4a8042a', 'options': 'recursive,force'} 2023-10-17T01:14:51.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:51 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3898288379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:51.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:51 smithi045 ceph-mon[98474]: scrub queued for path: / 2023-10-17T01:14:51.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:51 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:14:51.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:51 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:14:51.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:51 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:14:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:51 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3898288379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:51 smithi060 ceph-mon[134136]: scrub queued for path: / 2023-10-17T01:14:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:51 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:14:51.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:51 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:14:51.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:51 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:14:51.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:51 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3898288379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:14:51.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:51 smithi181 ceph-mon[134520]: scrub queued for path: / 2023-10-17T01:14:51.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:51 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:14:51.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:51 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:14:51.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:51 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:14:52.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:52 smithi060 ceph-mon[134136]: pgmap v325: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-10-17T01:14:52.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:52 smithi181 ceph-mon[134520]: pgmap v325: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-10-17T01:14:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:52 smithi045 ceph-mon[98474]: pgmap v325: 97 pgs: 97 active+clean; 651 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-10-17T01:14:54.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-10-17T01:14:54.681 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-10-17T01:14:54.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-10-17T01:14:54.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-17T01:14:54.895 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/: 2023-10-17T01:14:54.895 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:54.896 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:54.896 INFO:teuthology.orchestra.run.smithi045.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 .. 2023-10-17T01:14:54.896 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 data 2023-10-17T01:14:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:54.896 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data: 2023-10-17T01:14:54.897 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:54.897 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:54.897 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 .. 2023-10-17T01:14:54.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-10-17T01:14:55.024 INFO:teuthology.orchestra.run.smithi045.stdout:[Service] 2023-10-17T01:14:55.024 INFO:teuthology.orchestra.run.smithi045.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-10-17T01:14:55.025 INFO:teuthology.orchestra.run.smithi045.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-10-17T01:14:55.029 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-10-17T01:14:55.228 INFO:teuthology.orchestra.run.smithi045.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-10-17T01:14:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:55 smithi045 ceph-mon[98474]: pgmap v326: 97 pgs: 97 active+clean; 652 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-10-17T01:14:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:55 smithi045 ceph-mon[98474]: scrub summary: idle 2023-10-17T01:14:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:55 smithi060 ceph-mon[134136]: pgmap v326: 97 pgs: 97 active+clean; 652 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-10-17T01:14:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:55 smithi060 ceph-mon[134136]: scrub summary: idle 2023-10-17T01:14:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:55 smithi181 ceph-mon[134520]: pgmap v326: 97 pgs: 97 active+clean; 652 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-10-17T01:14:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:55 smithi181 ceph-mon[134520]: scrub summary: idle 2023-10-17T01:14:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:57 smithi045 ceph-mon[98474]: pgmap v327: 97 pgs: 97 active+clean; 652 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-10-17T01:14:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:57 smithi060 ceph-mon[134136]: pgmap v327: 97 pgs: 97 active+clean; 652 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-10-17T01:14:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:57 smithi181 ceph-mon[134520]: pgmap v327: 97 pgs: 97 active+clean; 652 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-10-17T01:14:58.730 INFO:teuthology.orchestra.run.smithi045.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-10-17T01:14:58.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-17T01:14:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/: 2023-10-17T01:14:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:58.852 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:58.852 INFO:teuthology.orchestra.run.smithi045.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 .. 2023-10-17T01:14:58.852 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 data 2023-10-17T01:14:58.852 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:58.852 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data: 2023-10-17T01:14:58.852 INFO:teuthology.orchestra.run.smithi045.stdout:total 30 2023-10-17T01:14:58.853 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 . 2023-10-17T01:14:58.853 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 .. 2023-10-17T01:14:58.853 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 base 2023-10-17T01:14:58.853 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 17 01:14 global 2023-10-17T01:14:58.853 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 log 2023-10-17T01:14:58.854 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_commit_ts 2023-10-17T01:14:58.854 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_dynshmem 2023-10-17T01:14:58.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 17 01:14 pg_hba.conf 2023-10-17T01:14:58.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 17 01:14 pg_ident.conf 2023-10-17T01:14:58.854 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 pg_logical 2023-10-17T01:14:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 pg_multixact 2023-10-17T01:14:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 pg_notify 2023-10-17T01:14:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_replslot 2023-10-17T01:14:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_serial 2023-10-17T01:14:58.856 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_snapshots 2023-10-17T01:14:58.856 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_stat 2023-10-17T01:14:58.856 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_stat_tmp 2023-10-17T01:14:58.856 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 pg_subtrans 2023-10-17T01:14:58.857 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_tblspc 2023-10-17T01:14:58.857 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_twophase 2023-10-17T01:14:58.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:14:58.857 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 pg_wal 2023-10-17T01:14:58.857 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 pg_xact 2023-10-17T01:14:58.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 17 01:14 postgresql.auto.conf 2023-10-17T01:14:58.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 17 01:14 postgresql.conf 2023-10-17T01:14:58.858 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:58.858 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base: 2023-10-17T01:14:58.858 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:58.859 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 . 2023-10-17T01:14:58.859 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:58.859 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 1 2023-10-17T01:14:58.859 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 12346 2023-10-17T01:14:58.859 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 12347 2023-10-17T01:14:58.860 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:58.860 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-17T01:14:58.860 INFO:teuthology.orchestra.run.smithi045.stdout:total 7185 2023-10-17T01:14:58.860 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 . 2023-10-17T01:14:58.860 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 .. 2023-10-17T01:14:58.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 112 2023-10-17T01:14:58.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 113 2023-10-17T01:14:58.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 17 01:14 12182 2023-10-17T01:14:58.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12182_fsm 2023-10-17T01:14:58.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12182_vm 2023-10-17T01:14:58.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12184 2023-10-17T01:14:58.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12186 2023-10-17T01:14:58.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187 2023-10-17T01:14:58.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12187_fsm 2023-10-17T01:14:58.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187_vm 2023-10-17T01:14:58.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12189 2023-10-17T01:14:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12191 2023-10-17T01:14:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192 2023-10-17T01:14:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12192_fsm 2023-10-17T01:14:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192_vm 2023-10-17T01:14:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12194 2023-10-17T01:14:58.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12196 2023-10-17T01:14:58.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197 2023-10-17T01:14:58.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12197_fsm 2023-10-17T01:14:58.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197_vm 2023-10-17T01:14:58.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12199 2023-10-17T01:14:58.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12201 2023-10-17T01:14:58.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202 2023-10-17T01:14:58.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12202_fsm 2023-10-17T01:14:58.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202_vm 2023-10-17T01:14:58.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12204 2023-10-17T01:14:58.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12206 2023-10-17T01:14:58.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207 2023-10-17T01:14:58.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12207_fsm 2023-10-17T01:14:58.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207_vm 2023-10-17T01:14:58.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12209 2023-10-17T01:14:58.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12211 2023-10-17T01:14:58.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12212 2023-10-17T01:14:58.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12214 2023-10-17T01:14:58.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12216 2023-10-17T01:14:58.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 17 01:14 1247 2023-10-17T01:14:58.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1247_fsm 2023-10-17T01:14:58.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1247_vm 2023-10-17T01:14:58.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 17 01:14 1249 2023-10-17T01:14:58.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1249_fsm 2023-10-17T01:14:58.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1249_vm 2023-10-17T01:14:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 17 01:14 1255 2023-10-17T01:14:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1255_fsm 2023-10-17T01:14:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1255_vm 2023-10-17T01:14:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:14 1259 2023-10-17T01:14:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1259_fsm 2023-10-17T01:14:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1259_vm 2023-10-17T01:14:58.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417 2023-10-17T01:14:58.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417_vm 2023-10-17T01:14:58.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418 2023-10-17T01:14:58.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418_vm 2023-10-17T01:14:58.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 174 2023-10-17T01:14:58.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 175 2023-10-17T01:14:58.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2187 2023-10-17T01:14:58.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224 2023-10-17T01:14:58.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224_vm 2023-10-17T01:14:58.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328 2023-10-17T01:14:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328_vm 2023-10-17T01:14:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336 2023-10-17T01:14:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336_vm 2023-10-17T01:14:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2337 2023-10-17T01:14:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2600 2023-10-17T01:14:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2600_fsm 2023-10-17T01:14:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2600_vm 2023-10-17T01:14:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601 2023-10-17T01:14:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2601_fsm 2023-10-17T01:14:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601_vm 2023-10-17T01:14:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2602 2023-10-17T01:14:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2602_fsm 2023-10-17T01:14:58.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2602_vm 2023-10-17T01:14:58.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2603 2023-10-17T01:14:58.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2603_fsm 2023-10-17T01:14:58.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2603_vm 2023-10-17T01:14:58.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604 2023-10-17T01:14:58.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604_vm 2023-10-17T01:14:58.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2605 2023-10-17T01:14:58.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2605_fsm 2023-10-17T01:14:58.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2605_vm 2023-10-17T01:14:58.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606 2023-10-17T01:14:58.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2606_fsm 2023-10-17T01:14:58.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606_vm 2023-10-17T01:14:58.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607 2023-10-17T01:14:58.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607_fsm 2023-10-17T01:14:58.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2607_vm 2023-10-17T01:14:58.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 17 01:14 2608 2023-10-17T01:14:58.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2608_fsm 2023-10-17T01:14:58.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2608_vm 2023-10-17T01:14:58.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 17 01:14 2609 2023-10-17T01:14:58.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2609_fsm 2023-10-17T01:14:58.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2609_vm 2023-10-17T01:14:58.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610 2023-10-17T01:14:58.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610_fsm 2023-10-17T01:14:58.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2610_vm 2023-10-17T01:14:58.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611 2023-10-17T01:14:58.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611_vm 2023-10-17T01:14:58.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612 2023-10-17T01:14:58.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2612_fsm 2023-10-17T01:14:58.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612_vm 2023-10-17T01:14:58.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613 2023-10-17T01:14:58.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613_vm 2023-10-17T01:14:58.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615 2023-10-17T01:14:58.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2615_fsm 2023-10-17T01:14:58.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615_vm 2023-10-17T01:14:58.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616 2023-10-17T01:14:58.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616_fsm 2023-10-17T01:14:58.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2616_vm 2023-10-17T01:14:58.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 17 01:14 2617 2023-10-17T01:14:58.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2617_fsm 2023-10-17T01:14:58.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2617_vm 2023-10-17T01:14:58.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 17 01:14 2618 2023-10-17T01:14:58.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2618_fsm 2023-10-17T01:14:58.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2618_vm 2023-10-17T01:14:58.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 17 01:14 2619 2023-10-17T01:14:58.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2619_fsm 2023-10-17T01:14:58.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2619_vm 2023-10-17T01:14:58.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620 2023-10-17T01:14:58.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620_vm 2023-10-17T01:14:58.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2650 2023-10-17T01:14:58.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2651 2023-10-17T01:14:58.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2652 2023-10-17T01:14:58.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2653 2023-10-17T01:14:58.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2654 2023-10-17T01:14:58.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2655 2023-10-17T01:14:58.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2656 2023-10-17T01:14:58.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2657 2023-10-17T01:14:58.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 17 01:14 2658 2023-10-17T01:14:58.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2659 2023-10-17T01:14:58.885 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2660 2023-10-17T01:14:58.885 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2661 2023-10-17T01:14:58.885 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2662 2023-10-17T01:14:58.885 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2663 2023-10-17T01:14:58.885 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2664 2023-10-17T01:14:58.886 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2665 2023-10-17T01:14:58.886 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2666 2023-10-17T01:14:58.886 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2667 2023-10-17T01:14:58.886 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2668 2023-10-17T01:14:58.886 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2669 2023-10-17T01:14:58.887 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2670 2023-10-17T01:14:58.887 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 17 01:14 2673 2023-10-17T01:14:58.887 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 17 01:14 2674 2023-10-17T01:14:58.887 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 17 01:14 2675 2023-10-17T01:14:58.888 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2678 2023-10-17T01:14:58.888 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2679 2023-10-17T01:14:58.888 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2680 2023-10-17T01:14:58.888 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2681 2023-10-17T01:14:58.888 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2682 2023-10-17T01:14:58.889 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2683 2023-10-17T01:14:58.889 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2684 2023-10-17T01:14:58.889 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2685 2023-10-17T01:14:58.889 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2686 2023-10-17T01:14:58.889 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2687 2023-10-17T01:14:58.889 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2688 2023-10-17T01:14:58.890 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2689 2023-10-17T01:14:58.890 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2690 2023-10-17T01:14:58.890 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 17 01:14 2691 2023-10-17T01:14:58.890 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2692 2023-10-17T01:14:58.891 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2693 2023-10-17T01:14:58.891 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2696 2023-10-17T01:14:58.891 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2699 2023-10-17T01:14:58.891 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2701 2023-10-17T01:14:58.891 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2702 2023-10-17T01:14:58.892 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2703 2023-10-17T01:14:58.892 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2704 2023-10-17T01:14:58.892 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2753 2023-10-17T01:14:58.892 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2753_fsm 2023-10-17T01:14:58.892 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2753_vm 2023-10-17T01:14:58.893 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2754 2023-10-17T01:14:58.893 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2755 2023-10-17T01:14:58.893 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2756 2023-10-17T01:14:58.893 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2757 2023-10-17T01:14:58.894 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830 2023-10-17T01:14:58.894 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830_vm 2023-10-17T01:14:58.894 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2831 2023-10-17T01:14:58.894 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832 2023-10-17T01:14:58.894 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832_vm 2023-10-17T01:14:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2833 2023-10-17T01:14:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834 2023-10-17T01:14:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834_vm 2023-10-17T01:14:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2835 2023-10-17T01:14:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836 2023-10-17T01:14:58.896 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836_vm 2023-10-17T01:14:58.896 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2837 2023-10-17T01:14:58.896 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 17 01:14 2838 2023-10-17T01:14:58.896 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2838_fsm 2023-10-17T01:14:58.896 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2838_vm 2023-10-17T01:14:58.897 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2839 2023-10-17T01:14:58.897 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840 2023-10-17T01:14:58.897 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840_fsm 2023-10-17T01:14:58.897 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2840_vm 2023-10-17T01:14:58.898 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2841 2023-10-17T01:14:58.898 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995 2023-10-17T01:14:58.898 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995_vm 2023-10-17T01:14:58.898 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2996 2023-10-17T01:14:58.898 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079 2023-10-17T01:14:58.899 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3079_fsm 2023-10-17T01:14:58.899 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079_vm 2023-10-17T01:14:58.899 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3080 2023-10-17T01:14:58.899 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3081 2023-10-17T01:14:58.899 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3085 2023-10-17T01:14:58.900 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118 2023-10-17T01:14:58.900 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118_vm 2023-10-17T01:14:58.900 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3119 2023-10-17T01:14:58.900 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3164 2023-10-17T01:14:58.900 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256 2023-10-17T01:14:58.900 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256_vm 2023-10-17T01:14:58.901 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3257 2023-10-17T01:14:58.901 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3258 2023-10-17T01:14:58.901 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350 2023-10-17T01:14:58.901 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350_vm 2023-10-17T01:14:58.901 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3351 2023-10-17T01:14:58.902 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3379 2023-10-17T01:14:58.902 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3380 2023-10-17T01:14:58.902 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381 2023-10-17T01:14:58.902 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381_vm 2023-10-17T01:14:58.902 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3394 2023-10-17T01:14:58.903 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3394_fsm 2023-10-17T01:14:58.903 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3394_vm 2023-10-17T01:14:58.903 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3395 2023-10-17T01:14:58.903 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439 2023-10-17T01:14:58.903 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439_vm 2023-10-17T01:14:58.903 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3440 2023-10-17T01:14:58.904 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 3455 2023-10-17T01:14:58.904 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3456 2023-10-17T01:14:58.904 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3456_fsm 2023-10-17T01:14:58.904 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3456_vm 2023-10-17T01:14:58.904 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466 2023-10-17T01:14:58.905 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466_vm 2023-10-17T01:14:58.905 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3467 2023-10-17T01:14:58.905 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3468 2023-10-17T01:14:58.905 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501 2023-10-17T01:14:58.905 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501_vm 2023-10-17T01:14:58.906 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3502 2023-10-17T01:14:58.906 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3503 2023-10-17T01:14:58.906 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3534 2023-10-17T01:14:58.906 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541 2023-10-17T01:14:58.906 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3541_fsm 2023-10-17T01:14:58.907 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541_vm 2023-10-17T01:14:58.907 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3542 2023-10-17T01:14:58.907 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3574 2023-10-17T01:14:58.907 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3575 2023-10-17T01:14:58.907 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576 2023-10-17T01:14:58.908 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576_vm 2023-10-17T01:14:58.908 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596 2023-10-17T01:14:58.908 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596_vm 2023-10-17T01:14:58.908 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3597 2023-10-17T01:14:58.908 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598 2023-10-17T01:14:58.909 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598_vm 2023-10-17T01:14:58.909 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3599 2023-10-17T01:14:58.909 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600 2023-10-17T01:14:58.909 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3600_fsm 2023-10-17T01:14:58.909 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600_vm 2023-10-17T01:14:58.909 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601 2023-10-17T01:14:58.910 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3601_fsm 2023-10-17T01:14:58.910 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601_vm 2023-10-17T01:14:58.910 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602 2023-10-17T01:14:58.910 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3602_fsm 2023-10-17T01:14:58.910 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602_vm 2023-10-17T01:14:58.911 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3603 2023-10-17T01:14:58.911 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3603_fsm 2023-10-17T01:14:58.911 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3603_vm 2023-10-17T01:14:58.911 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3604 2023-10-17T01:14:58.911 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3605 2023-10-17T01:14:58.912 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3606 2023-10-17T01:14:58.915 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3607 2023-10-17T01:14:58.915 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3608 2023-10-17T01:14:58.915 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 3609 2023-10-17T01:14:58.916 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3712 2023-10-17T01:14:58.916 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764 2023-10-17T01:14:58.916 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3764_fsm 2023-10-17T01:14:58.916 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764_vm 2023-10-17T01:14:58.916 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3766 2023-10-17T01:14:58.917 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3767 2023-10-17T01:14:58.917 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3997 2023-10-17T01:14:58.917 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 5002 2023-10-17T01:14:58.917 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 548 2023-10-17T01:14:58.917 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 549 2023-10-17T01:14:58.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102 2023-10-17T01:14:58.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102_vm 2023-10-17T01:14:58.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104 2023-10-17T01:14:58.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104_vm 2023-10-17T01:14:58.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106 2023-10-17T01:14:58.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106_vm 2023-10-17T01:14:58.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6110 2023-10-17T01:14:58.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6111 2023-10-17T01:14:58.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6112 2023-10-17T01:14:58.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6113 2023-10-17T01:14:58.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6117 2023-10-17T01:14:58.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826 2023-10-17T01:14:58.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826_vm 2023-10-17T01:14:58.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 827 2023-10-17T01:14:58.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 828 2023-10-17T01:14:58.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:14:58.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:14:58.921 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:58.921 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-17T01:14:58.921 INFO:teuthology.orchestra.run.smithi045.stdout:total 7185 2023-10-17T01:14:58.922 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 . 2023-10-17T01:14:58.922 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 .. 2023-10-17T01:14:58.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 112 2023-10-17T01:14:58.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 113 2023-10-17T01:14:58.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 17 01:14 12182 2023-10-17T01:14:58.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12182_fsm 2023-10-17T01:14:58.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12182_vm 2023-10-17T01:14:58.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12184 2023-10-17T01:14:58.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12186 2023-10-17T01:14:58.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187 2023-10-17T01:14:58.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12187_fsm 2023-10-17T01:14:58.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187_vm 2023-10-17T01:14:58.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12189 2023-10-17T01:14:58.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12191 2023-10-17T01:14:58.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192 2023-10-17T01:14:58.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12192_fsm 2023-10-17T01:14:58.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192_vm 2023-10-17T01:14:58.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12194 2023-10-17T01:14:58.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12196 2023-10-17T01:14:58.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197 2023-10-17T01:14:58.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12197_fsm 2023-10-17T01:14:58.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197_vm 2023-10-17T01:14:58.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12199 2023-10-17T01:14:58.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12201 2023-10-17T01:14:58.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202 2023-10-17T01:14:58.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12202_fsm 2023-10-17T01:14:58.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202_vm 2023-10-17T01:14:58.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12204 2023-10-17T01:14:58.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12206 2023-10-17T01:14:58.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207 2023-10-17T01:14:58.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12207_fsm 2023-10-17T01:14:58.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207_vm 2023-10-17T01:14:58.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12209 2023-10-17T01:14:58.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12211 2023-10-17T01:14:58.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12212 2023-10-17T01:14:58.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12214 2023-10-17T01:14:58.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12216 2023-10-17T01:14:58.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 17 01:14 1247 2023-10-17T01:14:58.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1247_fsm 2023-10-17T01:14:58.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1247_vm 2023-10-17T01:14:58.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 17 01:14 1249 2023-10-17T01:14:58.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1249_fsm 2023-10-17T01:14:58.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1249_vm 2023-10-17T01:14:58.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 17 01:14 1255 2023-10-17T01:14:58.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1255_fsm 2023-10-17T01:14:58.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1255_vm 2023-10-17T01:14:58.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:14 1259 2023-10-17T01:14:58.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1259_fsm 2023-10-17T01:14:58.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1259_vm 2023-10-17T01:14:58.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417 2023-10-17T01:14:58.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417_vm 2023-10-17T01:14:58.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418 2023-10-17T01:14:58.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418_vm 2023-10-17T01:14:58.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 174 2023-10-17T01:14:58.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 175 2023-10-17T01:14:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2187 2023-10-17T01:14:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224 2023-10-17T01:14:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224_vm 2023-10-17T01:14:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328 2023-10-17T01:14:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328_vm 2023-10-17T01:14:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336 2023-10-17T01:14:58.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336_vm 2023-10-17T01:14:58.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2337 2023-10-17T01:14:58.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2600 2023-10-17T01:14:58.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2600_fsm 2023-10-17T01:14:58.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2600_vm 2023-10-17T01:14:58.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601 2023-10-17T01:14:58.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2601_fsm 2023-10-17T01:14:58.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601_vm 2023-10-17T01:14:58.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2602 2023-10-17T01:14:58.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2602_fsm 2023-10-17T01:14:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2602_vm 2023-10-17T01:14:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2603 2023-10-17T01:14:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2603_fsm 2023-10-17T01:14:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2603_vm 2023-10-17T01:14:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604 2023-10-17T01:14:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604_vm 2023-10-17T01:14:58.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2605 2023-10-17T01:14:58.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2605_fsm 2023-10-17T01:14:58.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2605_vm 2023-10-17T01:14:58.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606 2023-10-17T01:14:58.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2606_fsm 2023-10-17T01:14:58.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606_vm 2023-10-17T01:14:58.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607 2023-10-17T01:14:58.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607_fsm 2023-10-17T01:14:58.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2607_vm 2023-10-17T01:14:58.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 17 01:14 2608 2023-10-17T01:14:58.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2608_fsm 2023-10-17T01:14:58.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2608_vm 2023-10-17T01:14:58.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 17 01:14 2609 2023-10-17T01:14:58.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2609_fsm 2023-10-17T01:14:58.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2609_vm 2023-10-17T01:14:58.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610 2023-10-17T01:14:58.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610_fsm 2023-10-17T01:14:58.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2610_vm 2023-10-17T01:14:58.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611 2023-10-17T01:14:58.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611_vm 2023-10-17T01:14:58.941 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612 2023-10-17T01:14:58.941 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2612_fsm 2023-10-17T01:14:58.941 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612_vm 2023-10-17T01:14:58.941 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613 2023-10-17T01:14:58.941 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613_vm 2023-10-17T01:14:58.942 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615 2023-10-17T01:14:58.942 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2615_fsm 2023-10-17T01:14:58.942 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615_vm 2023-10-17T01:14:58.942 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616 2023-10-17T01:14:58.942 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616_fsm 2023-10-17T01:14:58.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2616_vm 2023-10-17T01:14:58.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 17 01:14 2617 2023-10-17T01:14:58.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2617_fsm 2023-10-17T01:14:58.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2617_vm 2023-10-17T01:14:58.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 17 01:14 2618 2023-10-17T01:14:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2618_fsm 2023-10-17T01:14:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2618_vm 2023-10-17T01:14:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 17 01:14 2619 2023-10-17T01:14:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2619_fsm 2023-10-17T01:14:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2619_vm 2023-10-17T01:14:58.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620 2023-10-17T01:14:58.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620_vm 2023-10-17T01:14:58.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2650 2023-10-17T01:14:58.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2651 2023-10-17T01:14:58.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2652 2023-10-17T01:14:58.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2653 2023-10-17T01:14:58.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2654 2023-10-17T01:14:58.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2655 2023-10-17T01:14:58.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2656 2023-10-17T01:14:58.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2657 2023-10-17T01:14:58.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 17 01:14 2658 2023-10-17T01:14:58.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2659 2023-10-17T01:14:58.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2660 2023-10-17T01:14:58.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2661 2023-10-17T01:14:58.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2662 2023-10-17T01:14:58.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2663 2023-10-17T01:14:58.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2664 2023-10-17T01:14:58.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2665 2023-10-17T01:14:58.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2666 2023-10-17T01:14:58.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2667 2023-10-17T01:14:58.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2668 2023-10-17T01:14:58.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2669 2023-10-17T01:14:58.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2670 2023-10-17T01:14:58.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 17 01:14 2673 2023-10-17T01:14:58.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 17 01:14 2674 2023-10-17T01:14:58.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 17 01:14 2675 2023-10-17T01:14:58.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2678 2023-10-17T01:14:58.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2679 2023-10-17T01:14:58.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2680 2023-10-17T01:14:58.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2681 2023-10-17T01:14:58.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2682 2023-10-17T01:14:58.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2683 2023-10-17T01:14:58.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2684 2023-10-17T01:14:58.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2685 2023-10-17T01:14:58.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2686 2023-10-17T01:14:58.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2687 2023-10-17T01:14:58.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2688 2023-10-17T01:14:58.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2689 2023-10-17T01:14:58.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2690 2023-10-17T01:14:58.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 17 01:14 2691 2023-10-17T01:14:58.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2692 2023-10-17T01:14:58.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2693 2023-10-17T01:14:58.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2696 2023-10-17T01:14:58.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2699 2023-10-17T01:14:58.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2701 2023-10-17T01:14:58.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2702 2023-10-17T01:14:58.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2703 2023-10-17T01:14:58.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2704 2023-10-17T01:14:58.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2753 2023-10-17T01:14:58.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2753_fsm 2023-10-17T01:14:58.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2753_vm 2023-10-17T01:14:58.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2754 2023-10-17T01:14:58.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2755 2023-10-17T01:14:58.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2756 2023-10-17T01:14:58.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2757 2023-10-17T01:14:58.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830 2023-10-17T01:14:58.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830_vm 2023-10-17T01:14:58.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2831 2023-10-17T01:14:58.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832 2023-10-17T01:14:58.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832_vm 2023-10-17T01:14:58.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2833 2023-10-17T01:14:58.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834 2023-10-17T01:14:58.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834_vm 2023-10-17T01:14:58.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2835 2023-10-17T01:14:58.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836 2023-10-17T01:14:58.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836_vm 2023-10-17T01:14:58.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2837 2023-10-17T01:14:58.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 17 01:14 2838 2023-10-17T01:14:58.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2838_fsm 2023-10-17T01:14:58.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2838_vm 2023-10-17T01:14:58.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2839 2023-10-17T01:14:58.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840 2023-10-17T01:14:58.958 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840_fsm 2023-10-17T01:14:58.958 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2840_vm 2023-10-17T01:14:58.958 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2841 2023-10-17T01:14:58.958 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995 2023-10-17T01:14:58.958 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995_vm 2023-10-17T01:14:58.958 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2996 2023-10-17T01:14:58.959 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079 2023-10-17T01:14:58.959 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3079_fsm 2023-10-17T01:14:58.959 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079_vm 2023-10-17T01:14:58.959 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3080 2023-10-17T01:14:58.959 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3081 2023-10-17T01:14:58.960 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3085 2023-10-17T01:14:58.960 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118 2023-10-17T01:14:58.960 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118_vm 2023-10-17T01:14:58.960 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3119 2023-10-17T01:14:58.960 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3164 2023-10-17T01:14:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256 2023-10-17T01:14:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256_vm 2023-10-17T01:14:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3257 2023-10-17T01:14:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3258 2023-10-17T01:14:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350 2023-10-17T01:14:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350_vm 2023-10-17T01:14:58.962 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3351 2023-10-17T01:14:58.962 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3379 2023-10-17T01:14:58.962 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3380 2023-10-17T01:14:58.962 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381 2023-10-17T01:14:58.962 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381_vm 2023-10-17T01:14:58.962 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3394 2023-10-17T01:14:58.963 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3394_fsm 2023-10-17T01:14:58.963 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3394_vm 2023-10-17T01:14:58.963 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3395 2023-10-17T01:14:58.963 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439 2023-10-17T01:14:58.963 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439_vm 2023-10-17T01:14:58.964 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3440 2023-10-17T01:14:58.964 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 3455 2023-10-17T01:14:58.964 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3456 2023-10-17T01:14:58.964 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3456_fsm 2023-10-17T01:14:58.964 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3456_vm 2023-10-17T01:14:58.964 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466 2023-10-17T01:14:58.965 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466_vm 2023-10-17T01:14:58.965 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3467 2023-10-17T01:14:58.965 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3468 2023-10-17T01:14:58.965 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501 2023-10-17T01:14:58.965 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501_vm 2023-10-17T01:14:58.966 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3502 2023-10-17T01:14:58.966 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3503 2023-10-17T01:14:58.966 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3534 2023-10-17T01:14:58.966 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541 2023-10-17T01:14:58.966 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3541_fsm 2023-10-17T01:14:58.967 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541_vm 2023-10-17T01:14:58.967 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3542 2023-10-17T01:14:58.967 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3574 2023-10-17T01:14:58.967 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3575 2023-10-17T01:14:58.967 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576 2023-10-17T01:14:58.969 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576_vm 2023-10-17T01:14:58.969 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596 2023-10-17T01:14:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596_vm 2023-10-17T01:14:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3597 2023-10-17T01:14:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598 2023-10-17T01:14:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598_vm 2023-10-17T01:14:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3599 2023-10-17T01:14:58.971 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600 2023-10-17T01:14:58.971 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3600_fsm 2023-10-17T01:14:58.971 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600_vm 2023-10-17T01:14:58.971 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601 2023-10-17T01:14:58.971 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3601_fsm 2023-10-17T01:14:58.971 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601_vm 2023-10-17T01:14:58.972 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602 2023-10-17T01:14:58.972 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3602_fsm 2023-10-17T01:14:58.972 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602_vm 2023-10-17T01:14:58.972 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3603 2023-10-17T01:14:58.972 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3603_fsm 2023-10-17T01:14:58.973 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3603_vm 2023-10-17T01:14:58.973 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3604 2023-10-17T01:14:58.973 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3605 2023-10-17T01:14:58.973 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3606 2023-10-17T01:14:58.973 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3607 2023-10-17T01:14:58.973 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3608 2023-10-17T01:14:58.974 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 3609 2023-10-17T01:14:58.974 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3712 2023-10-17T01:14:58.974 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764 2023-10-17T01:14:58.974 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3764_fsm 2023-10-17T01:14:58.974 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764_vm 2023-10-17T01:14:58.975 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3766 2023-10-17T01:14:58.975 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3767 2023-10-17T01:14:58.975 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3997 2023-10-17T01:14:58.975 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 5002 2023-10-17T01:14:58.975 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 548 2023-10-17T01:14:58.976 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 549 2023-10-17T01:14:58.976 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102 2023-10-17T01:14:58.976 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102_vm 2023-10-17T01:14:58.976 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104 2023-10-17T01:14:58.976 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104_vm 2023-10-17T01:14:58.976 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106 2023-10-17T01:14:58.977 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106_vm 2023-10-17T01:14:58.977 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6110 2023-10-17T01:14:58.977 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6111 2023-10-17T01:14:58.977 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6112 2023-10-17T01:14:58.977 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6113 2023-10-17T01:14:58.978 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6117 2023-10-17T01:14:58.978 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826 2023-10-17T01:14:58.978 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826_vm 2023-10-17T01:14:58.978 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 827 2023-10-17T01:14:58.978 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 828 2023-10-17T01:14:58.979 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:14:58.979 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:14:58.979 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:58.979 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-17T01:14:58.979 INFO:teuthology.orchestra.run.smithi045.stdout:total 7185 2023-10-17T01:14:58.979 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 . 2023-10-17T01:14:58.980 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 .. 2023-10-17T01:14:58.980 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 112 2023-10-17T01:14:58.980 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 113 2023-10-17T01:14:58.980 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 17 01:14 12182 2023-10-17T01:14:58.980 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12182_fsm 2023-10-17T01:14:58.980 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12182_vm 2023-10-17T01:14:58.981 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12184 2023-10-17T01:14:58.981 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12186 2023-10-17T01:14:58.981 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187 2023-10-17T01:14:58.981 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12187_fsm 2023-10-17T01:14:58.981 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187_vm 2023-10-17T01:14:58.982 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12189 2023-10-17T01:14:58.982 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12191 2023-10-17T01:14:58.982 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192 2023-10-17T01:14:58.982 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12192_fsm 2023-10-17T01:14:58.982 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192_vm 2023-10-17T01:14:58.982 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12194 2023-10-17T01:14:58.983 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12196 2023-10-17T01:14:58.983 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197 2023-10-17T01:14:58.983 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12197_fsm 2023-10-17T01:14:58.983 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197_vm 2023-10-17T01:14:58.983 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12199 2023-10-17T01:14:58.984 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12201 2023-10-17T01:14:58.984 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202 2023-10-17T01:14:58.984 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12202_fsm 2023-10-17T01:14:58.984 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202_vm 2023-10-17T01:14:58.984 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12204 2023-10-17T01:14:58.984 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12206 2023-10-17T01:14:58.985 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207 2023-10-17T01:14:58.985 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12207_fsm 2023-10-17T01:14:58.985 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207_vm 2023-10-17T01:14:58.985 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12209 2023-10-17T01:14:58.985 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12211 2023-10-17T01:14:58.986 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12212 2023-10-17T01:14:58.986 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12214 2023-10-17T01:14:58.986 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12216 2023-10-17T01:14:58.986 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 17 01:14 1247 2023-10-17T01:14:58.986 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1247_fsm 2023-10-17T01:14:58.987 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1247_vm 2023-10-17T01:14:58.987 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 17 01:14 1249 2023-10-17T01:14:58.987 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1249_fsm 2023-10-17T01:14:58.987 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1249_vm 2023-10-17T01:14:58.987 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 17 01:14 1255 2023-10-17T01:14:58.988 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1255_fsm 2023-10-17T01:14:58.988 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1255_vm 2023-10-17T01:14:58.988 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:14 1259 2023-10-17T01:14:58.988 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1259_fsm 2023-10-17T01:14:58.988 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1259_vm 2023-10-17T01:14:58.988 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417 2023-10-17T01:14:58.989 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417_vm 2023-10-17T01:14:58.989 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418 2023-10-17T01:14:58.989 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418_vm 2023-10-17T01:14:58.989 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 174 2023-10-17T01:14:58.989 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 175 2023-10-17T01:14:58.989 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2187 2023-10-17T01:14:58.990 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224 2023-10-17T01:14:58.990 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224_vm 2023-10-17T01:14:58.990 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328 2023-10-17T01:14:58.990 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328_vm 2023-10-17T01:14:58.990 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336 2023-10-17T01:14:58.991 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336_vm 2023-10-17T01:14:58.991 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2337 2023-10-17T01:14:58.991 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2600 2023-10-17T01:14:58.991 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2600_fsm 2023-10-17T01:14:58.991 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2600_vm 2023-10-17T01:14:58.991 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601 2023-10-17T01:14:58.992 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2601_fsm 2023-10-17T01:14:58.992 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601_vm 2023-10-17T01:14:58.992 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2602 2023-10-17T01:14:58.992 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2602_fsm 2023-10-17T01:14:58.992 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2602_vm 2023-10-17T01:14:58.993 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2603 2023-10-17T01:14:58.993 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2603_fsm 2023-10-17T01:14:58.993 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2603_vm 2023-10-17T01:14:58.993 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604 2023-10-17T01:14:58.993 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604_vm 2023-10-17T01:14:58.993 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2605 2023-10-17T01:14:58.994 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2605_fsm 2023-10-17T01:14:58.994 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2605_vm 2023-10-17T01:14:58.994 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606 2023-10-17T01:14:58.994 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2606_fsm 2023-10-17T01:14:58.994 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606_vm 2023-10-17T01:14:58.995 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607 2023-10-17T01:14:58.995 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607_fsm 2023-10-17T01:14:58.995 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2607_vm 2023-10-17T01:14:58.995 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 17 01:14 2608 2023-10-17T01:14:58.995 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2608_fsm 2023-10-17T01:14:58.995 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2608_vm 2023-10-17T01:14:58.996 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 17 01:14 2609 2023-10-17T01:14:58.996 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2609_fsm 2023-10-17T01:14:58.996 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2609_vm 2023-10-17T01:14:58.996 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610 2023-10-17T01:14:58.996 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610_fsm 2023-10-17T01:14:58.997 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2610_vm 2023-10-17T01:14:58.997 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611 2023-10-17T01:14:58.997 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611_vm 2023-10-17T01:14:58.997 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612 2023-10-17T01:14:58.997 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2612_fsm 2023-10-17T01:14:58.997 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612_vm 2023-10-17T01:14:58.998 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613 2023-10-17T01:14:58.998 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613_vm 2023-10-17T01:14:58.998 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615 2023-10-17T01:14:58.998 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2615_fsm 2023-10-17T01:14:58.998 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615_vm 2023-10-17T01:14:58.999 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616 2023-10-17T01:14:58.999 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616_fsm 2023-10-17T01:14:58.999 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2616_vm 2023-10-17T01:14:58.999 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 17 01:14 2617 2023-10-17T01:14:58.999 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2617_fsm 2023-10-17T01:14:59.000 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2617_vm 2023-10-17T01:14:59.000 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 17 01:14 2618 2023-10-17T01:14:59.000 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2618_fsm 2023-10-17T01:14:59.000 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2618_vm 2023-10-17T01:14:59.000 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 17 01:14 2619 2023-10-17T01:14:59.000 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2619_fsm 2023-10-17T01:14:59.001 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2619_vm 2023-10-17T01:14:59.001 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620 2023-10-17T01:14:59.001 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620_vm 2023-10-17T01:14:59.001 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2650 2023-10-17T01:14:59.001 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2651 2023-10-17T01:14:59.001 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2652 2023-10-17T01:14:59.002 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2653 2023-10-17T01:14:59.002 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2654 2023-10-17T01:14:59.002 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2655 2023-10-17T01:14:59.002 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2656 2023-10-17T01:14:59.002 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2657 2023-10-17T01:14:59.003 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 17 01:14 2658 2023-10-17T01:14:59.003 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2659 2023-10-17T01:14:59.003 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2660 2023-10-17T01:14:59.003 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2661 2023-10-17T01:14:59.003 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2662 2023-10-17T01:14:59.003 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2663 2023-10-17T01:14:59.004 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2664 2023-10-17T01:14:59.004 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2665 2023-10-17T01:14:59.004 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2666 2023-10-17T01:14:59.004 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2667 2023-10-17T01:14:59.004 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2668 2023-10-17T01:14:59.005 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2669 2023-10-17T01:14:59.005 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2670 2023-10-17T01:14:59.005 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 17 01:14 2673 2023-10-17T01:14:59.005 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 17 01:14 2674 2023-10-17T01:14:59.005 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 17 01:14 2675 2023-10-17T01:14:59.005 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2678 2023-10-17T01:14:59.006 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2679 2023-10-17T01:14:59.006 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2680 2023-10-17T01:14:59.006 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2681 2023-10-17T01:14:59.006 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2682 2023-10-17T01:14:59.006 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2683 2023-10-17T01:14:59.007 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2684 2023-10-17T01:14:59.007 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2685 2023-10-17T01:14:59.007 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2686 2023-10-17T01:14:59.007 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2687 2023-10-17T01:14:59.007 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2688 2023-10-17T01:14:59.007 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2689 2023-10-17T01:14:59.008 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2690 2023-10-17T01:14:59.008 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 17 01:14 2691 2023-10-17T01:14:59.008 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2692 2023-10-17T01:14:59.008 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2693 2023-10-17T01:14:59.008 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2696 2023-10-17T01:14:59.009 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2699 2023-10-17T01:14:59.009 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2701 2023-10-17T01:14:59.009 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2702 2023-10-17T01:14:59.009 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2703 2023-10-17T01:14:59.009 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2704 2023-10-17T01:14:59.009 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2753 2023-10-17T01:14:59.010 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2753_fsm 2023-10-17T01:14:59.010 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2753_vm 2023-10-17T01:14:59.010 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2754 2023-10-17T01:14:59.010 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2755 2023-10-17T01:14:59.010 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2756 2023-10-17T01:14:59.011 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2757 2023-10-17T01:14:59.011 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830 2023-10-17T01:14:59.011 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830_vm 2023-10-17T01:14:59.011 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2831 2023-10-17T01:14:59.011 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832 2023-10-17T01:14:59.011 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832_vm 2023-10-17T01:14:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2833 2023-10-17T01:14:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834 2023-10-17T01:14:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834_vm 2023-10-17T01:14:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2835 2023-10-17T01:14:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836 2023-10-17T01:14:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836_vm 2023-10-17T01:14:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2837 2023-10-17T01:14:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 17 01:14 2838 2023-10-17T01:14:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2838_fsm 2023-10-17T01:14:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2838_vm 2023-10-17T01:14:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2839 2023-10-17T01:14:59.014 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840 2023-10-17T01:14:59.014 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840_fsm 2023-10-17T01:14:59.014 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2840_vm 2023-10-17T01:14:59.014 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2841 2023-10-17T01:14:59.014 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995 2023-10-17T01:14:59.015 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995_vm 2023-10-17T01:14:59.015 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2996 2023-10-17T01:14:59.015 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079 2023-10-17T01:14:59.015 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3079_fsm 2023-10-17T01:14:59.015 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079_vm 2023-10-17T01:14:59.016 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3080 2023-10-17T01:14:59.016 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3081 2023-10-17T01:14:59.016 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3085 2023-10-17T01:14:59.016 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118 2023-10-17T01:14:59.016 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118_vm 2023-10-17T01:14:59.016 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3119 2023-10-17T01:14:59.017 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3164 2023-10-17T01:14:59.017 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256 2023-10-17T01:14:59.017 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256_vm 2023-10-17T01:14:59.017 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3257 2023-10-17T01:14:59.017 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3258 2023-10-17T01:14:59.018 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350 2023-10-17T01:14:59.018 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350_vm 2023-10-17T01:14:59.018 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3351 2023-10-17T01:14:59.018 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3379 2023-10-17T01:14:59.018 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3380 2023-10-17T01:14:59.018 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381 2023-10-17T01:14:59.019 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381_vm 2023-10-17T01:14:59.019 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3394 2023-10-17T01:14:59.019 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3394_fsm 2023-10-17T01:14:59.019 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3394_vm 2023-10-17T01:14:59.019 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3395 2023-10-17T01:14:59.020 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439 2023-10-17T01:14:59.020 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439_vm 2023-10-17T01:14:59.020 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3440 2023-10-17T01:14:59.020 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 3455 2023-10-17T01:14:59.020 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3456 2023-10-17T01:14:59.020 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3456_fsm 2023-10-17T01:14:59.021 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3456_vm 2023-10-17T01:14:59.021 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466 2023-10-17T01:14:59.021 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466_vm 2023-10-17T01:14:59.021 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3467 2023-10-17T01:14:59.021 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3468 2023-10-17T01:14:59.022 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501 2023-10-17T01:14:59.022 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501_vm 2023-10-17T01:14:59.022 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3502 2023-10-17T01:14:59.022 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3503 2023-10-17T01:14:59.022 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3534 2023-10-17T01:14:59.022 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541 2023-10-17T01:14:59.023 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3541_fsm 2023-10-17T01:14:59.023 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541_vm 2023-10-17T01:14:59.023 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3542 2023-10-17T01:14:59.023 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3574 2023-10-17T01:14:59.023 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3575 2023-10-17T01:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576 2023-10-17T01:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576_vm 2023-10-17T01:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596 2023-10-17T01:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596_vm 2023-10-17T01:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3597 2023-10-17T01:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598 2023-10-17T01:14:59.025 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598_vm 2023-10-17T01:14:59.025 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3599 2023-10-17T01:14:59.025 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600 2023-10-17T01:14:59.025 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3600_fsm 2023-10-17T01:14:59.025 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600_vm 2023-10-17T01:14:59.026 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601 2023-10-17T01:14:59.026 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3601_fsm 2023-10-17T01:14:59.026 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601_vm 2023-10-17T01:14:59.026 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602 2023-10-17T01:14:59.026 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3602_fsm 2023-10-17T01:14:59.026 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602_vm 2023-10-17T01:14:59.027 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3603 2023-10-17T01:14:59.027 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3603_fsm 2023-10-17T01:14:59.027 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3603_vm 2023-10-17T01:14:59.027 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3604 2023-10-17T01:14:59.027 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3605 2023-10-17T01:14:59.028 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3606 2023-10-17T01:14:59.028 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3607 2023-10-17T01:14:59.028 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3608 2023-10-17T01:14:59.028 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 3609 2023-10-17T01:14:59.028 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3712 2023-10-17T01:14:59.029 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764 2023-10-17T01:14:59.029 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3764_fsm 2023-10-17T01:14:59.029 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764_vm 2023-10-17T01:14:59.029 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3766 2023-10-17T01:14:59.029 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3767 2023-10-17T01:14:59.029 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3997 2023-10-17T01:14:59.030 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 5002 2023-10-17T01:14:59.030 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 548 2023-10-17T01:14:59.030 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 549 2023-10-17T01:14:59.030 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102 2023-10-17T01:14:59.030 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102_vm 2023-10-17T01:14:59.031 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104 2023-10-17T01:14:59.031 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104_vm 2023-10-17T01:14:59.031 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106 2023-10-17T01:14:59.031 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106_vm 2023-10-17T01:14:59.031 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6110 2023-10-17T01:14:59.031 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6111 2023-10-17T01:14:59.032 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6112 2023-10-17T01:14:59.032 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6113 2023-10-17T01:14:59.032 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6117 2023-10-17T01:14:59.032 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826 2023-10-17T01:14:59.032 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826_vm 2023-10-17T01:14:59.033 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 827 2023-10-17T01:14:59.033 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 828 2023-10-17T01:14:59.033 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:14:59.033 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:14:59.033 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.033 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/global: 2023-10-17T01:14:59.034 INFO:teuthology.orchestra.run.smithi045.stdout:total 553 2023-10-17T01:14:59.034 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 17 01:14 . 2023-10-17T01:14:59.034 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.034 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1136 2023-10-17T01:14:59.034 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1136_fsm 2023-10-17T01:14:59.035 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1136_vm 2023-10-17T01:14:59.035 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 1137 2023-10-17T01:14:59.035 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1213 2023-10-17T01:14:59.035 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1213_fsm 2023-10-17T01:14:59.035 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1213_vm 2023-10-17T01:14:59.035 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1214 2023-10-17T01:14:59.036 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1214_fsm 2023-10-17T01:14:59.036 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1214_vm 2023-10-17T01:14:59.036 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 1232 2023-10-17T01:14:59.036 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 1233 2023-10-17T01:14:59.036 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1260 2023-10-17T01:14:59.037 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1260_fsm 2023-10-17T01:14:59.037 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1260_vm 2023-10-17T01:14:59.037 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1261 2023-10-17T01:14:59.037 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1261_fsm 2023-10-17T01:14:59.037 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1261_vm 2023-10-17T01:14:59.038 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1262 2023-10-17T01:14:59.038 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1262_fsm 2023-10-17T01:14:59.038 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1262_vm 2023-10-17T01:14:59.038 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2396 2023-10-17T01:14:59.038 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2396_fsm 2023-10-17T01:14:59.038 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2396_vm 2023-10-17T01:14:59.039 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2397 2023-10-17T01:14:59.039 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2671 2023-10-17T01:14:59.039 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2672 2023-10-17T01:14:59.039 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2676 2023-10-17T01:14:59.039 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2677 2023-10-17T01:14:59.040 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2694 2023-10-17T01:14:59.040 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2695 2023-10-17T01:14:59.040 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2697 2023-10-17T01:14:59.040 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2698 2023-10-17T01:14:59.040 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2846 2023-10-17T01:14:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2846_vm 2023-10-17T01:14:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2847 2023-10-17T01:14:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2964 2023-10-17T01:14:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2964_vm 2023-10-17T01:14:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2965 2023-10-17T01:14:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2966 2023-10-17T01:14:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2966_vm 2023-10-17T01:14:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2967 2023-10-17T01:14:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3592 2023-10-17T01:14:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3592_vm 2023-10-17T01:14:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3593 2023-10-17T01:14:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 4060 2023-10-17T01:14:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 4060_vm 2023-10-17T01:14:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 4061 2023-10-17T01:14:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6000 2023-10-17T01:14:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6000_vm 2023-10-17T01:14:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6001 2023-10-17T01:14:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6002 2023-10-17T01:14:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6100 2023-10-17T01:14:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6100_vm 2023-10-17T01:14:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6114 2023-10-17T01:14:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6115 2023-10-17T01:14:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 pg_control 2023-10-17T01:14:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:14:59.045 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/log: 2023-10-17T01:14:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.046 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.046 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-17T01:14:59.046 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.046 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.047 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.047 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.047 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-17T01:14:59.047 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.047 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.048 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.048 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.048 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-17T01:14:59.048 INFO:teuthology.orchestra.run.smithi045.stdout:total 1 2023-10-17T01:14:59.048 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 . 2023-10-17T01:14:59.048 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.049 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 mappings 2023-10-17T01:14:59.049 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 17 01:14 replorigin_checkpoint 2023-10-17T01:14:59.049 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 snapshots 2023-10-17T01:14:59.049 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.049 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-17T01:14:59.050 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.050 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.050 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 .. 2023-10-17T01:14:59.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.050 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-17T01:14:59.051 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.051 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.051 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 .. 2023-10-17T01:14:59.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.051 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-17T01:14:59.051 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.052 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 . 2023-10-17T01:14:59.052 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.052 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 members 2023-10-17T01:14:59.052 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 offsets 2023-10-17T01:14:59.052 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.052 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-17T01:14:59.053 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:14:59.053 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:59.053 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 .. 2023-10-17T01:14:59.053 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:14:59.053 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.054 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-17T01:14:59.054 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:14:59.054 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:59.054 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 .. 2023-10-17T01:14:59.054 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:14:59.055 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.055 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-17T01:14:59.055 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:14:59.055 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:59.055 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.055 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:14:59.056 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.056 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-17T01:14:59.056 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.056 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.056 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.057 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.057 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-17T01:14:59.057 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.057 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.057 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.057 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.058 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-17T01:14:59.058 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.058 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.058 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.058 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.059 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-17T01:14:59.059 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.059 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.059 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.059 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.059 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-17T01:14:59.060 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.060 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.060 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.060 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.060 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-17T01:14:59.061 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:14:59.061 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:59.061 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.061 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:14:59.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.061 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-17T01:14:59.062 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.062 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.062 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.062 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.062 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-17T01:14:59.063 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.063 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.063 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.063 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-17T01:14:59.064 INFO:teuthology.orchestra.run.smithi045.stdout:total 16384 2023-10-17T01:14:59.064 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 . 2023-10-17T01:14:59.064 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.064 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:14 000000010000000000000001 2023-10-17T01:14:59.064 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 archive_status 2023-10-17T01:14:59.064 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.065 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-17T01:14:59.065 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:14:59.065 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:14:59.065 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 .. 2023-10-17T01:14:59.065 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:14:59.066 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-17T01:14:59.066 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:14:59.066 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:14:59.066 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 17 01:14 .. 2023-10-17T01:14:59.066 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:14:59.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-10-17T01:14:59.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-10-17T01:14:59.432 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-10-17T01:14:59.437 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-10-17T01:14:59.440 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-10-17T01:14:59.446 INFO:teuthology.orchestra.run.smithi045.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-10-17T01:14:59.473 INFO:teuthology.orchestra.run.smithi045.stderr:creating tables... 2023-10-17T01:14:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.12 s, remaining 3.64 s) 2023-10-17T01:14:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:14:59 smithi045 ceph-mon[98474]: pgmap v328: 97 pgs: 97 active+clean; 13 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2023-10-17T01:14:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:14:59 smithi060 ceph-mon[134136]: pgmap v328: 97 pgs: 97 active+clean; 13 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2023-10-17T01:14:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.41 s, remaining 6.11 s) 2023-10-17T01:14:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:14:59 smithi181 ceph-mon[134520]: pgmap v328: 97 pgs: 97 active+clean; 13 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2023-10-17T01:15:00.147 INFO:teuthology.orchestra.run.smithi045.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.67 s, remaining 6.50 s) 2023-10-17T01:15:00.326 INFO:teuthology.orchestra.run.smithi045.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.85 s, remaining 5.96 s) 2023-10-17T01:15:00.606 INFO:teuthology.orchestra.run.smithi045.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.13 s, remaining 6.11 s) 2023-10-17T01:15:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.41 s, remaining 6.09 s) 2023-10-17T01:15:00.994 INFO:teuthology.orchestra.run.smithi045.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.52 s, remaining 5.43 s) 2023-10-17T01:15:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.80 s, remaining 5.40 s) 2023-10-17T01:15:01.573 INFO:teuthology.orchestra.run.smithi045.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.10 s, remaining 5.36 s) 2023-10-17T01:15:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.20 s, remaining 4.84 s) 2023-10-17T01:15:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:01 smithi045 ceph-mon[98474]: pgmap v329: 97 pgs: 97 active+clean; 31 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.8 MiB/s wr, 57 op/s 2023-10-17T01:15:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:01 smithi060 ceph-mon[134136]: pgmap v329: 97 pgs: 97 active+clean; 31 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.8 MiB/s wr, 57 op/s 2023-10-17T01:15:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:01 smithi181 ceph-mon[134520]: pgmap v329: 97 pgs: 97 active+clean; 31 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.8 MiB/s wr, 57 op/s 2023-10-17T01:15:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.60 s, remaining 4.97 s) 2023-10-17T01:15:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:1200000 of 3200000 tuples (37%) done (elapsed 2.72 s, remaining 4.54 s) 2023-10-17T01:15:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.01 s, remaining 4.40 s) 2023-10-17T01:15:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.33 s, remaining 4.28 s) 2023-10-17T01:15:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.44 s, remaining 3.90 s) 2023-10-17T01:15:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:1600000 of 3200000 tuples (50%) done (elapsed 3.70 s, remaining 3.70 s) 2023-10-17T01:15:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:1700000 of 3200000 tuples (53%) done (elapsed 3.87 s, remaining 3.41 s) 2023-10-17T01:15:03.603 INFO:teuthology.orchestra.run.smithi045.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.13 s, remaining 3.21 s) 2023-10-17T01:15:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:03 smithi045 ceph-mon[98474]: pgmap v330: 97 pgs: 97 active+clean; 66 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 8.2 MiB/s wr, 116 op/s 2023-10-17T01:15:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:03 smithi060 ceph-mon[134136]: pgmap v330: 97 pgs: 97 active+clean; 66 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 8.2 MiB/s wr, 116 op/s 2023-10-17T01:15:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:03 smithi181 ceph-mon[134520]: pgmap v330: 97 pgs: 97 active+clean; 66 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 8.2 MiB/s wr, 116 op/s 2023-10-17T01:15:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:1900000 of 3200000 tuples (59%) done (elapsed 4.45 s, remaining 3.04 s) 2023-10-17T01:15:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 -- 172.21.15.45:0/1703574981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc100940 msgr2=0x7ff1fc100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 --2- 172.21.15.45:0/1703574981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc100940 0x7ff1fc100dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400b0a0 tx=0x7ff1f400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 -- 172.21.15.45:0/1703574981 shutdown_connections 2023-10-17T01:15:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 --2- 172.21.15.45:0/1703574981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc100940 0x7ff1fc100dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 --2- 172.21.15.45:0/1703574981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc0ff620 0x7ff1fc0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 --2- 172.21.15.45:0/1703574981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0fff80 0x7ff1fc100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.019+0000 7ff200fff700 1 -- 172.21.15.45:0/1703574981 >> 172.21.15.45:0/1703574981 conn(0x7ff1fc0fb1a0 msgr2=0x7ff1fc0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.020+0000 7ff200fff700 1 -- 172.21.15.45:0/1703574981 shutdown_connections 2023-10-17T01:15:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.020+0000 7ff200fff700 1 -- 172.21.15.45:0/1703574981 wait complete. 2023-10-17T01:15:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.021+0000 7ff200fff700 1 Processor -- start 2023-10-17T01:15:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.021+0000 7ff200fff700 1 -- start start 2023-10-17T01:15:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff200fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0ff620 0x7ff1fc198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff200fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc0fff80 0x7ff1fc1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff200fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 0x7ff1fc19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff200fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc19a870 con 0x7ff1fc0ff620 2023-10-17T01:15:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff200fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc19a9e0 con 0x7ff1fc100940 2023-10-17T01:15:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff200fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc19aba0 con 0x7ff1fc0fff80 2023-10-17T01:15:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 0x7ff1fc19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0ff620 0x7ff1fc198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 0x7ff1fc19a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:37512/0 (socket says 172.21.15.45:37512) 2023-10-17T01:15:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 -- 172.21.15.45:0/290096356 learned_addr learned my addr 172.21.15.45:0/290096356 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc0fff80 0x7ff1fc1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 -- 172.21.15.45:0/290096356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc0fff80 msgr2=0x7ff1fc1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc0fff80 0x7ff1fc1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 -- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0ff620 msgr2=0x7ff1fc198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0ff620 0x7ff1fc198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.022+0000 7ff1fad9d700 1 -- 172.21.15.45:0/290096356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f4009d00 con 0x7ff1fc100940 2023-10-17T01:15:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.023+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 0x7ff1fc19a160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009850 tx=0x7ff1f400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.023+0000 7ff1eb7fe700 1 -- 172.21.15.45:0/290096356 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400e040 con 0x7ff1fc100940 2023-10-17T01:15:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.023+0000 7ff1eb7fe700 1 -- 172.21.15.45:0/290096356 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1f4003e10 con 0x7ff1fc100940 2023-10-17T01:15:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.023+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc19ae20 con 0x7ff1fc100940 2023-10-17T01:15:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.023+0000 7ff1eb7fe700 1 -- 172.21.15.45:0/290096356 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f401b6a0 con 0x7ff1fc100940 2023-10-17T01:15:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.023+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc1a1590 con 0x7ff1fc100940 2023-10-17T01:15:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.028+0000 7ff1eb7fe700 1 -- 172.21.15.45:0/290096356 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff1f401d470 con 0x7ff1fc100940 2023-10-17T01:15:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.029+0000 7ff1eb7fe700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1e40a0ba0 0x7ff1e40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.030+0000 7ff1eb7fe700 1 -- 172.21.15.45:0/290096356 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1f40cd480 con 0x7ff1fc100940 2023-10-17T01:15:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.030+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1e40a0ba0 0x7ff1e40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.031+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1e40a0ba0 0x7ff1e40a3060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc198bd0 tx=0x7ff1ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.033+0000 7ff214a00b80 1 Processor -- start 2023-10-17T01:15:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 -- start start 2023-10-17T01:15:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 0x562f9edc7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edc8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ed1f9b0 con 0x562f9edc8190 2023-10-17T01:15:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ed1d370 con 0x562f9edc8d90 2023-10-17T01:15:04.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.034+0000 7ff214a00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ed18c60 con 0x562f9edc7830 2023-10-17T01:15:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 0x562f9edc7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 0x562f9edc7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46976/0 (socket says 172.21.15.45:46976) 2023-10-17T01:15:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 -- 172.21.15.45:0/4193886980 learned_addr learned my addr 172.21.15.45:0/4193886980 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edc8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 -- 172.21.15.45:0/4193886980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 msgr2=0x562f9edc9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edc9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 -- 172.21.15.45:0/4193886980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 msgr2=0x562f9edc8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edc8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 -- 172.21.15.45:0/4193886980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f9eb46d80 con 0x562f9edc7830 2023-10-17T01:15:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.035+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 0x562f9edc7c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec010f70 tx=0x7ff1ec013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ccb14b55e4d8e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.036+0000 7ff1e8ff9700 1 -- 172.21.15.45:0/4193886980 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec01e620 con 0x562f9edc7830 2023-10-17T01:15:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.036+0000 7ff1e8ff9700 1 -- 172.21.15.45:0/4193886980 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1ec015070 con 0x562f9edc7830 2023-10-17T01:15:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.036+0000 7ff1e8ff9700 1 -- 172.21.15.45:0/4193886980 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec01c9b0 con 0x562f9edc7830 2023-10-17T01:15:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.036+0000 7ff214a00b80 1 -- 172.21.15.45:0/4193886980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 msgr2=0x562f9edc7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.036+0000 7ff214a00b80 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 0x562f9edc7c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec010f70 tx=0x7ff1ec013000 comp rx=0 tx=0).stop 2023-10-17T01:15:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 -- 172.21.15.45:0/4193886980 shutdown_connections 2023-10-17T01:15:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc7830 0x562f9edc7c50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 --2- 172.21.15.45:0/4193886980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edc8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 -- 172.21.15.45:0/4193886980 >> 172.21.15.45:0/4193886980 conn(0x562f9ed27070 msgr2=0x562f9edc22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 -- 172.21.15.45:0/4193886980 shutdown_connections 2023-10-17T01:15:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 -- 172.21.15.45:0/4193886980 wait complete. 2023-10-17T01:15:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.037+0000 7ff214a00b80 1 -- start start 2023-10-17T01:15:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 Processor -- start 2023-10-17T01:15:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 0x562f9edd1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edd1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8d90 0x562f9edd8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ecb54b0 con 0x562f9edc8190 2023-10-17T01:15:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ecb5150 con 0x562f9edc7830 2023-10-17T01:15:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9eb9c8c0 con 0x562f9edc8d90 2023-10-17T01:15:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.038+0000 7ff214a00b80 10 client.0 _ll_register_callbacks cb 0x562f9ec02a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 0x562f9edd1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 0x562f9edd1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:37532/0 (socket says 172.21.15.45:37532) 2023-10-17T01:15:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fa59c700 1 -- 172.21.15.45:0/1610684860 learned_addr learned my addr 172.21.15.45:0/1610684860 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edd1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8d90 0x562f9edd8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fa59c700 1 -- 172.21.15.45:0/1610684860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8d90 msgr2=0x562f9edd8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8d90 0x562f9edd8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.039+0000 7ff1fa59c700 1 -- 172.21.15.45:0/1610684860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 msgr2=0x562f9edd1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edd1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1fa59c700 1 -- 172.21.15.45:0/1610684860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec010c20 con 0x562f9edc7830 2023-10-17T01:15:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edd1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 0x562f9edd1310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec010f40 tx=0x7ff1ec01c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e2104b89b1b14d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/1610684860 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec02c070 con 0x562f9edc7830 2023-10-17T01:15:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/1610684860 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1ec017070 con 0x562f9edc7830 2023-10-17T01:15:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.040+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f9ed43cb0 con 0x562f9edc7830 2023-10-17T01:15:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.041+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/1610684860 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1ec00be60 con 0x562f9edc7830 2023-10-17T01:15:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.041+0000 7ff214a00b80 10 client.25406 fetch_fsmap learned FSMap version 26 2023-10-17T01:15:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.041+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562f9eb2fc00 con 0x562f9edc7830 2023-10-17T01:15:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.041+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/1610684860 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f9ed43cb0 con 0x562f9edc7830 2023-10-17T01:15:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/1610684860 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff1ec004ae0 con 0x562f9edc7830 2023-10-17T01:15:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff214a00b80 10 client.25406 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:15:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff214a00b80 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff214a00b80 20 client.25406 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff214a00b80 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9eddd060 0x562f9eddf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff214a00b80 4 client.25406 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f9ed96cd0 con 0x562f9eddd060 2023-10-17T01:15:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.042+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9eddd060 0x562f9eddf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.044+0000 7ff1e1ffb700 10 client.25406 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.044+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9eddd060 0x562f9eddf480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3286e9c824a1a2f0 server_cookie=6147af984d9535f7 in_seq=0 out_seq=0 2023-10-17T01:15:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/1610684860 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x562f9ed96cd0 con 0x562f9eddd060 2023-10-17T01:15:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff1e1ffb700 10 client.25406 handle_command_reply: tid=0 2023-10-17T01:15:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 client.25406 shutdown 2023-10-17T01:15:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 msgr2=0x562f9edd1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 0x562f9edd1310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec010f40 tx=0x7ff1ec01c980 comp rx=0 tx=0).stop 2023-10-17T01:15:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 shutdown_connections 2023-10-17T01:15:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8d90 0x562f9edd8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9eddd060 0x562f9eddf480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc7830 0x562f9edd1310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 --2- 172.21.15.45:0/1610684860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc8190 0x562f9edd1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff1e1ffb700 0 client.25406 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 >> 172.21.15.45:0/1610684860 conn(0x562f9ed27070 msgr2=0x562f9edc1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 shutdown_connections 2023-10-17T01:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.045+0000 7ff214a00b80 1 -- 172.21.15.45:0/1610684860 wait complete. 2023-10-17T01:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.046+0000 7ff214a00b80 20 client.25406 trim_cache size 0 max 16384 2023-10-17T01:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 Processor -- start 2023-10-17T01:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 -- start start 2023-10-17T01:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9edcf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9edcf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edcbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9edc37a0 con 0x562f9edc7830 2023-10-17T01:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9edbf040 con 0x562f9edc8d90 2023-10-17T01:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff214a00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9edc8bf0 con 0x562f9edc8190 2023-10-17T01:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9edcf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9edcf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51386/0 (socket says 172.21.15.45:51386) 2023-10-17T01:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff1fa59c700 1 -- 172.21.15.45:0/3901839021 learned_addr learned my addr 172.21.15.45:0/3901839021 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edcbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.061+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9edcf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1fa59c700 1 -- 172.21.15.45:0/3901839021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 msgr2=0x562f9edcf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9edcf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1fa59c700 1 -- 172.21.15.45:0/3901839021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 msgr2=0x562f9edcbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edcbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1fa59c700 1 -- 172.21.15.45:0/3901839021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f9eb6d1f0 con 0x562f9edc7830 2023-10-17T01:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9edcf170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec004e10 tx=0x7ff1ec010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8ad98de176ee07 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1e97fa700 1 -- 172.21.15.45:0/3901839021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec02c020 con 0x562f9edc7830 2023-10-17T01:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1e97fa700 1 -- 172.21.15.45:0/3901839021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1ec004b50 con 0x562f9edc7830 2023-10-17T01:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.062+0000 7ff1e97fa700 1 -- 172.21.15.45:0/3901839021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0055c0 con 0x562f9edc7830 2023-10-17T01:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 -- 172.21.15.45:0/3901839021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 msgr2=0x562f9edcf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9edcf170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec004e10 tx=0x7ff1ec010fd0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 -- 172.21.15.45:0/3901839021 shutdown_connections 2023-10-17T01:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9edcbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9edcf170 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 --2- 172.21.15.45:0/3901839021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9edcf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 -- 172.21.15.45:0/3901839021 >> 172.21.15.45:0/3901839021 conn(0x562f9ed27070 msgr2=0x562f9edd0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 -- 172.21.15.45:0/3901839021 shutdown_connections 2023-10-17T01:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.063+0000 7ff214a00b80 1 -- 172.21.15.45:0/3901839021 wait complete. 2023-10-17T01:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 -- start start 2023-10-17T01:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 Processor -- start 2023-10-17T01:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9ee28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9ee2aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9ee2d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ee2d9b0 con 0x562f9edc7830 2023-10-17T01:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ee2db20 con 0x562f9edc8d90 2023-10-17T01:15:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9ee2dce0 con 0x562f9edc8190 2023-10-17T01:15:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.064+0000 7ff214a00b80 10 client.0 _ll_register_callbacks cb 0x7ff1ec010e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9ee28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9ee28980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51388/0 (socket says 172.21.15.45:51388) 2023-10-17T01:15:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fa59c700 1 -- 172.21.15.45:0/115243345 learned_addr learned my addr 172.21.15.45:0/115243345 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9ee2d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1f9d9b700 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9ee2aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 -- 172.21.15.45:0/115243345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 msgr2=0x562f9ee2aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9ee2aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 -- 172.21.15.45:0/115243345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 msgr2=0x562f9ee28980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9ee28980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 -- 172.21.15.45:0/115243345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f9eb2fc00 con 0x562f9edc8d90 2023-10-17T01:15:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1fad9d700 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9ee2d420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40cb000 tx=0x7ff1f401dcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d2587be815c153 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.065+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/115243345 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f409aff0 con 0x562f9edc8d90 2023-10-17T01:15:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1ec015070 con 0x562f9edc8d90 2023-10-17T01:15:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/115243345 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1f409ab90 con 0x562f9edc8d90 2023-10-17T01:15:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/115243345 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f409e9c0 con 0x562f9edc8d90 2023-10-17T01:15:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 10 client.25412 fetch_fsmap learned FSMap version 26 2023-10-17T01:15:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562f9eb45250 con 0x562f9edc8d90 2023-10-17T01:15:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/115243345 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1ec015070 con 0x562f9edc8d90 2023-10-17T01:15:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/115243345 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff1f40a27e0 con 0x562f9edc8d90 2023-10-17T01:15:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 10 client.25412 fetch_fsmap finished waiting for FSMap version 26 2023-10-17T01:15:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 10 client.25412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 20 client.25412 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9ee2f530 0x562f9ee31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 4 client.25412 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:15:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.066+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f9ed11ce0 con 0x562f9ee2f530 2023-10-17T01:15:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.067+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9ee2f530 0x562f9ee31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.067+0000 7ff1fa59c700 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9ee2f530 0x562f9ee31950 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9e4ed4e61c9a50 server_cookie=60b283824ce73502 in_seq=0 out_seq=0 2023-10-17T01:15:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.067+0000 7ff1e1ffb700 10 client.25412 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:04.099 INFO:teuthology.orchestra.run.smithi045.stderr:2000000 of 3200000 tuples (62%) done (elapsed 4.62 s, remaining 2.77 s) 2023-10-17T01:15:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff1e1ffb700 1 -- 172.21.15.45:0/115243345 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f9ed11ce0 con 0x562f9ee2f530 2023-10-17T01:15:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff1e1ffb700 10 client.25412 handle_command_reply: tid=0 2023-10-17T01:15:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 client.25412 shutdown 2023-10-17T01:15:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 msgr2=0x562f9ee2d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9ee2d420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40cb000 tx=0x7ff1f401dcd0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 shutdown_connections 2023-10-17T01:15:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562f9ee2f530 0x562f9ee31950 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562f9edc8d90 0x562f9ee2d420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f9edc7830 0x562f9ee28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff1e1ffb700 0 client.25412 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 --2- 172.21.15.45:0/115243345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562f9edc8190 0x562f9ee2aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.295+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 >> 172.21.15.45:0/115243345 conn(0x562f9ed27070 msgr2=0x562f9edcf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.296+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 shutdown_connections 2023-10-17T01:15:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.296+0000 7ff214a00b80 1 -- 172.21.15.45:0/115243345 wait complete. 2023-10-17T01:15:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.296+0000 7ff214a00b80 20 client.25412 trim_cache size 0 max 16384 2023-10-17T01:15:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:15:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:15:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:15:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:15:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.296+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1e40a0ba0 msgr2=0x7ff1e40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.296+0000 7ff200fff700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1e40a0ba0 0x7ff1e40a3060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc198bd0 tx=0x7ff1ec00b410 comp rx=0 tx=0).stop 2023-10-17T01:15:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 msgr2=0x7ff1fc19a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 0x7ff1fc19a160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009850 tx=0x7ff1f400bce0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 shutdown_connections 2023-10-17T01:15:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1e40a0ba0 0x7ff1e40a3060 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1fc100940 0x7ff1fc19a160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0ff620 0x7ff1fc198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 --2- 172.21.15.45:0/290096356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1fc0fff80 0x7ff1fc1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 >> 172.21.15.45:0/290096356 conn(0x7ff1fc0fb1a0 msgr2=0x7ff1fc0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 shutdown_connections 2023-10-17T01:15:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:04.297+0000 7ff200fff700 1 -- 172.21.15.45:0/290096356 wait complete. 2023-10-17T01:15:04.495 INFO:teuthology.orchestra.run.smithi045.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.02 s, remaining 2.63 s) 2023-10-17T01:15:04.765 INFO:teuthology.orchestra.run.smithi045.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.29 s, remaining 2.40 s) 2023-10-17T01:15:04.926 INFO:teuthology.orchestra.run.smithi045.stderr:2300000 of 3200000 tuples (71%) done (elapsed 5.45 s, remaining 2.13 s) 2023-10-17T01:15:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2400000 of 3200000 tuples (75%) done (elapsed 5.71 s, remaining 1.90 s) 2023-10-17T01:15:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.01 s, remaining 1.68 s) 2023-10-17T01:15:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.12 s, remaining 1.41 s) 2023-10-17T01:15:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:05 smithi045 ceph-mon[98474]: pgmap v331: 97 pgs: 97 active+clean; 178 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 28 MiB/s wr, 136 op/s 2023-10-17T01:15:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:05 smithi060 ceph-mon[134136]: pgmap v331: 97 pgs: 97 active+clean; 178 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 28 MiB/s wr, 136 op/s 2023-10-17T01:15:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2700000 of 3200000 tuples (84%) done (elapsed 6.42 s, remaining 1.19 s) 2023-10-17T01:15:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:05 smithi181 ceph-mon[134520]: pgmap v331: 97 pgs: 97 active+clean; 178 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 28 MiB/s wr, 136 op/s 2023-10-17T01:15:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2800000 of 3200000 tuples (87%) done (elapsed 6.58 s, remaining 0.94 s) 2023-10-17T01:15:06.317 INFO:teuthology.orchestra.run.smithi045.stderr:2900000 of 3200000 tuples (90%) done (elapsed 6.84 s, remaining 0.71 s) 2023-10-17T01:15:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.14 s, remaining 0.48 s) 2023-10-17T01:15:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:3100000 of 3200000 tuples (96%) done (elapsed 7.32 s, remaining 0.24 s) 2023-10-17T01:15:06.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:06 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:06.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:06 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:06 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:3200000 of 3200000 tuples (100%) done (elapsed 7.58 s, remaining 0.00 s) 2023-10-17T01:15:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:vacuum... 2023-10-17T01:15:07.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:07 smithi045 ceph-mon[98474]: pgmap v332: 97 pgs: 97 active+clean; 207 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 32 MiB/s wr, 196 op/s 2023-10-17T01:15:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:07 smithi060 ceph-mon[134136]: pgmap v332: 97 pgs: 97 active+clean; 207 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 32 MiB/s wr, 196 op/s 2023-10-17T01:15:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:07 smithi181 ceph-mon[134520]: pgmap v332: 97 pgs: 97 active+clean; 207 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 32 MiB/s wr, 196 op/s 2023-10-17T01:15:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:set primary keys... 2023-10-17T01:15:08.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:08 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:15:08.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:08 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:15:08.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:08 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:15:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:09 smithi045 ceph-mon[98474]: pgmap v333: 97 pgs: 97 active+clean; 303 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 49 MiB/s wr, 265 op/s 2023-10-17T01:15:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:09 smithi060 ceph-mon[134136]: pgmap v333: 97 pgs: 97 active+clean; 303 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 49 MiB/s wr, 265 op/s 2023-10-17T01:15:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:09 smithi181 ceph-mon[134520]: pgmap v333: 97 pgs: 97 active+clean; 303 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 49 MiB/s wr, 265 op/s 2023-10-17T01:15:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:done. 2023-10-17T01:15:10.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-10-17T01:15:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:starting vacuum...end. 2023-10-17T01:15:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:11 smithi045 ceph-mon[98474]: pgmap v334: 97 pgs: 97 active+clean; 347 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 55 MiB/s wr, 309 op/s 2023-10-17T01:15:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:11 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:11.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:11 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:11 smithi060 ceph-mon[134136]: pgmap v334: 97 pgs: 97 active+clean; 347 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 55 MiB/s wr, 309 op/s 2023-10-17T01:15:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:11 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:11 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:11 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:11 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:11 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:11 smithi181 ceph-mon[134520]: pgmap v334: 97 pgs: 97 active+clean; 347 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 55 MiB/s wr, 309 op/s 2023-10-17T01:15:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:11 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:11.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:11 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:11 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:11 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:11.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:11 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:13 smithi045 ceph-mon[98474]: pgmap v335: 97 pgs: 97 active+clean; 442 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 64 MiB/s wr, 298 op/s 2023-10-17T01:15:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:13 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:13 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:13 smithi060 ceph-mon[134136]: pgmap v335: 97 pgs: 97 active+clean; 442 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 64 MiB/s wr, 298 op/s 2023-10-17T01:15:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:13 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:13 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:13 smithi181 ceph-mon[134520]: pgmap v335: 97 pgs: 97 active+clean; 442 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 64 MiB/s wr, 298 op/s 2023-10-17T01:15:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:13 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:13 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:14.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:15:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:15:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:15:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:15:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:15:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:15:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:15:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 5.0 s, 2083.7 tps, lat 44.910 ms stddev 52.976 2023-10-17T01:15:15.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:15 smithi045 ceph-mon[98474]: pgmap v336: 97 pgs: 97 active+clean; 792 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 MiB/s wr, 334 op/s 2023-10-17T01:15:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:15 smithi060 ceph-mon[134136]: pgmap v336: 97 pgs: 97 active+clean; 792 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 MiB/s wr, 334 op/s 2023-10-17T01:15:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:15 smithi181 ceph-mon[134520]: pgmap v336: 97 pgs: 97 active+clean; 792 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 MiB/s wr, 334 op/s 2023-10-17T01:15:16.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:16 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:16.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:16 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:16.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:16 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 -- 172.21.15.45:0/450752801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77780991b0 msgr2=0x7f777809b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 --2- 172.21.15.45:0/450752801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77780991b0 0x7f777809b5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f776c009990 tx=0x7f776c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 -- 172.21.15.45:0/450752801 shutdown_connections 2023-10-17T01:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 --2- 172.21.15.45:0/450752801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 0x7f777809df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 --2- 172.21.15.45:0/450752801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7778096880 0x7f7778098c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 --2- 172.21.15.45:0/450752801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77780991b0 0x7f777809b5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.547+0000 7f778599d700 1 -- 172.21.15.45:0/450752801 >> 172.21.15.45:0/450752801 conn(0x7f7778090250 msgr2=0x7f77780926b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.548+0000 7f778599d700 1 -- 172.21.15.45:0/450752801 shutdown_connections 2023-10-17T01:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.548+0000 7f778599d700 1 -- 172.21.15.45:0/450752801 wait complete. 2023-10-17T01:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.549+0000 7f778599d700 1 Processor -- start 2023-10-17T01:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 -- start start 2023-10-17T01:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7778096880 0x7f777813a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77780991b0 0x7f777813ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 0x7f7778141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778142480 con 0x7f77780991b0 2023-10-17T01:15:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77781425c0 con 0x7f7778096880 2023-10-17T01:15:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.550+0000 7f778599d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778142750 con 0x7f777809bae0 2023-10-17T01:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7778096880 0x7f777813a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7778096880 0x7f777813a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38754/0 (socket says 172.21.15.45:38754) 2023-10-17T01:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778499b700 1 -- 172.21.15.45:0/2172824033 learned_addr learned my addr 172.21.15.45:0/2172824033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778519c700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 0x7f7778141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f777ffff700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77780991b0 0x7f777813ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778519c700 1 -- 172.21.15.45:0/2172824033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7778096880 msgr2=0x7f777813a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778519c700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7778096880 0x7f777813a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778519c700 1 -- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77780991b0 msgr2=0x7f777813ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778519c700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77780991b0 0x7f777813ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.551+0000 7f778519c700 1 -- 172.21.15.45:0/2172824033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c0095f0 con 0x7f777809bae0 2023-10-17T01:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.552+0000 7f778519c700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 0x7f7778141eb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f778002d250 tx=0x7f778004f490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.552+0000 7f777dffb700 1 -- 172.21.15.45:0/2172824033 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778006e040 con 0x7f777809bae0 2023-10-17T01:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.552+0000 7f777dffb700 1 -- 172.21.15.45:0/2172824033 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7780072ab0 con 0x7f777809bae0 2023-10-17T01:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.552+0000 7f777dffb700 1 -- 172.21.15.45:0/2172824033 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778007db30 con 0x7f777809bae0 2023-10-17T01:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.552+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7778142a30 con 0x7f777809bae0 2023-10-17T01:15:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.552+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7778142ec0 con 0x7f777809bae0 2023-10-17T01:15:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.556+0000 7f777dffb700 1 -- 172.21.15.45:0/2172824033 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f77800863f0 con 0x7f777809bae0 2023-10-17T01:15:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.563+0000 7f777dffb700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f77700a4fb0 0x7f77700a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.563+0000 7f777dffb700 1 -- 172.21.15.45:0/2172824033 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f778012e720 con 0x7f777809bae0 2023-10-17T01:15:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.564+0000 7f778499b700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f77700a4fb0 0x7f77700a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.565+0000 7f778499b700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f77700a4fb0 0x7f77700a7470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7774005950 tx=0x7f777400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.569+0000 7f779a564b80 1 Processor -- start 2023-10-17T01:15:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.569+0000 7f779a564b80 1 -- start start 2023-10-17T01:15:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.570+0000 7f779a564b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c1f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.570+0000 7f779a564b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c1f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c1f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38760/0 (socket says 172.21.15.45:38760) 2023-10-17T01:15:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f778499b700 1 -- 172.21.15.45:0/1172738050 learned_addr learned my addr 172.21.15.45:0/1172738050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f779a564b80 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392b6e9b0 con 0x55d392c1fd00 2023-10-17T01:15:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392b6c370 con 0x55d392c1f3a0 2023-10-17T01:15:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.571+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392b67c60 con 0x55d392c207c0 2023-10-17T01:15:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f777ffff700 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f778519c700 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f778499b700 1 -- 172.21.15.45:0/1172738050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 msgr2=0x55d392c20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f778499b700 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c20c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f778499b700 1 -- 172.21.15.45:0/1172738050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 msgr2=0x55d392c201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f778499b700 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c201e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.572+0000 7f778499b700 1 -- 172.21.15.45:0/1172738050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d392995d80 con 0x55d392c1f3a0 2023-10-17T01:15:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.573+0000 7f778499b700 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c1f7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7774010f70 tx=0x7f7774013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174a722e53f3267c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.573+0000 7f776affd700 1 -- 172.21.15.45:0/1172738050 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774011db0 con 0x55d392c1f3a0 2023-10-17T01:15:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.573+0000 7f776affd700 1 -- 172.21.15.45:0/1172738050 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7774015070 con 0x55d392c1f3a0 2023-10-17T01:15:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.573+0000 7f776affd700 1 -- 172.21.15.45:0/1172738050 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777401c610 con 0x55d392c1f3a0 2023-10-17T01:15:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 msgr2=0x55d392c1f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c1f7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7774010f70 tx=0x7f7774013000 comp rx=0 tx=0).stop 2023-10-17T01:15:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 shutdown_connections 2023-10-17T01:15:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c1f7c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 --2- 172.21.15.45:0/1172738050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 >> 172.21.15.45:0/1172738050 conn(0x55d392b76070 msgr2=0x55d392c11500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 shutdown_connections 2023-10-17T01:15:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.574+0000 7f779a564b80 1 -- 172.21.15.45:0/1172738050 wait complete. 2023-10-17T01:15:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.575+0000 7f779a564b80 1 -- start start 2023-10-17T01:15:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.575+0000 7f779a564b80 1 Processor -- start 2023-10-17T01:15:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c207c0 0x55d392c302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392b044b0 con 0x55d392c1fd00 2023-10-17T01:15:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392b04150 con 0x55d392c207c0 2023-10-17T01:15:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3929eb8c0 con 0x55d392c1f3a0 2023-10-17T01:15:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.576+0000 7f779a564b80 10 client.0 _ll_register_callbacks cb 0x55d392a51a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c207c0 0x55d392c302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778499b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38352/0 (socket says 172.21.15.45:38352) 2023-10-17T01:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778499b700 1 -- 172.21.15.45:0/3884725857 learned_addr learned my addr 172.21.15.45:0/3884725857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f777ffff700 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778519c700 1 -- 172.21.15.45:0/3884725857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c1f3a0 msgr2=0x55d392c28d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778519c700 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778519c700 1 -- 172.21.15.45:0/3884725857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 msgr2=0x55d392c29260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778519c700 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.578+0000 7f778519c700 1 -- 172.21.15.45:0/3884725857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7774010c50 con 0x55d392c207c0 2023-10-17T01:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.579+0000 7f778519c700 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c207c0 0x55d392c302a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f778012c250 tx=0x7f77800fd480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec059f009a00fbe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.579+0000 7f775ffff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.579+0000 7f775ffff700 1 -- 172.21.15.45:0/3884725857 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780104df0 con 0x55d392c207c0 2023-10-17T01:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.579+0000 7f775ffff700 1 -- 172.21.15.45:0/3884725857 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7780115170 con 0x55d392c207c0 2023-10-17T01:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d392b92cb0 con 0x55d392c207c0 2023-10-17T01:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f775ffff700 1 -- 172.21.15.45:0/3884725857 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7780115330 con 0x55d392c207c0 2023-10-17T01:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f779a564b80 10 client.25424 fetch_fsmap learned FSMap version 29 2023-10-17T01:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d39297ec00 con 0x55d392c207c0 2023-10-17T01:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f775ffff700 1 -- 172.21.15.45:0/3884725857 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d392b92cb0 con 0x55d392c207c0 2023-10-17T01:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f775ffff700 1 -- 172.21.15.45:0/3884725857 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f77800fcfe0 con 0x55d392c207c0 2023-10-17T01:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f779a564b80 10 client.25424 fetch_fsmap finished waiting for FSMap version 29 2023-10-17T01:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.580+0000 7f779a564b80 10 client.25424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.581+0000 7f779a564b80 20 client.25424 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.581+0000 7f779a564b80 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c34bd0 0x55d392c36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.581+0000 7f779a564b80 4 client.25424 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.581+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d392be5cd0 con 0x55d392c34bd0 2023-10-17T01:15:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.581+0000 7f778499b700 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c34bd0 0x55d392c36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.583+0000 7f778499b700 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c34bd0 0x55d392c36ff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed98c2f759f0c7e5 server_cookie=c88b65994d4c076c in_seq=0 out_seq=0 2023-10-17T01:15:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.583+0000 7f775ffff700 10 client.25424 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.584+0000 7f775ffff700 1 -- 172.21.15.45:0/3884725857 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d392be5cd0 con 0x55d392c34bd0 2023-10-17T01:15:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.584+0000 7f775ffff700 10 client.25424 handle_command_reply: tid=0 2023-10-17T01:15:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.584+0000 7f779a564b80 1 client.25424 shutdown 2023-10-17T01:15:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c207c0 msgr2=0x55d392c302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c207c0 0x55d392c302a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f778012c250 tx=0x7f77800fd480 comp rx=0 tx=0).stop 2023-10-17T01:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 shutdown_connections 2023-10-17T01:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c34bd0 0x55d392c36ff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c207c0 0x55d392c302a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f775ffff700 0 client.25424 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 --2- 172.21.15.45:0/3884725857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 >> 172.21.15.45:0/3884725857 conn(0x55d392b76070 msgr2=0x55d392c1e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 shutdown_connections 2023-10-17T01:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.585+0000 7f779a564b80 1 -- 172.21.15.45:0/3884725857 wait complete. 2023-10-17T01:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.586+0000 7f779a564b80 20 client.25424 trim_cache size 0 max 16384 2023-10-17T01:15:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.619+0000 7f779a564b80 1 Processor -- start 2023-10-17T01:15:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 -- start start 2023-10-17T01:15:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392c12cd0 con 0x55d392c1fd00 2023-10-17T01:15:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392c09450 con 0x55d392c1f3a0 2023-10-17T01:15:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.620+0000 7f779a564b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392c133e0 con 0x55d392c207c0 2023-10-17T01:15:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778519c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778519c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c26b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38358/0 (socket says 172.21.15.45:38358) 2023-10-17T01:15:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778519c700 1 -- 172.21.15.45:0/3855678656 learned_addr learned my addr 172.21.15.45:0/3855678656 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f777ffff700 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778519c700 1 -- 172.21.15.45:0/3855678656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 msgr2=0x55d392c28d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778519c700 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.621+0000 7f778519c700 1 -- 172.21.15.45:0/3855678656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 msgr2=0x55d392c29260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f778519c700 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f778519c700 1 -- 172.21.15.45:0/3855678656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3929bc1f0 con 0x55d392c207c0 2023-10-17T01:15:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f777ffff700 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f778519c700 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c26b10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f778004fd60 tx=0x7f7780066e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a137230bd32bae server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f776b7fe700 1 -- 172.21.15.45:0/3855678656 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780072c20 con 0x55d392c207c0 2023-10-17T01:15:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f776b7fe700 1 -- 172.21.15.45:0/3855678656 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7780072d80 con 0x55d392c207c0 2023-10-17T01:15:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.622+0000 7f776b7fe700 1 -- 172.21.15.45:0/3855678656 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800702e0 con 0x55d392c207c0 2023-10-17T01:15:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 -- 172.21.15.45:0/3855678656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 msgr2=0x55d392c26b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c26b10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f778004fd60 tx=0x7f7780066e80 comp rx=0 tx=0).stop 2023-10-17T01:15:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 -- 172.21.15.45:0/3855678656 shutdown_connections 2023-10-17T01:15:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c26b10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c28d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 --2- 172.21.15.45:0/3855678656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c29260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 -- 172.21.15.45:0/3855678656 >> 172.21.15.45:0/3855678656 conn(0x55d392b76070 msgr2=0x55d392c30b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 -- 172.21.15.45:0/3855678656 shutdown_connections 2023-10-17T01:15:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.625+0000 7f779a564b80 1 -- 172.21.15.45:0/3855678656 wait complete. 2023-10-17T01:15:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.626+0000 7f779a564b80 1 -- start start 2023-10-17T01:15:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.626+0000 7f779a564b80 1 Processor -- start 2023-10-17T01:15:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.626+0000 7f779a564b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c80530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.626+0000 7f779a564b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c82a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f779a564b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c85070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f779a564b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392c857d0 con 0x55d392c1fd00 2023-10-17T01:15:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f779a564b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392c85940 con 0x55d392c1f3a0 2023-10-17T01:15:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f779a564b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d392c85b00 con 0x55d392c207c0 2023-10-17T01:15:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f779a564b80 10 client.0 _ll_register_callbacks cb 0x55d392a51a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c80530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f778499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c80530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38788/0 (socket says 172.21.15.45:38788) 2023-10-17T01:15:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f778499b700 1 -- 172.21.15.45:0/2808659143 learned_addr learned my addr 172.21.15.45:0/2808659143 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.627+0000 7f777ffff700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c82a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778519c700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c85070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778499b700 1 -- 172.21.15.45:0/2808659143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 msgr2=0x55d392c85070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778499b700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c85070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778499b700 1 -- 172.21.15.45:0/2808659143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 msgr2=0x55d392c82a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778499b700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c82a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778499b700 1 -- 172.21.15.45:0/2808659143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d39297ec00 con 0x55d392c1f3a0 2023-10-17T01:15:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f778499b700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c80530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7774010a40 tx=0x7f7774011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a98b13fb2d80df8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f775ffff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f775ffff700 1 -- 172.21.15.45:0/2808659143 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777402d040 con 0x55d392c1f3a0 2023-10-17T01:15:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.628+0000 7f775ffff700 1 -- 172.21.15.45:0/2808659143 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7774017070 con 0x55d392c1f3a0 2023-10-17T01:15:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.629+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7774015070 con 0x55d392c1f3a0 2023-10-17T01:15:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.629+0000 7f775ffff700 1 -- 172.21.15.45:0/2808659143 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7774005240 con 0x55d392c1f3a0 2023-10-17T01:15:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.630+0000 7f779a564b80 10 client.25430 fetch_fsmap learned FSMap version 29 2023-10-17T01:15:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.630+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d392994250 con 0x55d392c1f3a0 2023-10-17T01:15:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.630+0000 7f775ffff700 1 -- 172.21.15.45:0/2808659143 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7774015070 con 0x55d392c1f3a0 2023-10-17T01:15:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.630+0000 7f775ffff700 1 -- 172.21.15.45:0/2808659143 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f777401e420 con 0x55d392c1f3a0 2023-10-17T01:15:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.631+0000 7f779a564b80 10 client.25430 fetch_fsmap finished waiting for FSMap version 29 2023-10-17T01:15:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.631+0000 7f779a564b80 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.631+0000 7f779a564b80 20 client.25430 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.631+0000 7f779a564b80 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c873a0 0x55d392c897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.631+0000 7f779a564b80 4 client.25430 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:15:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.631+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d392b60ce0 con 0x55d392c873a0 2023-10-17T01:15:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.632+0000 7f777ffff700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c873a0 0x55d392c897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.632+0000 7f775ffff700 10 client.25430 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.632+0000 7f777ffff700 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c873a0 0x55d392c897c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e41beb7a3af139 server_cookie=21e99d3f58ff574b in_seq=0 out_seq=0 2023-10-17T01:15:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.653+0000 7f775ffff700 1 -- 172.21.15.45:0/2808659143 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d392b60ce0 con 0x55d392c873a0 2023-10-17T01:15:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.653+0000 7f775ffff700 10 client.25430 handle_command_reply: tid=0 2023-10-17T01:15:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.653+0000 7f779a564b80 1 client.25430 shutdown 2023-10-17T01:15:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.653+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 msgr2=0x55d392c80530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.653+0000 7f779a564b80 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c80530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7774010a40 tx=0x7f7774011b30 comp rx=0 tx=0).stop 2023-10-17T01:15:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 shutdown_connections 2023-10-17T01:15:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d392c207c0 0x55d392c85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d392c873a0 0x55d392c897c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d392c1f3a0 0x55d392c80530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 --2- 172.21.15.45:0/2808659143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d392c1fd00 0x55d392c82a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 >> 172.21.15.45:0/2808659143 conn(0x55d392b76070 msgr2=0x55d392c26b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f775ffff700 0 client.25430 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 shutdown_connections 2023-10-17T01:15:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 1 -- 172.21.15.45:0/2808659143 wait complete. 2023-10-17T01:15:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.654+0000 7f779a564b80 20 client.25430 trim_cache size 0 max 16384 2023-10-17T01:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f77700a4fb0 msgr2=0x7f77700a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f77700a4fb0 0x7f77700a7470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7774005950 tx=0x7f777400b500 comp rx=0 tx=0).stop 2023-10-17T01:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 msgr2=0x7f7778141eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 0x7f7778141eb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f778002d250 tx=0x7f778004f490 comp rx=0 tx=0).stop 2023-10-17T01:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 shutdown_connections 2023-10-17T01:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f77700a4fb0 0x7f77700a7470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f777809bae0 0x7f7778141eb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7778096880 0x7f777813a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 --2- 172.21.15.45:0/2172824033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77780991b0 0x7f777813ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.656+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 >> 172.21.15.45:0/2172824033 conn(0x7f7778090250 msgr2=0x7f77780974f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.657+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 shutdown_connections 2023-10-17T01:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:17.657+0000 7f778599d700 1 -- 172.21.15.45:0/2172824033 wait complete. 2023-10-17T01:15:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:17 smithi045 ceph-mon[98474]: pgmap v337: 97 pgs: 97 active+clean; 861 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 58 MiB/s wr, 350 op/s 2023-10-17T01:15:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:17 smithi060 ceph-mon[134136]: pgmap v337: 97 pgs: 97 active+clean; 861 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 58 MiB/s wr, 350 op/s 2023-10-17T01:15:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:17 smithi181 ceph-mon[134520]: pgmap v337: 97 pgs: 97 active+clean; 861 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 58 MiB/s wr, 350 op/s 2023-10-17T01:15:18.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:18 smithi060 ceph-mon[134136]: pgmap v338: 97 pgs: 97 active+clean; 989 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 65 MiB/s wr, 393 op/s 2023-10-17T01:15:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:18 smithi181 ceph-mon[134520]: pgmap v338: 97 pgs: 97 active+clean; 989 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 65 MiB/s wr, 393 op/s 2023-10-17T01:15:19.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:18 smithi045 ceph-mon[98474]: pgmap v338: 97 pgs: 97 active+clean; 989 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 65 MiB/s wr, 393 op/s 2023-10-17T01:15:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 10.0 s, 2439.6 tps, lat 40.611 ms stddev 50.230 2023-10-17T01:15:21.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:15:21.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 -- 172.21.15.45:0/1687959034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0141040f0 msgr2=0x7fc014104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 --2- 172.21.15.45:0/1687959034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0141040f0 0x7fc014104570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009f90 tx=0x7fbffc00b330 comp rx=0 tx=0).stop 2023-10-17T01:15:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 -- 172.21.15.45:0/1687959034 shutdown_connections 2023-10-17T01:15:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 --2- 172.21.15.45:0/1687959034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014104ab0 0x7fc01410b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 --2- 172.21.15.45:0/1687959034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 0x7fc014103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 --2- 172.21.15.45:0/1687959034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0141040f0 0x7fc014104570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 -- 172.21.15.45:0/1687959034 >> 172.21.15.45:0/1687959034 conn(0x7fc0140fe500 msgr2=0x7fc014100920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.728+0000 7fc0197ff700 1 -- 172.21.15.45:0/1687959034 shutdown_connections 2023-10-17T01:15:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.729+0000 7fc0197ff700 1 -- 172.21.15.45:0/1687959034 wait complete. 2023-10-17T01:15:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.730+0000 7fc0197ff700 1 Processor -- start 2023-10-17T01:15:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 -- start start 2023-10-17T01:15:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 0x7fc01419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 0x7fc01419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc014104ab0 0x7fc0141a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141a4d90 con 0x7fc014102f20 2023-10-17T01:15:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141a4ed0 con 0x7fc014104ab0 2023-10-17T01:15:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.731+0000 7fc0197ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141a5060 con 0x7fc0141040f0 2023-10-17T01:15:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 0x7fc01419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc014104ab0 0x7fc0141a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 0x7fc01419d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43360/0 (socket says 172.21.15.45:43360) 2023-10-17T01:15:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 -- 172.21.15.45:0/3158456852 learned_addr learned my addr 172.21.15.45:0/3158456852 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc012ffd700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 0x7fc01419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 -- 172.21.15.45:0/3158456852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc014104ab0 msgr2=0x7fc0141a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc014104ab0 0x7fc0141a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 -- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 msgr2=0x7fc01419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 0x7fc01419d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0127fc700 1 -- 172.21.15.45:0/3158456852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009c40 con 0x7fc0141040f0 2023-10-17T01:15:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc0137fe700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc014104ab0 0x7fc0141a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.732+0000 7fc012ffd700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 0x7fc01419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.733+0000 7fc0127fc700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 0x7fc01419d750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbffc004c60 tx=0x7fbffc004d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:21.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.733+0000 7fc00bfff700 1 -- 172.21.15.45:0/3158456852 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00e040 con 0x7fc0141040f0 2023-10-17T01:15:21.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.733+0000 7fc00bfff700 1 -- 172.21.15.45:0/3158456852 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbffc00bda0 con 0x7fc0141040f0 2023-10-17T01:15:21.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.733+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0141a52e0 con 0x7fc0141040f0 2023-10-17T01:15:21.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.733+0000 7fc00bfff700 1 -- 172.21.15.45:0/3158456852 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc005330 con 0x7fc0141040f0 2023-10-17T01:15:21.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.733+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0141a57b0 con 0x7fc0141040f0 2023-10-17T01:15:21.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.741+0000 7fc00bfff700 1 -- 172.21.15.45:0/3158456852 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fbffc005490 con 0x7fc0141040f0 2023-10-17T01:15:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.743+0000 7fc02d11fb80 1 Processor -- start 2023-10-17T01:15:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.744+0000 7fc00bfff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc0000a0bf0 0x7fc0000a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.745+0000 7fc00bfff700 1 -- 172.21.15.45:0/3158456852 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbffc0cd780 con 0x7fc0141040f0 2023-10-17T01:15:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.745+0000 7fc0137fe700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc0000a0bf0 0x7fc0000a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.746+0000 7fc02d11fb80 1 -- start start 2023-10-17T01:15:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.747+0000 7fc0137fe700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc0000a0bf0 0x7fc0000a30b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc004005950 tx=0x7fc0040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.747+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bdba0 0x55c6a06bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0615a10 con 0x55c6a06be500 2023-10-17T01:15:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a06133d0 con 0x55c6a06bf2b0 2023-10-17T01:15:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a060ecc0 con 0x55c6a06bdba0 2023-10-17T01:15:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc0137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc0137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06bf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54692/0 (socket says 172.21.15.45:54692) 2023-10-17T01:15:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.748+0000 7fc0137fe700 1 -- 172.21.15.45:0/1511666264 learned_addr learned my addr 172.21.15.45:0/1511666264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.749+0000 7fc012ffd700 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bdba0 0x55c6a06bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.749+0000 7fc012ffd700 1 -- 172.21.15.45:0/1511666264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 msgr2=0x55c6a06bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.749+0000 7fc012ffd700 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.749+0000 7fc012ffd700 1 -- 172.21.15.45:0/1511666264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 msgr2=0x55c6a06bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.749+0000 7fc012ffd700 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.749+0000 7fc012ffd700 1 -- 172.21.15.45:0/1511666264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a043cde0 con 0x55c6a06bdba0 2023-10-17T01:15:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.750+0000 7fc012ffd700 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bdba0 0x55c6a06bdfc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005fd0 tx=0x7fc00c0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0acc36059ee5258 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.750+0000 7fc0097fa700 1 -- 172.21.15.45:0/1511666264 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c011c00 con 0x55c6a06bdba0 2023-10-17T01:15:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.751+0000 7fc0097fa700 1 -- 172.21.15.45:0/1511666264 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc00c0044f0 con 0x55c6a06bdba0 2023-10-17T01:15:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.751+0000 7fc0097fa700 1 -- 172.21.15.45:0/1511666264 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c005280 con 0x55c6a06bdba0 2023-10-17T01:15:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.753+0000 7fc02d11fb80 1 -- 172.21.15.45:0/1511666264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bdba0 msgr2=0x55c6a06bdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.753+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bdba0 0x55c6a06bdfc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005fd0 tx=0x7fc00c0113e0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 -- 172.21.15.45:0/1511666264 shutdown_connections 2023-10-17T01:15:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bdba0 0x55c6a06bdfc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/1511666264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 -- 172.21.15.45:0/1511666264 >> 172.21.15.45:0/1511666264 conn(0x55c6a061d0d0 msgr2=0x55c6a06b8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 -- 172.21.15.45:0/1511666264 shutdown_connections 2023-10-17T01:15:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.755+0000 7fc02d11fb80 1 -- 172.21.15.45:0/1511666264 wait complete. 2023-10-17T01:15:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.757+0000 7fc02d11fb80 1 -- start start 2023-10-17T01:15:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.758+0000 7fc02d11fb80 1 Processor -- start 2023-10-17T01:15:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 0x55c6a06c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a06cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a05ab510 con 0x55c6a06be500 2023-10-17T01:15:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a05ab1b0 con 0x55c6a06bdba0 2023-10-17T01:15:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0492920 con 0x55c6a06bf2b0 2023-10-17T01:15:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.760+0000 7fc02d11fb80 10 client.0 _ll_register_callbacks cb 0x55c6a04f8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 0x55c6a06c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc0137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a06cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 0x55c6a06c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54698/0 (socket says 172.21.15.45:54698) 2023-10-17T01:15:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 -- 172.21.15.45:0/792306178 learned_addr learned my addr 172.21.15.45:0/792306178 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 -- 172.21.15.45:0/792306178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 msgr2=0x55c6a06cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a06cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 -- 172.21.15.45:0/792306178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 msgr2=0x55c6a06c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.761+0000 7fc012ffd700 1 -- 172.21.15.45:0/792306178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c009920 con 0x55c6a06bdba0 2023-10-17T01:15:21.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.764+0000 7fc012ffd700 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 0x55c6a06c7a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc00c011730 tx=0x7fc00c004fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99a0a1879b44027 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:21.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.764+0000 7fbffa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.764+0000 7fbffa7fc700 1 -- 172.21.15.45:0/792306178 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c021070 con 0x55c6a06bdba0 2023-10-17T01:15:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.764+0000 7fbffa7fc700 1 -- 172.21.15.45:0/792306178 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc00c005380 con 0x55c6a06bdba0 2023-10-17T01:15:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.765+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a0639d10 con 0x55c6a06bdba0 2023-10-17T01:15:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.765+0000 7fbffa7fc700 1 -- 172.21.15.45:0/792306178 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc00c005540 con 0x55c6a06bdba0 2023-10-17T01:15:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.765+0000 7fc02d11fb80 10 client.25442 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.765+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6a0425c60 con 0x55c6a06bdba0 2023-10-17T01:15:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.765+0000 7fbffa7fc700 1 -- 172.21.15.45:0/792306178 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6a0639d10 con 0x55c6a06bdba0 2023-10-17T01:15:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.765+0000 7fbffa7fc700 1 -- 172.21.15.45:0/792306178 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc00c012b10 con 0x55c6a06bdba0 2023-10-17T01:15:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc02d11fb80 10 client.25442 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc02d11fb80 10 client.25442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc02d11fb80 20 client.25442 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a06d3650 0x55c6a06d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc02d11fb80 4 client.25442 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a068cd30 con 0x55c6a06d3650 2023-10-17T01:15:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.766+0000 7fc0127fc700 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a06d3650 0x55c6a06d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.767+0000 7fc0127fc700 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a06d3650 0x55c6a06d5a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb83ed78b6ab641b server_cookie=770d2af6a6cc92c3 in_seq=0 out_seq=0 2023-10-17T01:15:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.767+0000 7fbffa7fc700 10 client.25442 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.767+0000 7fbffa7fc700 1 -- 172.21.15.45:0/792306178 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55c6a068cd30 con 0x55c6a06d3650 2023-10-17T01:15:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.767+0000 7fbffa7fc700 10 client.25442 handle_command_reply: tid=0 2023-10-17T01:15:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.767+0000 7fc02d11fb80 1 client.25442 shutdown 2023-10-17T01:15:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 msgr2=0x55c6a06c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 0x55c6a06c7a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc00c011730 tx=0x7fc00c004fd0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 shutdown_connections 2023-10-17T01:15:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a06cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a06d3650 0x55c6a06d5a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bdba0 0x55c6a06c7a00 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc00c011730 tx=0x7fc00c004fd0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fbffa7fc700 0 client.25442 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/792306178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 >> 172.21.15.45:0/792306178 conn(0x55c6a061d0d0 msgr2=0x55c6a06b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 shutdown_connections 2023-10-17T01:15:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.768+0000 7fc02d11fb80 1 -- 172.21.15.45:0/792306178 wait complete. 2023-10-17T01:15:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.770+0000 7fc02d11fb80 20 client.25442 trim_cache size 0 max 16384 2023-10-17T01:15:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:21 smithi045 ceph-mon[98474]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 57 MiB/s wr, 436 op/s 2023-10-17T01:15:21.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:21 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.806+0000 7fc02d11fb80 1 Processor -- start 2023-10-17T01:15:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.806+0000 7fc02d11fb80 1 -- start start 2023-10-17T01:15:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06ccf30 0x55c6a06c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a06bebb0 con 0x55c6a06be500 2023-10-17T01:15:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a06c6220 con 0x55c6a06bf2b0 2023-10-17T01:15:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a06c8bf0 con 0x55c6a06ccf30 2023-10-17T01:15:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.807+0000 7fc0137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06ccf30 0x55c6a06c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06ccf30 0x55c6a06c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43388/0 (socket says 172.21.15.45:43388) 2023-10-17T01:15:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0137fe700 1 -- 172.21.15.45:0/4238742351 learned_addr learned my addr 172.21.15.45:0/4238742351 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0127fc700 1 -- 172.21.15.45:0/4238742351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06ccf30 msgr2=0x55c6a06c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0127fc700 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06ccf30 0x55c6a06c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0127fc700 1 -- 172.21.15.45:0/4238742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 msgr2=0x55c6a06c65e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:21.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc012ffd700 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.808+0000 7fc0127fc700 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.809+0000 7fc0127fc700 1 -- 172.21.15.45:0/4238742351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a0463250 con 0x55c6a06bf2b0 2023-10-17T01:15:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.809+0000 7fc0127fc700 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06c2300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55c6a06be900 tx=0x7fbffc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d761a5caba143b19 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.809+0000 7fc009ffb700 1 -- 172.21.15.45:0/4238742351 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc09bea0 con 0x55c6a06bf2b0 2023-10-17T01:15:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.809+0000 7fc009ffb700 1 -- 172.21.15.45:0/4238742351 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbffc09ae90 con 0x55c6a06bf2b0 2023-10-17T01:15:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.809+0000 7fc009ffb700 1 -- 172.21.15.45:0/4238742351 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc09b0f0 con 0x55c6a06bf2b0 2023-10-17T01:15:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.810+0000 7fc02d11fb80 1 -- 172.21.15.45:0/4238742351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 msgr2=0x55c6a06c2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.810+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06c2300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55c6a06be900 tx=0x7fbffc009920 comp rx=0 tx=0).stop 2023-10-17T01:15:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 -- 172.21.15.45:0/4238742351 shutdown_connections 2023-10-17T01:15:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06ccf30 0x55c6a06c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06be500 0x55c6a06c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/4238742351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06bf2b0 0x55c6a06c2300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 -- 172.21.15.45:0/4238742351 >> 172.21.15.45:0/4238742351 conn(0x55c6a061d0d0 msgr2=0x55c6a06cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 -- 172.21.15.45:0/4238742351 shutdown_connections 2023-10-17T01:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.811+0000 7fc02d11fb80 1 -- 172.21.15.45:0/4238742351 wait complete. 2023-10-17T01:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.812+0000 7fc02d11fb80 1 -- start start 2023-10-17T01:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.812+0000 7fc02d11fb80 1 Processor -- start 2023-10-17T01:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.814+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 0x55c6a071d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.814+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a071d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.814+0000 7fc02d11fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06ccf30 0x55c6a07249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0724f00 con 0x55c6a06ccf30 2023-10-17T01:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 0x55c6a071d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a0725040 con 0x55c6a06be500 2023-10-17T01:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc02d11fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a07251d0 con 0x55c6a06bf2b0 2023-10-17T01:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 0x55c6a071d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54722/0 (socket says 172.21.15.45:54722) 2023-10-17T01:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 -- 172.21.15.45:0/3879786418 learned_addr learned my addr 172.21.15.45:0/3879786418 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc02d11fb80 10 client.0 _ll_register_callbacks cb 0x7fc004015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc0127fc700 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a071d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 -- 172.21.15.45:0/3879786418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 msgr2=0x55c6a071d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a071d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 -- 172.21.15.45:0/3879786418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06ccf30 msgr2=0x55c6a07249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06ccf30 0x55c6a07249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.815+0000 7fc012ffd700 1 -- 172.21.15.45:0/3879786418 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0141a57b0 con 0x55c6a06be500 2023-10-17T01:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.816+0000 7fc012ffd700 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 0x55c6a071d440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e680 tx=0x7fc00c01a460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edaf239ed3de6ee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.816+0000 7fbffa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.816+0000 7fbffa7fc700 1 -- 172.21.15.45:0/3879786418 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c021070 con 0x55c6a06be500 2023-10-17T01:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.816+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc00c0044f0 con 0x55c6a06be500 2023-10-17T01:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.816+0000 7fbffa7fc700 1 -- 172.21.15.45:0/3879786418 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc00c000e40 con 0x55c6a06be500 2023-10-17T01:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.817+0000 7fbffa7fc700 1 -- 172.21.15.45:0/3879786418 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc00c009390 con 0x55c6a06be500 2023-10-17T01:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.817+0000 7fc02d11fb80 10 client.25448 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.817+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6a043b2b0 con 0x55c6a06be500 2023-10-17T01:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.817+0000 7fbffa7fc700 1 -- 172.21.15.45:0/3879786418 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc00c041020 con 0x55c6a06be500 2023-10-17T01:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fbffa7fc700 1 -- 172.21.15.45:0/3879786418 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc00c004950 con 0x55c6a06be500 2023-10-17T01:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc02d11fb80 10 client.25448 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc02d11fb80 10 client.25448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc02d11fb80 20 client.25448 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a0726870 0x55c6a0728c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc02d11fb80 4 client.25448 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6a0607d40 con 0x55c6a0726870 2023-10-17T01:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.818+0000 7fc0127fc700 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a0726870 0x55c6a0728c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.819+0000 7fc0127fc700 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a0726870 0x55c6a0728c90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd858f76a1082cfe server_cookie=a558454af6fc1f79 in_seq=0 out_seq=0 2023-10-17T01:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.819+0000 7fbffa7fc700 10 client.25448 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.820+0000 7fbffa7fc700 1 -- 172.21.15.45:0/3879786418 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c6a0607d40 con 0x55c6a0726870 2023-10-17T01:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.820+0000 7fbffa7fc700 10 client.25448 handle_command_reply: tid=0 2023-10-17T01:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.820+0000 7fc02d11fb80 1 client.25448 shutdown 2023-10-17T01:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.820+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 msgr2=0x55c6a071d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.820+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 0x55c6a071d440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e680 tx=0x7fc00c01a460 comp rx=0 tx=0).stop 2023-10-17T01:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 shutdown_connections 2023-10-17T01:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6a06ccf30 0x55c6a07249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c6a0726870 0x55c6a0728c90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fbffa7fc700 0 client.25448 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6a06be500 0x55c6a071d440 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e680 tx=0x7fc00c01a460 comp rx=0 tx=0).stop 2023-10-17T01:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 --2- 172.21.15.45:0/3879786418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6a06bf2b0 0x55c6a071d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 >> 172.21.15.45:0/3879786418 conn(0x55c6a061d0d0 msgr2=0x55c6a06c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.821+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 shutdown_connections 2023-10-17T01:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.822+0000 7fc02d11fb80 1 -- 172.21.15.45:0/3879786418 wait complete. 2023-10-17T01:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.822+0000 7fc02d11fb80 20 client.25448 trim_cache size 0 max 16384 2023-10-17T01:15:21.832 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:15:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2023-10-17T01:15:21.832 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2023-10-17T01:15:21.832 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.823+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc0000a0bf0 msgr2=0x7fc0000a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.823+0000 7fc0197ff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc0000a0bf0 0x7fc0000a30b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc004005950 tx=0x7fc0040058e0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.823+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 msgr2=0x7fc01419d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.823+0000 7fc0197ff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 0x7fc01419d750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbffc004c60 tx=0x7fbffc004d40 comp rx=0 tx=0).stop 2023-10-17T01:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 shutdown_connections 2023-10-17T01:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc0000a0bf0 0x7fc0000a30b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc014104ab0 0x7fc0141a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014102f20 0x7fc01419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 --2- 172.21.15.45:0/3158456852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0141040f0 0x7fc01419d750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 >> 172.21.15.45:0/3158456852 conn(0x7fc0140fe500 msgr2=0x7fc014100730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 shutdown_connections 2023-10-17T01:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:21.824+0000 7fc0197ff700 1 -- 172.21.15.45:0/3158456852 wait complete. 2023-10-17T01:15:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:21 smithi060 ceph-mon[134136]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 57 MiB/s wr, 436 op/s 2023-10-17T01:15:21.836 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:21 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:21.858 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-17T01:15:21.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:15:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:21 smithi181 ceph-mon[134520]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 57 MiB/s wr, 436 op/s 2023-10-17T01:15:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:21 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.225+0000 7fe7743ff700 1 -- 172.21.15.45:0/2801311351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 msgr2=0x7fe76c117ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.225+0000 7fe7743ff700 1 --2- 172.21.15.45:0/2801311351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 0x7fe76c117ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe764009f90 tx=0x7fe76400b330 comp rx=0 tx=0).stop 2023-10-17T01:15:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 -- 172.21.15.45:0/2801311351 shutdown_connections 2023-10-17T01:15:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 --2- 172.21.15.45:0/2801311351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 0x7fe76c117ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 --2- 172.21.15.45:0/2801311351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c06a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 --2- 172.21.15.45:0/2801311351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe76c06aab0 0x7fe76c06af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 -- 172.21.15.45:0/2801311351 >> 172.21.15.45:0/2801311351 conn(0x7fe76c06fcf0 msgr2=0x7fe76c072110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 -- 172.21.15.45:0/2801311351 shutdown_connections 2023-10-17T01:15:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.226+0000 7fe7743ff700 1 -- 172.21.15.45:0/2801311351 wait complete. 2023-10-17T01:15:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.228+0000 7fe7743ff700 1 Processor -- start 2023-10-17T01:15:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.228+0000 7fe7743ff700 1 -- start start 2023-10-17T01:15:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.229+0000 7fe7743ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe76c06aab0 0x7fe76c1a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.229+0000 7fe7743ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.229+0000 7fe7743ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 0x7fe76c1accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.229+0000 7fe7743ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1ad290 con 0x7fe76c06aab0 2023-10-17T01:15:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.229+0000 7fe7743ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1ad3d0 con 0x7fe76c06c090 2023-10-17T01:15:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.229+0000 7fe7743ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c1ad560 con 0x7fe76c115b00 2023-10-17T01:15:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c1a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c1a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54730/0 (socket says 172.21.15.45:54730) 2023-10-17T01:15:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 -- 172.21.15.45:0/922877182 learned_addr learned my addr 172.21.15.45:0/922877182 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77219b700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe76c06aab0 0x7fe76c1a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77299c700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 0x7fe76c1accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 -- 172.21.15.45:0/922877182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 msgr2=0x7fe76c1accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:22.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 0x7fe76c1accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 -- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe76c06aab0 msgr2=0x7fe76c1a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe76c06aab0 0x7fe76c1a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.230+0000 7fe77199a700 1 -- 172.21.15.45:0/922877182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764009c40 con 0x7fe76c06c090 2023-10-17T01:15:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.231+0000 7fe77199a700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c1a5c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c990 tx=0x7fe75c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.231+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007b00 con 0x7fe76c06c090 2023-10-17T01:15:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.231+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75c007c60 con 0x7fe76c06c090 2023-10-17T01:15:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.231+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c018660 con 0x7fe76c06c090 2023-10-17T01:15:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.231+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c1ad840 con 0x7fe76c06c090 2023-10-17T01:15:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.231+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c1add00 con 0x7fe76c06c090 2023-10-17T01:15:22.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.233+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750005320 con 0x7fe76c06c090 2023-10-17T01:15:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.242+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fe75c004030 con 0x7fe76c06c090 2023-10-17T01:15:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.245+0000 7fe7637fe700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7580a0bf0 0x7fe7580a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.245+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe75c0cc460 con 0x7fe76c06c090 2023-10-17T01:15:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.246+0000 7fe77219b700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7580a0bf0 0x7fe7580a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:22.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.247+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe75c0cc940 con 0x7fe76c06c090 2023-10-17T01:15:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.248+0000 7fe77219b700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7580a0bf0 0x7fe7580a30b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe768005fd0 tx=0x7fe768009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:22.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.716+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe750005190 con 0x7fe76c06c090 2023-10-17T01:15:22.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.717+0000 7fe7637fe700 1 -- 172.21.15.45:0/922877182 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10830 (secure 0 0 0) 0x7fe75c095ea0 con 0x7fe76c06c090 2023-10-17T01:15:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:15:22.720 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:15:20.459102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:15:22.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.720+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7580a0bf0 msgr2=0x7fe7580a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:22.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.720+0000 7fe7743ff700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7580a0bf0 0x7fe7580a30b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe768005fd0 tx=0x7fe768009310 comp rx=0 tx=0).stop 2023-10-17T01:15:22.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.720+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 msgr2=0x7fe76c1a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:22.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.720+0000 7fe7743ff700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c1a5c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c990 tx=0x7fe75c00cd50 comp rx=0 tx=0).stop 2023-10-17T01:15:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 shutdown_connections 2023-10-17T01:15:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe7580a0bf0 0x7fe7580a30b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe76c115b00 0x7fe76c1accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe76c06aab0 0x7fe76c1a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 --2- 172.21.15.45:0/922877182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe76c06c090 0x7fe76c1a5c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 >> 172.21.15.45:0/922877182 conn(0x7fe76c06fcf0 msgr2=0x7fe76c117c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 shutdown_connections 2023-10-17T01:15:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:22.721+0000 7fe7743ff700 1 -- 172.21.15.45:0/922877182 wait complete. 2023-10-17T01:15:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2023-10-17T01:15:22.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-17T01:15:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.116+0000 7fb43cf9d700 1 -- 172.21.15.45:0/704271994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 msgr2=0x7fb4300945b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.116+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/704271994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb4300945b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0098f0 tx=0x7fb42c009c00 comp rx=0 tx=0).stop 2023-10-17T01:15:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.117+0000 7fb43cf9d700 1 -- 172.21.15.45:0/704271994 shutdown_connections 2023-10-17T01:15:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.117+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/704271994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb430097ba0 0x7fb43009e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.117+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/704271994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb4300945b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.117+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/704271994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb430097200 0x7fb430097660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.117+0000 7fb43cf9d700 1 -- 172.21.15.45:0/704271994 >> 172.21.15.45:0/704271994 conn(0x7fb43008fa80 msgr2=0x7fb430091ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.117+0000 7fb43cf9d700 1 -- 172.21.15.45:0/704271994 shutdown_connections 2023-10-17T01:15:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.118+0000 7fb43cf9d700 1 -- 172.21.15.45:0/704271994 wait complete. 2023-10-17T01:15:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.119+0000 7fb43cf9d700 1 Processor -- start 2023-10-17T01:15:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.120+0000 7fb43cf9d700 1 -- start start 2023-10-17T01:15:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.122+0000 7fb43cf9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb43012de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.122+0000 7fb43cf9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb430097200 0x7fb43012e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.122+0000 7fb43cf9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb430097ba0 0x7fb430135420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.122+0000 7fb43cf9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4301359f0 con 0x7fb430097ba0 2023-10-17T01:15:23.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.122+0000 7fb43cf9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430135b30 con 0x7fb430097200 2023-10-17T01:15:23.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.122+0000 7fb43cf9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430135cc0 con 0x7fb430094190 2023-10-17T01:15:23.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb43012de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb43012de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43438/0 (socket says 172.21.15.45:43438) 2023-10-17T01:15:23.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 -- 172.21.15.45:0/823034460 learned_addr learned my addr 172.21.15.45:0/823034460 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb436ffd700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb430097200 0x7fb43012e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 -- 172.21.15.45:0/823034460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb430097200 msgr2=0x7fb43012e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb430097200 0x7fb43012e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 -- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb430097ba0 msgr2=0x7fb430135420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb430097ba0 0x7fb430135420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.123+0000 7fb4377fe700 1 -- 172.21.15.45:0/823034460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009520 con 0x7fb430094190 2023-10-17T01:15:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.124+0000 7fb4377fe700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb43012de70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009230 tx=0x7fb42c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.124+0000 7fb434ff9700 1 -- 172.21.15.45:0/823034460 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c01c070 con 0x7fb430094190 2023-10-17T01:15:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.124+0000 7fb434ff9700 1 -- 172.21.15.45:0/823034460 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42c00bbc0 con 0x7fb430094190 2023-10-17T01:15:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.124+0000 7fb434ff9700 1 -- 172.21.15.45:0/823034460 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c003e10 con 0x7fb430094190 2023-10-17T01:15:23.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.124+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb430098420 con 0x7fb430094190 2023-10-17T01:15:23.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.125+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430098880 con 0x7fb430094190 2023-10-17T01:15:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.135+0000 7fb451bc7b80 1 Processor -- start 2023-10-17T01:15:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.135+0000 7fb434ff9700 1 -- 172.21.15.45:0/823034460 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb42c017400 con 0x7fb430094190 2023-10-17T01:15:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 -- start start 2023-10-17T01:15:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 0x55f57524e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524e660 0x55f57524ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57524f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5751a5a10 con 0x55f57524f490 2023-10-17T01:15:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5751a33d0 con 0x55f57524e660 2023-10-17T01:15:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.136+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57519ecc0 con 0x55f57524dd00 2023-10-17T01:15:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.137+0000 7fb437fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 0x55f57524e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.137+0000 7fb437fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 0x55f57524e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43442/0 (socket says 172.21.15.45:43442) 2023-10-17T01:15:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.137+0000 7fb437fff700 1 -- 172.21.15.45:0/1349745206 learned_addr learned my addr 172.21.15.45:0/1349745206 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.137+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524e660 0x55f57524ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.137+0000 7fb436ffd700 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57524f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb4377fe700 1 -- 172.21.15.45:0/1349745206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 msgr2=0x55f57524e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 0x55f57524e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb4377fe700 1 -- 172.21.15.45:0/1349745206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 msgr2=0x55f57524f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57524f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb4377fe700 1 -- 172.21.15.45:0/1349745206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f574fccde0 con 0x55f57524e660 2023-10-17T01:15:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb434ff9700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb4280a0bf0 0x7fb4280a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb437fff700 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 0x55f57524e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.138+0000 7fb434ff9700 1 -- 172.21.15.45:0/823034460 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb42c0cf1e0 con 0x7fb430094190 2023-10-17T01:15:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.139+0000 7fb437fff700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb4280a0bf0 0x7fb4280a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.139+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524e660 0x55f57524ef00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb430135e50 tx=0x7fb42c0cfeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6347ce14f137bd81 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.140+0000 7fb421ffb700 1 -- 172.21.15.45:0/1349745206 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c096030 con 0x55f57524e660 2023-10-17T01:15:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.140+0000 7fb421ffb700 1 -- 172.21.15.45:0/1349745206 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42c096740 con 0x55f57524e660 2023-10-17T01:15:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.140+0000 7fb421ffb700 1 -- 172.21.15.45:0/1349745206 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c099d60 con 0x55f57524e660 2023-10-17T01:15:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1349745206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524e660 msgr2=0x55f57524ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524e660 0x55f57524ef00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb430135e50 tx=0x7fb42c0cfeb0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb437fff700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb4280a0bf0 0x7fb4280a30b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4380503f0 tx=0x7fb43806a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1349745206 shutdown_connections 2023-10-17T01:15:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57524f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524dd00 0x55f57524e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1349745206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524e660 0x55f57524ef00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.141+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1349745206 >> 172.21.15.45:0/1349745206 conn(0x55f5751ad0d0 msgr2=0x55f575248aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.142+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1349745206 shutdown_connections 2023-10-17T01:15:23.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.142+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1349745206 wait complete. 2023-10-17T01:15:23.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.142+0000 7fb451bc7b80 1 -- start start 2023-10-17T01:15:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.143+0000 7fb451bc7b80 1 Processor -- start 2023-10-17T01:15:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.143+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524dd00 0x55f5752576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.143+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f575257c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.143+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57525ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f575257c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f575257c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43446/0 (socket says 172.21.15.45:43446) 2023-10-17T01:15:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb436ffd700 1 -- 172.21.15.45:0/3317836230 learned_addr learned my addr 172.21.15.45:0/3317836230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524dd00 0x55f5752576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57513b510 con 0x55f57524f490 2023-10-17T01:15:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57513b1b0 con 0x55f57524dd00 2023-10-17T01:15:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575022920 con 0x55f57524e660 2023-10-17T01:15:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb437fff700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57525ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb451bc7b80 10 client.0 _ll_register_callbacks cb 0x55f575088a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:23.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 -- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 msgr2=0x55f575257c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f575257c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 -- 172.21.15.45:0/3317836230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 msgr2=0x55f57525ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57525ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43806a260 con 0x55f57524dd00 2023-10-17T01:15:23.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5751c9d10 con 0x55f57524dd00 2023-10-17T01:15:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb437fff700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57525ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb4377fe700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524dd00 0x55f5752576e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0968b0 tx=0x7fb42c003860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df762e7991f552f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.144+0000 7fb416ffd700 10 client.25466 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.145+0000 7fb416ffd700 1 -- 172.21.15.45:0/3317836230 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c099d60 con 0x55f57524dd00 2023-10-17T01:15:23.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.145+0000 7fb416ffd700 1 -- 172.21.15.45:0/3317836230 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb42c003680 con 0x55f57524dd00 2023-10-17T01:15:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.145+0000 7fb416ffd700 1 -- 172.21.15.45:0/3317836230 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb42c0cfad0 con 0x55f57524dd00 2023-10-17T01:15:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.145+0000 7fb451bc7b80 10 client.25466 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.145+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x7fb430098880 con 0x55f57524dd00 2023-10-17T01:15:23.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.145+0000 7fb416ffd700 1 -- 172.21.15.45:0/3317836230 <== mon.2 v2:172.21.15.60:3300/0 4 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb42c096b10 con 0x55f57524dd00 2023-10-17T01:15:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb451bc7b80 10 client.25466 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb451bc7b80 10 client.25466 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-10-17T01:15:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb451bc7b80 20 client.25466 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f575262d50 0x55f575265170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb451bc7b80 4 client.25466 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 --> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f57521cd30 con 0x55f575262d50 2023-10-17T01:15:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.146+0000 7fb436ffd700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f575262d50 0x55f575265170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.147+0000 7fb436ffd700 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f575262d50 0x55f575265170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f04729a25ee3f913 server_cookie=949119bbb21f0848 in_seq=0 out_seq=0 2023-10-17T01:15:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.147+0000 7fb416ffd700 10 client.25466 ms_handle_connect on v2:172.21.15.181:6834/1888494346 2023-10-17T01:15:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.147+0000 7fb416ffd700 1 -- 172.21.15.45:0/3317836230 <== mds.4 v2:172.21.15.181:6834/1888494346 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55f57521cd30 con 0x55f575262d50 2023-10-17T01:15:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.147+0000 7fb416ffd700 10 client.25466 handle_command_reply: tid=0 2023-10-17T01:15:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 client.25466 shutdown 2023-10-17T01:15:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524dd00 msgr2=0x55f5752576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524dd00 0x55f5752576e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0968b0 tx=0x7fb42c003860 comp rx=0 tx=0).stop 2023-10-17T01:15:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 shutdown_connections 2023-10-17T01:15:23.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f575262d50 0x55f575265170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524f490 0x55f57525ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524dd00 0x55f5752576e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/3317836230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f575257c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 >> 172.21.15.45:0/3317836230 conn(0x55f5751ad0d0 msgr2=0x55f575248180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb416ffd700 0 client.25466 ms_handle_reset on v2:172.21.15.181:6834/1888494346 2023-10-17T01:15:23.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.148+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 shutdown_connections 2023-10-17T01:15:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.149+0000 7fb451bc7b80 1 -- 172.21.15.45:0/3317836230 wait complete. 2023-10-17T01:15:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.149+0000 7fb451bc7b80 20 client.25466 trim_cache size 0 max 16384 2023-10-17T01:15:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.185+0000 7fb451bc7b80 1 Processor -- start 2023-10-17T01:15:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.185+0000 7fb451bc7b80 1 -- start start 2023-10-17T01:15:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524e660 0x55f575252320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524f490 0x55f575252860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f5752449d0 0x55f575255810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575256bc0 con 0x55f57524e660 2023-10-17T01:15:23.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57521cd30 con 0x55f5752449d0 2023-10-17T01:15:23.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57524ed90 con 0x55f57524f490 2023-10-17T01:15:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524f490 0x55f575252860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.186+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524f490 0x55f575252860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43458/0 (socket says 172.21.15.45:43458) 2023-10-17T01:15:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.187+0000 7fb436ffd700 1 -- 172.21.15.45:0/1814204437 learned_addr learned my addr 172.21.15.45:0/1814204437 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.187+0000 7fb437fff700 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f5752449d0 0x55f575255810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.187+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524e660 0x55f575252320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb437fff700 1 -- 172.21.15.45:0/1814204437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524f490 msgr2=0x55f575252860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb437fff700 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524f490 0x55f575252860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb437fff700 1 -- 172.21.15.45:0/1814204437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524e660 msgr2=0x55f575252320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb437fff700 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524e660 0x55f575252320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb437fff700 1 -- 172.21.15.45:0/1814204437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f574fb5c60 con 0x55f5752449d0 2023-10-17T01:15:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb437fff700 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f5752449d0 0x55f575255810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb4380502d0 tx=0x7fb438080000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1a969c4ec5130d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb4227fc700 1 -- 172.21.15.45:0/1814204437 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43807fb70 con 0x55f5752449d0 2023-10-17T01:15:23.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb4227fc700 1 -- 172.21.15.45:0/1814204437 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43807fcd0 con 0x55f5752449d0 2023-10-17T01:15:23.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.188+0000 7fb4227fc700 1 -- 172.21.15.45:0/1814204437 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4380883f0 con 0x55f5752449d0 2023-10-17T01:15:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1814204437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f5752449d0 msgr2=0x55f575255810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f5752449d0 0x55f575255810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb4380502d0 tx=0x7fb438080000 comp rx=0 tx=0).stop 2023-10-17T01:15:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1814204437 shutdown_connections 2023-10-17T01:15:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f5752449d0 0x55f575255810 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57524e660 0x55f575252320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1814204437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524f490 0x55f575252860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1814204437 >> 172.21.15.45:0/1814204437 conn(0x55f5751ad0d0 msgr2=0x55f57525f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1814204437 shutdown_connections 2023-10-17T01:15:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.189+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1814204437 wait complete. 2023-10-17T01:15:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.190+0000 7fb451bc7b80 1 -- start start 2023-10-17T01:15:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.190+0000 7fb451bc7b80 1 Processor -- start 2023-10-17T01:15:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5752449d0 0x55f57525bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f57525c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb451bc7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524f490 0x55f5752b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57525c730 con 0x55f5752449d0 2023-10-17T01:15:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5752b4c50 con 0x55f57524f490 2023-10-17T01:15:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb451bc7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5752b4e10 con 0x55f57524e660 2023-10-17T01:15:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f57525c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f57525c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43460/0 (socket says 172.21.15.45:43460) 2023-10-17T01:15:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.191+0000 7fb436ffd700 1 -- 172.21.15.45:0/1842125658 learned_addr learned my addr 172.21.15.45:0/1842125658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb436ffd700 1 -- 172.21.15.45:0/1842125658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524f490 msgr2=0x55f5752b46e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:15:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb436ffd700 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524f490 0x55f5752b46e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb436ffd700 1 -- 172.21.15.45:0/1842125658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5752449d0 msgr2=0x55f57525bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb436ffd700 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5752449d0 0x55f57525bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb436ffd700 1 -- 172.21.15.45:0/1842125658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43807dcb0 con 0x55f57524e660 2023-10-17T01:15:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb436ffd700 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f57525c000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55f575265a80 tx=0x7fb42400a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=392245fd0dba76b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb416ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.192+0000 7fb416ffd700 1 -- 172.21.15.45:0/1842125658 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424004a30 con 0x55f57524e660 2023-10-17T01:15:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.193+0000 7fb416ffd700 1 -- 172.21.15.45:0/1842125658 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb424004b90 con 0x55f57524e660 2023-10-17T01:15:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.193+0000 7fb416ffd700 10 client.0 inactive, discarding osd_map(85..85 src has 1..85) v4 2023-10-17T01:15:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.193+0000 7fb451bc7b80 10 client.0 _ll_register_callbacks cb 0x7fb438067000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.193+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb42c096740 con 0x55f57524e660 2023-10-17T01:15:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.193+0000 7fb416ffd700 1 -- 172.21.15.45:0/1842125658 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb424004d50 con 0x55f57524e660 2023-10-17T01:15:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 10 client.25474 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x55f574ff3250 con 0x55f57524e660 2023-10-17T01:15:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb416ffd700 1 -- 172.21.15.45:0/1842125658 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb424004eb0 con 0x55f57524e660 2023-10-17T01:15:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 10 client.25474 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-10-17T01:15:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 20 client.25474 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f5752b6480 0x55f5752b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 4 client.25474 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-10-17T01:15:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.194+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 --> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f575197d40 con 0x55f5752b6480 2023-10-17T01:15:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.195+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f5752b6480 0x55f5752b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.195+0000 7fb416ffd700 10 client.25474 ms_handle_connect on v2:172.21.15.181:6834/1888494346 2023-10-17T01:15:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.195+0000 7fb4377fe700 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f5752b6480 0x55f5752b8940 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=580d093d1f80438c server_cookie=547200b2bf5129f6 in_seq=0 out_seq=0 2023-10-17T01:15:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.195+0000 7fb416ffd700 1 -- 172.21.15.45:0/1842125658 <== mds.4 v2:172.21.15.181:6834/1888494346 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f575197d40 con 0x55f5752b6480 2023-10-17T01:15:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.195+0000 7fb416ffd700 10 client.25474 handle_command_reply: tid=0 2023-10-17T01:15:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.195+0000 7fb451bc7b80 1 client.25474 shutdown 2023-10-17T01:15:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.196+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 msgr2=0x55f57525c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.196+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f57525c000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55f575265a80 tx=0x7fb42400a300 comp rx=0 tx=0).stop 2023-10-17T01:15:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 shutdown_connections 2023-10-17T01:15:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:6834/1888494346,v1:172.21.15.181:6835/1888494346] conn(0x55f5752b6480 0x55f5752b8940 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f57524f490 0x55f5752b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5752449d0 0x55f57525bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 --2- 172.21.15.45:0/1842125658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f57524e660 0x55f57525c000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb416ffd700 0 client.25474 ms_handle_reset on v2:172.21.15.181:6834/1888494346 2023-10-17T01:15:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 >> 172.21.15.45:0/1842125658 conn(0x55f5751ad0d0 msgr2=0x55f575252320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.197+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 shutdown_connections 2023-10-17T01:15:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.198+0000 7fb451bc7b80 1 -- 172.21.15.45:0/1842125658 wait complete. 2023-10-17T01:15:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.198+0000 7fb451bc7b80 20 client.25474 trim_cache size 0 max 16384 2023-10-17T01:15:23.207 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-10-17T01:15:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.199+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb4280a0bf0 msgr2=0x7fb4280a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.199+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb4280a0bf0 0x7fb4280a30b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4380503f0 tx=0x7fb43806a410 comp rx=0 tx=0).stop 2023-10-17T01:15:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.199+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 msgr2=0x7fb43012de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.199+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb43012de70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009230 tx=0x7fb42c00b840 comp rx=0 tx=0).stop 2023-10-17T01:15:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 shutdown_connections 2023-10-17T01:15:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb4280a0bf0 0x7fb4280a30b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb430097ba0 0x7fb430135420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb430094190 0x7fb43012de70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 --2- 172.21.15.45:0/823034460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb430097200 0x7fb43012e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 >> 172.21.15.45:0/823034460 conn(0x7fb43008fa80 msgr2=0x7fb43009cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 shutdown_connections 2023-10-17T01:15:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.200+0000 7fb43cf9d700 1 -- 172.21.15.45:0/823034460 wait complete. 2023-10-17T01:15:23.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-17T01:15:23.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.584+0000 7f09cffff700 1 -- 172.21.15.45:0/2948281325 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000e070 con 0x7f09d0074d30 2023-10-17T01:15:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.584+0000 7f09d71ff700 1 -- 172.21.15.45:0/2948281325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0074d30 msgr2=0x7f09d0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.584+0000 7f09d71ff700 1 --2- 172.21.15.45:0/2948281325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0074d30 0x7f09d0073180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b0a0 tx=0x7f09c000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 -- 172.21.15.45:0/2948281325 shutdown_connections 2023-10-17T01:15:23.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 --2- 172.21.15.45:0/2948281325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0108c40 0x7f09d010b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 --2- 172.21.15.45:0/2948281325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0074d30 0x7f09d0073180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 --2- 172.21.15.45:0/2948281325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d0073750 0x7f09d0073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 -- 172.21.15.45:0/2948281325 >> 172.21.15.45:0/2948281325 conn(0x7f09d00fc300 msgr2=0x7f09d00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 -- 172.21.15.45:0/2948281325 shutdown_connections 2023-10-17T01:15:23.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.585+0000 7f09d71ff700 1 -- 172.21.15.45:0/2948281325 wait complete. 2023-10-17T01:15:23.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.586+0000 7f09d71ff700 1 Processor -- start 2023-10-17T01:15:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 -- start start 2023-10-17T01:15:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 0x7f09d0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0074d30 0x7f09d01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d0108c40 0x7f09d01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d01a0730 con 0x7f09d0074d30 2023-10-17T01:15:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d01a0870 con 0x7f09d0073750 2023-10-17T01:15:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d71ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d01a0a00 con 0x7f09d0108c40 2023-10-17T01:15:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.587+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 0x7f09d0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 0x7f09d0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54786/0 (socket says 172.21.15.45:54786) 2023-10-17T01:15:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 -- 172.21.15.45:0/400647279 learned_addr learned my addr 172.21.15.45:0/400647279 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d579c700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d0108c40 0x7f09d01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09c7fff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0074d30 0x7f09d01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 -- 172.21.15.45:0/400647279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d0108c40 msgr2=0x7f09d01a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d0108c40 0x7f09d01a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 -- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0074d30 msgr2=0x7f09d01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0074d30 0x7f09d01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.588+0000 7f09d4f9b700 1 -- 172.21.15.45:0/400647279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c8009520 con 0x7f09d0073750 2023-10-17T01:15:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.589+0000 7f09c7fff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0074d30 0x7f09d01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.589+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 0x7f09d0198bb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09c0003a80 tx=0x7f09c0003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.590+0000 7f09ce7fc700 1 -- 172.21.15.45:0/400647279 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000e050 con 0x7f09d0073750 2023-10-17T01:15:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.590+0000 7f09ce7fc700 1 -- 172.21.15.45:0/400647279 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c0005980 con 0x7f09d0073750 2023-10-17T01:15:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.590+0000 7f09ce7fc700 1 -- 172.21.15.45:0/400647279 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c001bd50 con 0x7f09d0073750 2023-10-17T01:15:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.590+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c0009d00 con 0x7f09d0073750 2023-10-17T01:15:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.590+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d0100a20 con 0x7f09d0073750 2023-10-17T01:15:23.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.595+0000 7f09ce7fc700 1 -- 172.21.15.45:0/400647279 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f09c001b450 con 0x7f09d0073750 2023-10-17T01:15:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.597+0000 7f09ce7fc700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09bc0a0c20 0x7f09bc0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.597+0000 7f09ce7fc700 1 -- 172.21.15.45:0/400647279 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f09c00cdf50 con 0x7f09d0073750 2023-10-17T01:15:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.598+0000 7f09c7fff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09bc0a0c20 0x7f09bc0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.598+0000 7f09c7fff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09bc0a0c20 0x7f09bc0a30e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09b8005950 tx=0x7f09b8005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.599+0000 7f09eaaa6b80 1 Processor -- start 2023-10-17T01:15:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.599+0000 7f09eaaa6b80 1 -- start start 2023-10-17T01:15:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52bfffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c00d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c01730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52b57a10 con 0x55ac52c012b0 2023-10-17T01:15:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52b553d0 con 0x55ac52c00500 2023-10-17T01:15:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52b50cc0 con 0x55ac52bffba0 2023-10-17T01:15:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52bfffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52bfffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43500/0 (socket says 172.21.15.45:43500) 2023-10-17T01:15:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 -- 172.21.15.45:0/2501655680 learned_addr learned my addr 172.21.15.45:0/2501655680 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09c7fff700 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c00d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d579c700 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c01730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 -- 172.21.15.45:0/2501655680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 msgr2=0x55ac52c00d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c00d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 -- 172.21.15.45:0/2501655680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 msgr2=0x55ac52c01730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c01730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.600+0000 7f09d4f9b700 1 -- 172.21.15.45:0/2501655680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac5297ede0 con 0x55ac52bffba0 2023-10-17T01:15:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.601+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52bfffc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09c0097b00 tx=0x7f09c001b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6acb7b110182a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.601+0000 7f09c6ffd700 1 -- 172.21.15.45:0/2501655680 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c009bd00 con 0x55ac52bffba0 2023-10-17T01:15:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.601+0000 7f09c6ffd700 1 -- 172.21.15.45:0/2501655680 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c009b600 con 0x55ac52bffba0 2023-10-17T01:15:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.601+0000 7f09c6ffd700 1 -- 172.21.15.45:0/2501655680 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c009bd00 con 0x55ac52bffba0 2023-10-17T01:15:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.604+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/2501655680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 msgr2=0x55ac52bfffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.604+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52bfffc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09c0097b00 tx=0x7f09c001b7f0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/2501655680 shutdown_connections 2023-10-17T01:15:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c01730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52bfffc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/2501655680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c00d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/2501655680 >> 172.21.15.45:0/2501655680 conn(0x55ac52b5f0d0 msgr2=0x55ac52bfa950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/2501655680 shutdown_connections 2023-10-17T01:15:23.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.605+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/2501655680 wait complete. 2023-10-17T01:15:23.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.606+0000 7f09eaaa6b80 1 -- start start 2023-10-17T01:15:23.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.606+0000 7f09eaaa6b80 1 Processor -- start 2023-10-17T01:15:23.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.608+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c09620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.608+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c09b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.608+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c10ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c09620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c09620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43506/0 (socket says 172.21.15.45:43506) 2023-10-17T01:15:23.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c09b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c09b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54806/0 (socket says 172.21.15.45:54806) 2023-10-17T01:15:23.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52aed510 con 0x55ac52c012b0 2023-10-17T01:15:23.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52aed1b0 con 0x55ac52c00500 2023-10-17T01:15:23.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac529d4920 con 0x55ac52bffba0 2023-10-17T01:15:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09eaaa6b80 10 client.0 _ll_register_callbacks cb 0x55ac52a3aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.609+0000 7f09d4f9b700 1 -- 172.21.15.45:0/1184083230 learned_addr learned my addr 172.21.15.45:0/1184083230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d579c700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c10ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d4f9b700 1 -- 172.21.15.45:0/1184083230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 msgr2=0x55ac52c09b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c09b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d4f9b700 1 -- 172.21.15.45:0/1184083230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 msgr2=0x55ac52c10ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c10ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d4f9b700 1 -- 172.21.15.45:0/1184083230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d0100a20 con 0x55ac52bffba0 2023-10-17T01:15:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d579c700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c10ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09c7fff700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c09620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09c0097380 tx=0x7f09c000b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8203864362e849c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09affff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.610+0000 7f09affff700 1 -- 172.21.15.45:0/1184083230 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0019e00 con 0x55ac52bffba0 2023-10-17T01:15:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09affff700 1 -- 172.21.15.45:0/1184083230 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f09c00a35e0 con 0x55ac52bffba0 2023-10-17T01:15:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac52b7bd10 con 0x55ac52bffba0 2023-10-17T01:15:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09affff700 1 -- 172.21.15.45:0/1184083230 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c009b860 con 0x55ac52bffba0 2023-10-17T01:15:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09eaaa6b80 10 client.25483 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ac52967c60 con 0x55ac52bffba0 2023-10-17T01:15:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09affff700 1 -- 172.21.15.45:0/1184083230 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c0080080 con 0x55ac52bffba0 2023-10-17T01:15:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.611+0000 7f09affff700 1 -- 172.21.15.45:0/1184083230 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f09c00a3150 con 0x55ac52bffba0 2023-10-17T01:15:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.612+0000 7f09eaaa6b80 10 client.25483 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.612+0000 7f09eaaa6b80 10 client.25483 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-10-17T01:15:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.612+0000 7f09eaaa6b80 20 client.25483 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.612+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.612+0000 7f09eaaa6b80 4 client.25483 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.612+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 --> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac52bced30 con 0x55ac52c14e10 2023-10-17T01:15:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.614+0000 7f09c7fff700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.615+0000 7f09c7fff700 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c17230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c95cb814a712b71 server_cookie=c08f0aebdbf30094 in_seq=0 out_seq=0 2023-10-17T01:15:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.615+0000 7f09affff700 10 client.25483 ms_handle_connect on v2:172.21.15.181:6836/1396793294 2023-10-17T01:15:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.616+0000 7f09affff700 1 -- 172.21.15.45:0/1184083230 <== mds.3 v2:172.21.15.181:6836/1396793294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ac52bced30 con 0x55ac52c14e10 2023-10-17T01:15:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.616+0000 7f09affff700 10 client.25483 handle_command_reply: tid=0 2023-10-17T01:15:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.616+0000 7f09eaaa6b80 1 client.25483 shutdown 2023-10-17T01:15:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.616+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 msgr2=0x55ac52c09620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.616+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c09620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09c0097380 tx=0x7f09c000b700 comp rx=0 tx=0).stop 2023-10-17T01:15:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 shutdown_connections 2023-10-17T01:15:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c17230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c10ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09affff700 0 client.25483 ms_handle_reset on v2:172.21.15.181:6836/1396793294 2023-10-17T01:15:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c09620 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/1184083230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c00500 0x55ac52c09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 >> 172.21.15.45:0/1184083230 conn(0x55ac52b5f0d0 msgr2=0x55ac52bfa020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 shutdown_connections 2023-10-17T01:15:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/1184083230 wait complete. 2023-10-17T01:15:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.617+0000 7f09eaaa6b80 20 client.25483 trim_cache size 0 max 16384 2023-10-17T01:15:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 Processor -- start 2023-10-17T01:15:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 -- start start 2023-10-17T01:15:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 0x55ac52bfa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 0x55ac52c113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52bced60 con 0x55ac52c012b0 2023-10-17T01:15:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52c0a840 con 0x55ac52bffba0 2023-10-17T01:15:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.631+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52bf7420 con 0x55ac52c00500 2023-10-17T01:15:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 0x55ac52c113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 0x55ac52bfa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 0x55ac52bfa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54814/0 (socket says 172.21.15.45:54814) 2023-10-17T01:15:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 -- 172.21.15.45:0/4243972512 learned_addr learned my addr 172.21.15.45:0/4243972512 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 0x55ac52c113d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43516/0 (socket says 172.21.15.45:43516) 2023-10-17T01:15:23.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d579c700 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 -- 172.21.15.45:0/4243972512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 msgr2=0x55ac52c113d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 0x55ac52c113d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 -- 172.21.15.45:0/4243972512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 msgr2=0x55ac52c042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 -- 172.21.15.45:0/4243972512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac529a5250 con 0x55ac52bffba0 2023-10-17T01:15:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09c7fff700 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 0x55ac52c113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 0x55ac52bfa020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09c009b590 tx=0x7f09c0099a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6d4217fe68cb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.632+0000 7f09c77fe700 1 -- 172.21.15.45:0/4243972512 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00b2040 con 0x55ac52bffba0 2023-10-17T01:15:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09c77fe700 1 -- 172.21.15.45:0/4243972512 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac52b7bd10 con 0x55ac52bffba0 2023-10-17T01:15:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09c77fe700 1 -- 172.21.15.45:0/4243972512 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0019e60 con 0x55ac52bffba0 2023-10-17T01:15:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/4243972512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 msgr2=0x55ac52bfa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 0x55ac52bfa020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09c009b590 tx=0x7f09c0099a10 comp rx=0 tx=0).stop 2023-10-17T01:15:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/4243972512 shutdown_connections 2023-10-17T01:15:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c012b0 0x55ac52c042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52bffba0 0x55ac52bfa020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/4243972512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52c00500 0x55ac52c113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/4243972512 >> 172.21.15.45:0/4243972512 conn(0x55ac52b5f0d0 msgr2=0x55ac52c082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.633+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/4243972512 shutdown_connections 2023-10-17T01:15:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.634+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/4243972512 wait complete. 2023-10-17T01:15:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.634+0000 7f09eaaa6b80 1 -- start start 2023-10-17T01:15:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.634+0000 7f09eaaa6b80 1 Processor -- start 2023-10-17T01:15:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c00500 0x55ac52c62f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c012b0 0x55ac52c654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52c65c10 con 0x55ac52c00500 2023-10-17T01:15:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52c65d80 con 0x55ac52c012b0 2023-10-17T01:15:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac52c65f40 con 0x55ac52bffba0 2023-10-17T01:15:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09eaaa6b80 10 client.0 _ll_register_callbacks cb 0x7f09c0097a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43526/0 (socket says 172.21.15.45:43526) 2023-10-17T01:15:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 -- 172.21.15.45:0/583647284 learned_addr learned my addr 172.21.15.45:0/583647284 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09c7fff700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c00500 0x55ac52c62f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d579c700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c012b0 0x55ac52c654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 -- 172.21.15.45:0/583647284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c012b0 msgr2=0x55ac52c654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c012b0 0x55ac52c654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 -- 172.21.15.45:0/583647284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c00500 msgr2=0x55ac52c62f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c00500 0x55ac52c62f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 -- 172.21.15.45:0/583647284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b800a900 con 0x55ac52bffba0 2023-10-17T01:15:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09c7fff700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c00500 0x55ac52c62f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09d4f9b700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c609b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09c00a1470 tx=0x7f09c009f570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5126cafd4f17bf9e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.635+0000 7f09affff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09affff700 1 -- 172.21.15.45:0/583647284 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00b2070 con 0x55ac52bffba0 2023-10-17T01:15:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09affff700 1 -- 172.21.15.45:0/583647284 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f09c00a14f0 con 0x55ac52bffba0 2023-10-17T01:15:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09c009b600 con 0x55ac52bffba0 2023-10-17T01:15:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09affff700 1 -- 172.21.15.45:0/583647284 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c0099a80 con 0x55ac52bffba0 2023-10-17T01:15:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 10 client.25489 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ac52967c60 con 0x55ac52bffba0 2023-10-17T01:15:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09affff700 1 -- 172.21.15.45:0/583647284 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c0080020 con 0x55ac52bffba0 2023-10-17T01:15:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09affff700 1 -- 172.21.15.45:0/583647284 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f09c0097580 con 0x55ac52bffba0 2023-10-17T01:15:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 10 client.25489 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 10 client.25489 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-10-17T01:15:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 20 client.25489 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 4 client.25489 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-10-17T01:15:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.636+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 --> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac52b49d40 con 0x55ac52c14e10 2023-10-17T01:15:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09c7fff700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09c7fff700 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c692b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e2b6ac45031c7ad server_cookie=aa14c23ec968dd37 in_seq=0 out_seq=0 2023-10-17T01:15:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09affff700 10 client.25489 ms_handle_connect on v2:172.21.15.181:6836/1396793294 2023-10-17T01:15:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09affff700 1 -- 172.21.15.45:0/583647284 <== mds.3 v2:172.21.15.181:6836/1396793294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac52b49d40 con 0x55ac52c14e10 2023-10-17T01:15:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09affff700 10 client.25489 handle_command_reply: tid=0 2023-10-17T01:15:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09eaaa6b80 1 client.25489 shutdown 2023-10-17T01:15:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 msgr2=0x55ac52c609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.637+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c609b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09c00a1470 tx=0x7f09c009f570 comp rx=0 tx=0).stop 2023-10-17T01:15:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 shutdown_connections 2023-10-17T01:15:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ac52c012b0 0x55ac52c654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac52bffba0 0x55ac52c609b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.181:6836/1396793294,v1:172.21.15.181:6837/1396793294] conn(0x55ac52c14e10 0x55ac52c692b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 --2- 172.21.15.45:0/583647284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac52c00500 0x55ac52c62f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09affff700 0 client.25489 ms_handle_reset on v2:172.21.15.181:6836/1396793294 2023-10-17T01:15:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 >> 172.21.15.45:0/583647284 conn(0x55ac52b5f0d0 msgr2=0x55ac52bfa020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 shutdown_connections 2023-10-17T01:15:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 1 -- 172.21.15.45:0/583647284 wait complete. 2023-10-17T01:15:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09eaaa6b80 20 client.25489 trim_cache size 0 max 16384 2023-10-17T01:15:23.657 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-10-17T01:15:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09bc0a0c20 msgr2=0x7f09bc0a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09d71ff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09bc0a0c20 0x7f09bc0a30e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09b8005950 tx=0x7f09b8005f50 comp rx=0 tx=0).stop 2023-10-17T01:15:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 msgr2=0x7f09d0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.638+0000 7f09d71ff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 0x7f09d0198bb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09c0003a80 tx=0x7f09c0003b60 comp rx=0 tx=0).stop 2023-10-17T01:15:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 shutdown_connections 2023-10-17T01:15:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09bc0a0c20 0x7f09bc0a30e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d0108c40 0x7f09d01a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09d0073750 0x7f09d0198bb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 --2- 172.21.15.45:0/400647279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d0074d30 0x7f09d01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 >> 172.21.15.45:0/400647279 conn(0x7f09d00fc300 msgr2=0x7f09d010acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 shutdown_connections 2023-10-17T01:15:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:23.639+0000 7f09d71ff700 1 -- 172.21.15.45:0/400647279 wait complete. 2023-10-17T01:15:23.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-17T01:15:23.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:23 smithi045 ceph-mon[98474]: pgmap v340: 97 pgs: 97 active+clean; 1.1 GiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 63 MiB/s wr, 612 op/s 2023-10-17T01:15:23.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:23 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/922877182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:15:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:23 smithi060 ceph-mon[134136]: pgmap v340: 97 pgs: 97 active+clean; 1.1 GiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 63 MiB/s wr, 612 op/s 2023-10-17T01:15:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:23 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/922877182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:15:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:23 smithi181 ceph-mon[134520]: pgmap v340: 97 pgs: 97 active+clean; 1.1 GiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 63 MiB/s wr, 612 op/s 2023-10-17T01:15:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:23 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/922877182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:15:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 -- 172.21.15.45:0/145037346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c096ff0 msgr2=0x7fcd1c099480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 --2- 172.21.15.45:0/145037346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c096ff0 0x7fcd1c099480 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009990 tx=0x7fcd10009ca0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 -- 172.21.15.45:0/145037346 shutdown_connections 2023-10-17T01:15:24.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 --2- 172.21.15.45:0/145037346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c099e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 --2- 172.21.15.45:0/145037346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1c094680 0x7fcd1c096ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 --2- 172.21.15.45:0/145037346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c096ff0 0x7fcd1c099480 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.004+0000 7fcd2939d700 1 -- 172.21.15.45:0/145037346 >> 172.21.15.45:0/145037346 conn(0x7fcd1c090260 msgr2=0x7fcd1c0926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.005+0000 7fcd2939d700 1 -- 172.21.15.45:0/145037346 shutdown_connections 2023-10-17T01:15:24.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.005+0000 7fcd2939d700 1 -- 172.21.15.45:0/145037346 wait complete. 2023-10-17T01:15:24.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.006+0000 7fcd2939d700 1 Processor -- start 2023-10-17T01:15:24.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 -- start start 2023-10-17T01:15:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c094680 0x7fcd1c1321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1c096ff0 0x7fcd1c132720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c139d60 con 0x7fcd1c096ff0 2023-10-17T01:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c139ea0 con 0x7fcd1c094680 2023-10-17T01:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.007+0000 7fcd2939d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c13a030 con 0x7fcd1c0999c0 2023-10-17T01:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd23fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c094680 0x7fcd1c1321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43546/0 (socket says 172.21.15.45:43546) 2023-10-17T01:15:24.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 -- 172.21.15.45:0/2240752879 learned_addr learned my addr 172.21.15.45:0/2240752879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 -- 172.21.15.45:0/2240752879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c094680 msgr2=0x7fcd1c1321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c094680 0x7fcd1c1321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 -- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1c096ff0 msgr2=0x7fcd1c132720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1c096ff0 0x7fcd1c132720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.008+0000 7fcd28b9c700 1 -- 172.21.15.45:0/2240752879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2404f2c0 con 0x7fcd1c0999c0 2023-10-17T01:15:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.009+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c139790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcd240647d0 tx=0x7fcd2407bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.009+0000 7fcd217fa700 1 -- 172.21.15.45:0/2240752879 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24069a90 con 0x7fcd1c0999c0 2023-10-17T01:15:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.009+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd100095f0 con 0x7fcd1c0999c0 2023-10-17T01:15:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.009+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c13a610 con 0x7fcd1c0999c0 2023-10-17T01:15:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.009+0000 7fcd217fa700 1 -- 172.21.15.45:0/2240752879 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd24069bf0 con 0x7fcd1c0999c0 2023-10-17T01:15:24.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.009+0000 7fcd217fa700 1 -- 172.21.15.45:0/2240752879 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2407d370 con 0x7fcd1c0999c0 2023-10-17T01:15:24.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.014+0000 7fcd217fa700 1 -- 172.21.15.45:0/2240752879 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fcd2407e410 con 0x7fcd1c0999c0 2023-10-17T01:15:24.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.014+0000 7fcd3dea4b80 1 Processor -- start 2023-10-17T01:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 -- start start 2023-10-17T01:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd217fa700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcd140a0c20 0x7fcd140a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6d660 0x55cd7dc6df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6e490 0x55cd7dc6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dbc4a10 con 0x55cd7dc6d660 2023-10-17T01:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dbc23d0 con 0x55cd7dc6cd00 2023-10-17T01:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dbbdcc0 con 0x55cd7dc6e490 2023-10-17T01:15:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.015+0000 7fcd217fa700 1 -- 172.21.15.45:0/2240752879 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcd24078070 con 0x7fcd1c0999c0 2023-10-17T01:15:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54852/0 (socket says 172.21.15.45:54852) 2023-10-17T01:15:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 -- 172.21.15.45:0/3772643207 learned_addr learned my addr 172.21.15.45:0/3772643207 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6e490 0x55cd7dc6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 -- 172.21.15.45:0/3772643207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6e490 msgr2=0x55cd7dc6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6e490 0x55cd7dc6e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 -- 172.21.15.45:0/3772643207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6d660 msgr2=0x55cd7dc6df00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6d660 0x55cd7dc6df00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd23fff700 1 -- 172.21.15.45:0/3772643207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd7d9ebde0 con 0x55cd7dc6cd00 2023-10-17T01:15:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.016+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6e490 0x55cd7dc6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.017+0000 7fcd23fff700 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6d120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009d80 tx=0x7fcd18011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69a2851eb350a52 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.017+0000 7fcd237fe700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcd140a0c20 0x7fcd140a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.017+0000 7fcd0e7fc700 1 -- 172.21.15.45:0/3772643207 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18004230 con 0x55cd7dc6cd00 2023-10-17T01:15:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.017+0000 7fcd0e7fc700 1 -- 172.21.15.45:0/3772643207 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd18004390 con 0x55cd7dc6cd00 2023-10-17T01:15:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.017+0000 7fcd0e7fc700 1 -- 172.21.15.45:0/3772643207 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18019860 con 0x55cd7dc6cd00 2023-10-17T01:15:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.018+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/3772643207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 msgr2=0x55cd7dc6d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.018+0000 7fcd237fe700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcd140a0c20 0x7fcd140a30e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd100098e0 tx=0x7fcd10005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.018+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6d120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009d80 tx=0x7fcd18011720 comp rx=0 tx=0).stop 2023-10-17T01:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.019+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/3772643207 shutdown_connections 2023-10-17T01:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.019+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6e490 0x55cd7dc6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.019+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6d120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.019+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/3772643207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6d660 0x55cd7dc6df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.019+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/3772643207 >> 172.21.15.45:0/3772643207 conn(0x55cd7dbcc0d0 msgr2=0x55cd7dc67aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.020+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/3772643207 shutdown_connections 2023-10-17T01:15:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.020+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/3772643207 wait complete. 2023-10-17T01:15:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.020+0000 7fcd3dea4b80 1 -- start start 2023-10-17T01:15:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.021+0000 7fcd3dea4b80 1 Processor -- start 2023-10-17T01:15:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc76af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dc77030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dc7e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7db5a510 con 0x55cd7dc6e490 2023-10-17T01:15:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7db5a1b0 con 0x55cd7dc6d660 2023-10-17T01:15:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7da41920 con 0x55cd7dc6cd00 2023-10-17T01:15:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd3dea4b80 10 client.0 _ll_register_callbacks cb 0x55cd7daa7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd237fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dc77030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd237fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dc77030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54866/0 (socket says 172.21.15.45:54866) 2023-10-17T01:15:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.022+0000 7fcd237fe700 1 -- 172.21.15.45:0/4047737325 learned_addr learned my addr 172.21.15.45:0/4047737325 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.023+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc76af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.023+0000 7fcd237fe700 1 -- 172.21.15.45:0/4047737325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 msgr2=0x55cd7dc76af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.023+0000 7fcd237fe700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc76af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.023+0000 7fcd237fe700 1 -- 172.21.15.45:0/4047737325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 msgr2=0x55cd7dc7e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.024+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dc7e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.024+0000 7fcd237fe700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dc7e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.024+0000 7fcd237fe700 1 -- 172.21.15.45:0/4047737325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd180099e0 con 0x55cd7dc6d660 2023-10-17T01:15:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.024+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dc7e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.024+0000 7fcd237fe700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dc77030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd10015000 tx=0x7fcd10009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46cfc6a64ab7ad37 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.024+0000 7fcd037fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.025+0000 7fcd037fe700 1 -- 172.21.15.45:0/4047737325 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1001d070 con 0x55cd7dc6d660 2023-10-17T01:15:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.025+0000 7fcd037fe700 1 -- 172.21.15.45:0/4047737325 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcd1000bd70 con 0x55cd7dc6d660 2023-10-17T01:15:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.025+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd7dbe8d10 con 0x55cd7dc6d660 2023-10-17T01:15:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.025+0000 7fcd037fe700 1 -- 172.21.15.45:0/4047737325 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd10004d20 con 0x55cd7dc6d660 2023-10-17T01:15:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.026+0000 7fcd3dea4b80 10 client.25499 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.026+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cd7d9d4c60 con 0x55cd7dc6d660 2023-10-17T01:15:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.026+0000 7fcd037fe700 1 -- 172.21.15.45:0/4047737325 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd7dbe8d10 con 0x55cd7dc6d660 2023-10-17T01:15:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.026+0000 7fcd037fe700 1 -- 172.21.15.45:0/4047737325 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd10025420 con 0x55cd7dc6d660 2023-10-17T01:15:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd3dea4b80 10 client.25499 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd3dea4b80 10 client.25499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-17T01:15:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd3dea4b80 20 client.25499 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dc93910 0x55cd7dc95d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd3dea4b80 4 client.25499 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 --> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd7dc3bd30 con 0x55cd7dc93910 2023-10-17T01:15:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.027+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dc93910 0x55cd7dc95d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.029+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dc93910 0x55cd7dc95d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=741aed475735ec82 server_cookie=b1c23f46c88a630d in_seq=0 out_seq=0 2023-10-17T01:15:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.029+0000 7fcd037fe700 10 client.25499 ms_handle_connect on v2:172.21.15.60:6836/1246290891 2023-10-17T01:15:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.030+0000 7fcd037fe700 1 -- 172.21.15.45:0/4047737325 <== mds.2 v2:172.21.15.60:6836/1246290891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55cd7dc3bd30 con 0x55cd7dc93910 2023-10-17T01:15:24.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.030+0000 7fcd037fe700 10 client.25499 handle_command_reply: tid=0 2023-10-17T01:15:24.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.030+0000 7fcd3dea4b80 1 client.25499 shutdown 2023-10-17T01:15:24.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.032+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 msgr2=0x55cd7dc77030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.032+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dc77030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd10015000 tx=0x7fcd10009310 comp rx=0 tx=0).stop 2023-10-17T01:15:24.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 shutdown_connections 2023-10-17T01:15:24.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dc93910 0x55cd7dc95d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dc7e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc76af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4047737325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dc77030 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd037fe700 0 client.25499 ms_handle_reset on v2:172.21.15.60:6836/1246290891 2023-10-17T01:15:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.033+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 >> 172.21.15.45:0/4047737325 conn(0x55cd7dbcc0d0 msgr2=0x55cd7dc67180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.034+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 shutdown_connections 2023-10-17T01:15:24.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.034+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4047737325 wait complete. 2023-10-17T01:15:24.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.034+0000 7fcd3dea4b80 20 client.25499 trim_cache size 0 max 16384 2023-10-17T01:15:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.059+0000 7fcd3dea4b80 1 Processor -- start 2023-10-17T01:15:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.059+0000 7fcd3dea4b80 1 -- start start 2023-10-17T01:15:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6d660 0x55cd7dc761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6e490 0x55cd7dc8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd3dea4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dc779c0 con 0x55cd7dc6cd00 2023-10-17T01:15:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6d660 0x55cd7dc761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6d660 0x55cd7dc761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43570/0 (socket says 172.21.15.45:43570) 2023-10-17T01:15:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.060+0000 7fcd237fe700 1 -- 172.21.15.45:0/608612094 learned_addr learned my addr 172.21.15.45:0/608612094 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd23fff700 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dc5f800 con 0x55cd7dc6e490 2023-10-17T01:15:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dc74660 con 0x55cd7dc6d660 2023-10-17T01:15:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6e490 0x55cd7dc8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd28b9c700 1 -- 172.21.15.45:0/608612094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6d660 msgr2=0x55cd7dc761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6d660 0x55cd7dc761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.061+0000 7fcd28b9c700 1 -- 172.21.15.45:0/608612094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6cd00 msgr2=0x55cd7dc6e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.062+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.062+0000 7fcd28b9c700 1 -- 172.21.15.45:0/608612094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd7da12250 con 0x55cd7dc6e490 2023-10-17T01:15:24.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.062+0000 7fcd23fff700 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:24.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.062+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6e490 0x55cd7dc8f970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd24135300 tx=0x7fcd24104ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f825f06d2517597 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.063+0000 7fcd0effd700 1 -- 172.21.15.45:0/608612094 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2408eb60 con 0x55cd7dc6e490 2023-10-17T01:15:24.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.063+0000 7fcd0effd700 1 -- 172.21.15.45:0/608612094 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd24105550 con 0x55cd7dc6e490 2023-10-17T01:15:24.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.063+0000 7fcd0effd700 1 -- 172.21.15.45:0/608612094 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24069d60 con 0x55cd7dc6e490 2023-10-17T01:15:24.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6e490 msgr2=0x55cd7dc8f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6e490 0x55cd7dc8f970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd24135300 tx=0x7fcd24104ac0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 shutdown_connections 2023-10-17T01:15:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6e490 0x55cd7dc8f970 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6cd00 0x55cd7dc6e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/608612094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6d660 0x55cd7dc761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.064+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 >> 172.21.15.45:0/608612094 conn(0x55cd7dbcc0d0 msgr2=0x55cd7dc67a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.065+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 shutdown_connections 2023-10-17T01:15:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.065+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/608612094 wait complete. 2023-10-17T01:15:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.065+0000 7fcd3dea4b80 1 -- start start 2023-10-17T01:15:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.066+0000 7fcd3dea4b80 1 Processor -- start 2023-10-17T01:15:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.067+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dcdd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.067+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dcdd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.067+0000 7fcd3dea4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dce4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.067+0000 7fcd237fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dcdd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd237fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dcdd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54882/0 (socket says 172.21.15.45:54882) 2023-10-17T01:15:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd237fe700 1 -- 172.21.15.45:0/4010325222 learned_addr learned my addr 172.21.15.45:0/4010325222 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dce4aa0 con 0x55cd7dc6e490 2023-10-17T01:15:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dce4be0 con 0x55cd7dc6d660 2023-10-17T01:15:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7dce4d70 con 0x55cd7dc6cd00 2023-10-17T01:15:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd28b9c700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dce4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dcdd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd237fe700 1 -- 172.21.15.45:0/4010325222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 msgr2=0x55cd7dcdd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd237fe700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dcdd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd237fe700 1 -- 172.21.15.45:0/4010325222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 msgr2=0x55cd7dce4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd237fe700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dce4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd237fe700 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd7d9ebde0 con 0x55cd7dc6d660 2023-10-17T01:15:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dcdd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.068+0000 7fcd3dea4b80 10 client.0 _ll_register_callbacks cb 0x7fcd10005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd18004390 con 0x55cd7dc6d660 2023-10-17T01:15:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd237fe700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dcdd9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd10004b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ef8179c47d367f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd037fe700 10 client.25505 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd037fe700 1 -- 172.21.15.45:0/4010325222 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1002f850 con 0x55cd7dc6d660 2023-10-17T01:15:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.069+0000 7fcd037fe700 1 -- 172.21.15.45:0/4010325222 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcd1002f9b0 con 0x55cd7dc6d660 2023-10-17T01:15:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.070+0000 7fcd037fe700 1 -- 172.21.15.45:0/4010325222 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd1002fb70 con 0x55cd7dc6d660 2023-10-17T01:15:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.070+0000 7fcd3dea4b80 10 client.25505 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.070+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x7fcd1c13a610 con 0x55cd7dc6d660 2023-10-17T01:15:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd037fe700 1 -- 172.21.15.45:0/4010325222 <== mon.2 v2:172.21.15.60:3300/0 4 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd1002fcd0 con 0x55cd7dc6d660 2023-10-17T01:15:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd3dea4b80 10 client.25505 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd3dea4b80 10 client.25505 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-17T01:15:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd3dea4b80 20 client.25505 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dce61d0 0x55cd7dce85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd3dea4b80 4 client.25505 mds_command: new command op to 24539 tid=0[{"prefix": "damage ls"}] 2023-10-17T01:15:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 --> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd7dbb6d40 con 0x55cd7dce61d0 2023-10-17T01:15:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.071+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dce61d0 0x55cd7dce85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.072+0000 7fcd037fe700 10 client.25505 ms_handle_connect on v2:172.21.15.60:6836/1246290891 2023-10-17T01:15:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.072+0000 7fcd23fff700 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dce61d0 0x55cd7dce85c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2442816d83751aa5 server_cookie=151bcdcc545979eb in_seq=0 out_seq=0 2023-10-17T01:15:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd037fe700 1 -- 172.21.15.45:0/4010325222 <== mds.2 v2:172.21.15.60:6836/1246290891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd7dbb6d40 con 0x55cd7dce61d0 2023-10-17T01:15:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd037fe700 10 client.25505 handle_command_reply: tid=0 2023-10-17T01:15:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 client.25505 shutdown 2023-10-17T01:15:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 msgr2=0x55cd7dcdd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dcdd9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd10004b70 comp rx=0 tx=0).stop 2023-10-17T01:15:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 shutdown_connections 2023-10-17T01:15:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:6836/1246290891,v1:172.21.15.60:6837/1246290891] conn(0x55cd7dce61d0 0x55cd7dce85c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd7dc6e490 0x55cd7dce4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd7dc6cd00 0x55cd7dcdd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd037fe700 0 client.25505 ms_handle_reset on v2:172.21.15.60:6836/1246290891 2023-10-17T01:15:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 --2- 172.21.15.45:0/4010325222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd7dc6d660 0x55cd7dcdd9f0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd10004b70 comp rx=0 tx=0).stop 2023-10-17T01:15:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 >> 172.21.15.45:0/4010325222 conn(0x55cd7dbcc0d0 msgr2=0x55cd7dc8f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.073+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 shutdown_connections 2023-10-17T01:15:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.074+0000 7fcd3dea4b80 1 -- 172.21.15.45:0/4010325222 wait complete. 2023-10-17T01:15:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.074+0000 7fcd3dea4b80 20 client.25505 trim_cache size 0 max 16384 2023-10-17T01:15:24.083 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-10-17T01:15:24.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcd140a0c20 msgr2=0x7fcd140a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcd140a0c20 0x7fcd140a30e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd100098e0 tx=0x7fcd10005fb0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 msgr2=0x7fcd1c139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c139790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcd240647d0 tx=0x7fcd2407bf90 comp rx=0 tx=0).stop 2023-10-17T01:15:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 shutdown_connections 2023-10-17T01:15:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcd140a0c20 0x7fcd140a30e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0999c0 0x7fcd1c139790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c094680 0x7fcd1c1321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 --2- 172.21.15.45:0/2240752879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd1c096ff0 0x7fcd1c132720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 >> 172.21.15.45:0/2240752879 conn(0x7fcd1c090260 msgr2=0x7fcd1c095330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 shutdown_connections 2023-10-17T01:15:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.075+0000 7fcd2939d700 1 -- 172.21.15.45:0/2240752879 wait complete. 2023-10-17T01:15:24.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-17T01:15:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 -- 172.21.15.45:0/3664493571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c106a60 msgr2=0x7fb81c108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 --2- 172.21.15.45:0/3664493571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c106a60 0x7fb81c108ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b0a0 tx=0x7fb81400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 -- 172.21.15.45:0/3664493571 shutdown_connections 2023-10-17T01:15:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 --2- 172.21.15.45:0/3664493571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c106a60 0x7fb81c108ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 --2- 172.21.15.45:0/3664493571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c101800 0x7fb81c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 --2- 172.21.15.45:0/3664493571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c104130 0x7fb81c106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 -- 172.21.15.45:0/3664493571 >> 172.21.15.45:0/3664493571 conn(0x7fb81c0fb1d0 msgr2=0x7fb81c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 -- 172.21.15.45:0/3664493571 shutdown_connections 2023-10-17T01:15:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.453+0000 7fb8237ff700 1 -- 172.21.15.45:0/3664493571 wait complete. 2023-10-17T01:15:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 Processor -- start 2023-10-17T01:15:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 -- start start 2023-10-17T01:15:24.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 0x7fb81c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c104130 0x7fb81c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c106a60 0x7fb81c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a08f0 con 0x7fb81c106a60 2023-10-17T01:15:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a0a60 con 0x7fb81c101800 2023-10-17T01:15:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb8237ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a0c20 con 0x7fb81c104130 2023-10-17T01:15:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb82159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 0x7fb81c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.456+0000 7fb82159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 0x7fb81c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54904/0 (socket says 172.21.15.45:54904) 2023-10-17T01:15:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 -- 172.21.15.45:0/279746045 learned_addr learned my addr 172.21.15.45:0/279746045 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 -- 172.21.15.45:0/279746045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c104130 msgr2=0x7fb81c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c104130 0x7fb81c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 -- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c106a60 msgr2=0x7fb81c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c106a60 0x7fb81c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 -- 172.21.15.45:0/279746045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814009d00 con 0x7fb81c101800 2023-10-17T01:15:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb82159b700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 0x7fb81c198db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c920 tx=0x7fb81800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb8127fc700 1 -- 172.21.15.45:0/279746045 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818007b00 con 0x7fb81c101800 2023-10-17T01:15:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb8127fc700 1 -- 172.21.15.45:0/279746045 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb818004cf0 con 0x7fb81c101800 2023-10-17T01:15:24.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.457+0000 7fb8127fc700 1 -- 172.21.15.45:0/279746045 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818005670 con 0x7fb81c101800 2023-10-17T01:15:24.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.458+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c1a0f00 con 0x7fb81c101800 2023-10-17T01:15:24.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.458+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c1a1270 con 0x7fb81c101800 2023-10-17T01:15:24.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.463+0000 7fb83714fb80 1 Processor -- start 2023-10-17T01:15:24.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 -- start start 2023-10-17T01:15:24.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac7cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7d500 0x5636eac7dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7e2b0 0x5636eac7e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eabd4a10 con 0x5636eac7e2b0 2023-10-17T01:15:24.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eabd23d0 con 0x5636eac7d500 2023-10-17T01:15:24.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.464+0000 7fb83714fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eabcdcc0 con 0x5636eac7cba0 2023-10-17T01:15:24.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.465+0000 7fb821d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac7cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.466+0000 7fb821d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac7cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43618/0 (socket says 172.21.15.45:43618) 2023-10-17T01:15:24.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.466+0000 7fb821d9c700 1 -- 172.21.15.45:0/2144451469 learned_addr learned my addr 172.21.15.45:0/2144451469 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.467+0000 7fb820d9a700 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7e2b0 0x5636eac7e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.467+0000 7fb821d9c700 1 -- 172.21.15.45:0/2144451469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7d500 msgr2=0x5636eac7dd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:24.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.467+0000 7fb821d9c700 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7d500 0x5636eac7dd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.467+0000 7fb821d9c700 1 -- 172.21.15.45:0/2144451469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7e2b0 msgr2=0x5636eac7e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.467+0000 7fb821d9c700 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7e2b0 0x5636eac7e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.467+0000 7fb821d9c700 1 -- 172.21.15.45:0/2144451469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ea9fbde0 con 0x5636eac7cba0 2023-10-17T01:15:24.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.468+0000 7fb820d9a700 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7e2b0 0x5636eac7e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:24.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.468+0000 7fb8127fc700 1 -- 172.21.15.45:0/279746045 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb818004750 con 0x7fb81c101800 2023-10-17T01:15:24.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.469+0000 7fb821d9c700 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac7cfc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b070 tx=0x7fb814009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79f4c895839e9e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.469+0000 7fb8077fe700 1 -- 172.21.15.45:0/2144451469 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81400e040 con 0x5636eac7cba0 2023-10-17T01:15:24.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.469+0000 7fb8077fe700 1 -- 172.21.15.45:0/2144451469 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8140076e0 con 0x5636eac7cba0 2023-10-17T01:15:24.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.469+0000 7fb8077fe700 1 -- 172.21.15.45:0/2144451469 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81401d6a0 con 0x5636eac7cba0 2023-10-17T01:15:24.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.470+0000 7fb8127fc700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8080a0b20 0x7fb8080a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.471+0000 7fb8127fc700 1 -- 172.21.15.45:0/279746045 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb8180cc0f0 con 0x7fb81c101800 2023-10-17T01:15:24.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.472+0000 7fb83714fb80 1 -- 172.21.15.45:0/2144451469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 msgr2=0x5636eac7cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.472+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac7cfc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b070 tx=0x7fb814009bc0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.472+0000 7fb82159b700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8080a0b20 0x7fb8080a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.472+0000 7fb83714fb80 1 -- 172.21.15.45:0/2144451469 shutdown_connections 2023-10-17T01:15:24.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.472+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7e2b0 0x5636eac7e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.473+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac7cfc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.473+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2144451469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7d500 0x5636eac7dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.473+0000 7fb83714fb80 1 -- 172.21.15.45:0/2144451469 >> 172.21.15.45:0/2144451469 conn(0x5636eabdc0d0 msgr2=0x5636eac77950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.474+0000 7fb82159b700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8080a0b20 0x7fb8080a2fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb818096180 tx=0x7fb8180b9050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.475+0000 7fb83714fb80 1 -- 172.21.15.45:0/2144451469 shutdown_connections 2023-10-17T01:15:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.475+0000 7fb83714fb80 1 -- 172.21.15.45:0/2144451469 wait complete. 2023-10-17T01:15:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.475+0000 7fb83714fb80 1 -- start start 2023-10-17T01:15:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.476+0000 7fb83714fb80 1 Processor -- start 2023-10-17T01:15:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.477+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac868c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.477+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7d500 0x5636eac86e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.477+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac8de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.478+0000 7fb83714fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eab6a510 con 0x5636eac7d500 2023-10-17T01:15:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.478+0000 7fb821d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac8de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.478+0000 7fb820d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7d500 0x5636eac86e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.479+0000 7fb821d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac8de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54920/0 (socket says 172.21.15.45:54920) 2023-10-17T01:15:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.479+0000 7fb821d9c700 1 -- 172.21.15.45:0/1799727811 learned_addr learned my addr 172.21.15.45:0/1799727811 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.479+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eab6a1b0 con 0x5636eac7e2b0 2023-10-17T01:15:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.479+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eaa51920 con 0x5636eac7cba0 2023-10-17T01:15:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.479+0000 7fb83714fb80 10 client.0 _ll_register_callbacks cb 0x5636eaab7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.479+0000 7fb820d9a700 1 -- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 msgr2=0x5636eac868c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb821d9c700 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac868c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb820d9a700 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac868c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb820d9a700 1 -- 172.21.15.45:0/1799727811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 msgr2=0x5636eac8de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb820d9a700 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac8de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb820d9a700 1 -- 172.21.15.45:0/1799727811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c109ac0 con 0x5636eac7d500 2023-10-17T01:15:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb820d9a700 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7d500 0x5636eac86e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb80c005d90 tx=0x7fb80c00e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e566a62bd6e7bc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.480+0000 7fb804ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:15:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.481+0000 7fb804ff9700 1 -- 172.21.15.45:0/1799727811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004230 con 0x5636eac7d500 2023-10-17T01:15:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.481+0000 7fb804ff9700 1 -- 172.21.15.45:0/1799727811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb80c004390 con 0x5636eac7d500 2023-10-17T01:15:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.481+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636eabf8d10 con 0x5636eac7d500 2023-10-17T01:15:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.481+0000 7fb804ff9700 1 -- 172.21.15.45:0/1799727811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80c004550 con 0x5636eac7d500 2023-10-17T01:15:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.481+0000 7fb83714fb80 10 client.15444 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.481+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5636ea9e4c60 con 0x5636eac7d500 2023-10-17T01:15:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb804ff9700 1 -- 172.21.15.45:0/1799727811 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb80c021020 con 0x5636eac7d500 2023-10-17T01:15:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb804ff9700 1 -- 172.21.15.45:0/1799727811 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb80c003680 con 0x5636eac7d500 2023-10-17T01:15:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb83714fb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb83714fb80 10 client.15444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-17T01:15:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb83714fb80 20 client.15444 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:24.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb83714fb80 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eac926f0 0x5636eac94b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.482+0000 7fb83714fb80 4 client.15444 mds_command: new command op to 24550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:24.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.483+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 --> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636eac4bd30 con 0x5636eac926f0 2023-10-17T01:15:24.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.483+0000 7fb821d9c700 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eac926f0 0x5636eac94b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.484+0000 7fb804ff9700 10 client.15444 ms_handle_connect on v2:172.21.15.181:6838/1273587217 2023-10-17T01:15:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.484+0000 7fb821d9c700 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eac926f0 0x5636eac94b10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a42051edfb5bb24 server_cookie=b895923c65140584 in_seq=0 out_seq=0 2023-10-17T01:15:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.485+0000 7fb804ff9700 1 -- 172.21.15.45:0/1799727811 <== mds.1 v2:172.21.15.181:6838/1273587217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5636eac4bd30 con 0x5636eac926f0 2023-10-17T01:15:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.485+0000 7fb804ff9700 10 client.15444 handle_command_reply: tid=0 2023-10-17T01:15:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.485+0000 7fb83714fb80 1 client.15444 shutdown 2023-10-17T01:15:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7d500 msgr2=0x5636eac86e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7d500 0x5636eac86e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb80c005d90 tx=0x7fb80c00e680 comp rx=0 tx=0).stop 2023-10-17T01:15:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 shutdown_connections 2023-10-17T01:15:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac8de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7cba0 0x5636eac868c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eac926f0 0x5636eac94b10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 --2- 172.21.15.45:0/1799727811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7d500 0x5636eac86e00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 >> 172.21.15.45:0/1799727811 conn(0x5636eabdc0d0 msgr2=0x5636eac77020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb804ff9700 0 client.15444 ms_handle_reset on v2:172.21.15.181:6838/1273587217 2023-10-17T01:15:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 shutdown_connections 2023-10-17T01:15:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 1 -- 172.21.15.45:0/1799727811 wait complete. 2023-10-17T01:15:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.486+0000 7fb83714fb80 20 client.15444 trim_cache size 0 max 16384 2023-10-17T01:15:24.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 Processor -- start 2023-10-17T01:15:24.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 -- start start 2023-10-17T01:15:24.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7cba0 0x5636eac7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac94700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac94c40 0x5636eac8e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eac87520 con 0x5636eac7cba0 2023-10-17T01:15:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eac6f6a0 con 0x5636eac7e2b0 2023-10-17T01:15:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.518+0000 7fb83714fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eac844f0 con 0x5636eac94c40 2023-10-17T01:15:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.519+0000 7fb820d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac94700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.519+0000 7fb820d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac94700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54924/0 (socket says 172.21.15.45:54924) 2023-10-17T01:15:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.519+0000 7fb820d9a700 1 -- 172.21.15.45:0/2110560289 learned_addr learned my addr 172.21.15.45:0/2110560289 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.519+0000 7fb821d9c700 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7cba0 0x5636eac7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.520+0000 7fb820d9a700 1 -- 172.21.15.45:0/2110560289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac94c40 msgr2=0x5636eac8e610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:15:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.520+0000 7fb820d9a700 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac94c40 0x5636eac8e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.520+0000 7fb820d9a700 1 -- 172.21.15.45:0/2110560289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7cba0 msgr2=0x5636eac7e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.521+0000 7fb820d9a700 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7cba0 0x5636eac7e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.521+0000 7fb820d9a700 1 -- 172.21.15.45:0/2110560289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636eaa22250 con 0x5636eac7e2b0 2023-10-17T01:15:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.521+0000 7fb821d9c700 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7cba0 0x5636eac7e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.522+0000 7fb820d9a700 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac94700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb80c003b30 tx=0x7fb80c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44395c3112ec90d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.523+0000 7fb807fff700 1 -- 172.21.15.45:0/2110560289 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c003c50 con 0x5636eac7e2b0 2023-10-17T01:15:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.523+0000 7fb807fff700 1 -- 172.21.15.45:0/2110560289 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb80c021030 con 0x5636eac7e2b0 2023-10-17T01:15:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.523+0000 7fb807fff700 1 -- 172.21.15.45:0/2110560289 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004df0 con 0x5636eac7e2b0 2023-10-17T01:15:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 -- 172.21.15.45:0/2110560289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 msgr2=0x5636eac94700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac94700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb80c003b30 tx=0x7fb80c005f80 comp rx=0 tx=0).stop 2023-10-17T01:15:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 -- 172.21.15.45:0/2110560289 shutdown_connections 2023-10-17T01:15:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac94c40 0x5636eac8e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac7cba0 0x5636eac7e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 --2- 172.21.15.45:0/2110560289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7e2b0 0x5636eac94700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 -- 172.21.15.45:0/2110560289 >> 172.21.15.45:0/2110560289 conn(0x5636eabdc0d0 msgr2=0x5636eac85590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 -- 172.21.15.45:0/2110560289 shutdown_connections 2023-10-17T01:15:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.524+0000 7fb83714fb80 1 -- 172.21.15.45:0/2110560289 wait complete. 2023-10-17T01:15:24.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.526+0000 7fb83714fb80 1 -- start start 2023-10-17T01:15:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.526+0000 7fb83714fb80 1 Processor -- start 2023-10-17T01:15:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7cba0 0x5636eace4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 0x5636eace5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac94c40 0x5636eaceb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eacebf00 con 0x5636eac94c40 2023-10-17T01:15:24.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eacec040 con 0x5636eac7cba0 2023-10-17T01:15:24.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eacec200 con 0x5636eac7e2b0 2023-10-17T01:15:24.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.527+0000 7fb83714fb80 10 client.0 _ll_register_callbacks cb 0x7fb80c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:24.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 0x5636eace5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 0x5636eace5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43648/0 (socket says 172.21.15.45:43648) 2023-10-17T01:15:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 -- 172.21.15.45:0/3587751110 learned_addr learned my addr 172.21.15.45:0/3587751110 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb821d9c700 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7cba0 0x5636eace4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 -- 172.21.15.45:0/3587751110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7cba0 msgr2=0x5636eace4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7cba0 0x5636eace4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 -- 172.21.15.45:0/3587751110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac94c40 msgr2=0x5636eaceb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac94c40 0x5636eaceb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 -- 172.21.15.45:0/3587751110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636ea9e4c60 con 0x5636eac7e2b0 2023-10-17T01:15:24.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb820d9a700 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 0x5636eace5000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb80c006010 tx=0x7fb80c00ea80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cfa0e283587e51e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.528+0000 7fb804ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.529+0000 7fb804ff9700 1 -- 172.21.15.45:0/3587751110 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c005670 con 0x5636eac7e2b0 2023-10-17T01:15:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.529+0000 7fb804ff9700 1 -- 172.21.15.45:0/3587751110 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb80c000e40 con 0x5636eac7e2b0 2023-10-17T01:15:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.530+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8140076e0 con 0x5636eac7e2b0 2023-10-17T01:15:24.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.530+0000 7fb804ff9700 1 -- 172.21.15.45:0/3587751110 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80c011470 con 0x5636eac7e2b0 2023-10-17T01:15:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.530+0000 7fb83714fb80 10 client.25510 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.530+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5636ea9fa2b0 con 0x5636eac7e2b0 2023-10-17T01:15:24.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.531+0000 7fb804ff9700 1 -- 172.21.15.45:0/3587751110 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8140076e0 con 0x5636eac7e2b0 2023-10-17T01:15:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.531+0000 7fb804ff9700 1 -- 172.21.15.45:0/3587751110 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb80c024070 con 0x5636eac7e2b0 2023-10-17T01:15:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb83714fb80 10 client.25510 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb83714fb80 10 client.25510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-17T01:15:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb83714fb80 20 client.25510 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb83714fb80 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eaced830 0x5636eacefc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb83714fb80 4 client.25510 mds_command: new command op to 24550 tid=0[{"prefix": "damage ls"}] 2023-10-17T01:15:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 --> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636eabc6d40 con 0x5636eaced830 2023-10-17T01:15:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.532+0000 7fb821d9c700 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eaced830 0x5636eacefc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.533+0000 7fb804ff9700 10 client.25510 ms_handle_connect on v2:172.21.15.181:6838/1273587217 2023-10-17T01:15:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.533+0000 7fb821d9c700 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eaced830 0x5636eacefc20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c328d1424cfbb02 server_cookie=3ca0db7556cd51f4 in_seq=0 out_seq=0 2023-10-17T01:15:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.533+0000 7fb804ff9700 1 -- 172.21.15.45:0/3587751110 <== mds.1 v2:172.21.15.181:6838/1273587217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636eabc6d40 con 0x5636eaced830 2023-10-17T01:15:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.533+0000 7fb804ff9700 10 client.25510 handle_command_reply: tid=0 2023-10-17T01:15:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.533+0000 7fb83714fb80 1 client.25510 shutdown 2023-10-17T01:15:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 msgr2=0x5636eace5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 0x5636eace5000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb80c006010 tx=0x7fb80c00ea80 comp rx=0 tx=0).stop 2023-10-17T01:15:24.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 shutdown_connections 2023-10-17T01:15:24.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:6838/1273587217,v1:172.21.15.181:6839/1273587217] conn(0x5636eaced830 0x5636eacefc20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636eac94c40 0x5636eaceb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5636eac7cba0 0x5636eace4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 --2- 172.21.15.45:0/3587751110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636eac7e2b0 0x5636eace5000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 >> 172.21.15.45:0/3587751110 conn(0x5636eabdc0d0 msgr2=0x5636eac8e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.534+0000 7fb804ff9700 0 client.25510 ms_handle_reset on v2:172.21.15.181:6838/1273587217 2023-10-17T01:15:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.535+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 shutdown_connections 2023-10-17T01:15:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.535+0000 7fb83714fb80 1 -- 172.21.15.45:0/3587751110 wait complete. 2023-10-17T01:15:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.536+0000 7fb83714fb80 20 client.25510 trim_cache size 0 max 16384 2023-10-17T01:15:24.542 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-10-17T01:15:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8080a0b20 msgr2=0x7fb8080a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8080a0b20 0x7fb8080a2fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb818096180 tx=0x7fb8180b9050 comp rx=0 tx=0).stop 2023-10-17T01:15:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 msgr2=0x7fb81c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 0x7fb81c198db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c920 tx=0x7fb81800cce0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 shutdown_connections 2023-10-17T01:15:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8080a0b20 0x7fb8080a2fe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c106a60 0x7fb81c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c101800 0x7fb81c198db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 --2- 172.21.15.45:0/279746045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c104130 0x7fb81c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.537+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 >> 172.21.15.45:0/279746045 conn(0x7fb81c0fb1d0 msgr2=0x7fb81c102470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.538+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 shutdown_connections 2023-10-17T01:15:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.538+0000 7fb8237ff700 1 -- 172.21.15.45:0/279746045 wait complete. 2023-10-17T01:15:24.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-17T01:15:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 -- 172.21.15.45:0/2732694181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 msgr2=0x7f9dcc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/2732694181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc101180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0099c0 tx=0x7f9dbc009cd0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 -- 172.21.15.45:0/2732694181 shutdown_connections 2023-10-17T01:15:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/2732694181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc1028f0 0x7f9dcc1091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/2732694181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc101180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/2732694181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc101f50 0x7f9dcc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 -- 172.21.15.45:0/2732694181 >> 172.21.15.45:0/2732694181 conn(0x7f9dcc0fc2d0 msgr2=0x7f9dcc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.948+0000 7f9dd15ff700 1 -- 172.21.15.45:0/2732694181 shutdown_connections 2023-10-17T01:15:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.949+0000 7f9dd15ff700 1 -- 172.21.15.45:0/2732694181 wait complete. 2023-10-17T01:15:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.949+0000 7f9dd15ff700 1 Processor -- start 2023-10-17T01:15:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 -- start start 2023-10-17T01:15:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc101f50 0x7f9dcc073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc1028f0 0x7f9dcc073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc073e20 con 0x7f9dcc101f50 2023-10-17T01:15:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc073f90 con 0x7f9dcc100d60 2023-10-17T01:15:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dd15ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc1a0b50 con 0x7f9dcc1028f0 2023-10-17T01:15:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dca7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc101f50 0x7f9dcc073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc1028f0 0x7f9dcc073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.950+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc1028f0 0x7f9dcc073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43658/0 (socket says 172.21.15.45:43658) 2023-10-17T01:15:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/3925989835 learned_addr learned my addr 172.21.15.45:0/3925989835 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54952/0 (socket says 172.21.15.45:54952) 2023-10-17T01:15:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 -- 172.21.15.45:0/3925989835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc1028f0 msgr2=0x7f9dcc073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc1028f0 0x7f9dcc073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 -- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc101f50 msgr2=0x7f9dcc073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc101f50 0x7f9dcc073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 -- 172.21.15.45:0/3925989835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0095f0 con 0x7f9dcc100d60 2023-10-17T01:15:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc074b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0058e0 tx=0x7f9dbc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.951+0000 7f9db3fff700 1 -- 172.21.15.45:0/3925989835 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc01d070 con 0x7f9dcc100d60 2023-10-17T01:15:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.952+0000 7f9db3fff700 1 -- 172.21.15.45:0/3925989835 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dbc005020 con 0x7f9dcc100d60 2023-10-17T01:15:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.952+0000 7f9db3fff700 1 -- 172.21.15.45:0/3925989835 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc004a50 con 0x7f9dcc100d60 2023-10-17T01:15:24.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.952+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc1a0dd0 con 0x7f9dcc100d60 2023-10-17T01:15:24.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.952+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc1a1140 con 0x7f9dcc100d60 2023-10-17T01:15:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.956+0000 7f9db3fff700 1 -- 172.21.15.45:0/3925989835 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9dbc005400 con 0x7f9dcc100d60 2023-10-17T01:15:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.956+0000 7f9de4fa4b80 1 Processor -- start 2023-10-17T01:15:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.956+0000 7f9de4fa4b80 1 -- start start 2023-10-17T01:15:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.956+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcee7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.956+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dcee8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dcee9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dce3fa10 con 0x5641dcee92b0 2023-10-17T01:15:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dce3d3d0 con 0x5641dcee8500 2023-10-17T01:15:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dce38cc0 con 0x5641dcee7ba0 2023-10-17T01:15:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dcee8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcee7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dcee8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54966/0 (socket says 172.21.15.45:54966) 2023-10-17T01:15:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcaffd700 1 -- 172.21.15.45:0/2018535632 learned_addr learned my addr 172.21.15.45:0/2018535632 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dcee9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2018535632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 msgr2=0x5641dcee8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dcee8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2018535632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 msgr2=0x5641dcee9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dcee9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.957+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2018535632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641dcc66de0 con 0x5641dcee7ba0 2023-10-17T01:15:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.958+0000 7f9db3fff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9db40a0b20 0x7f9db40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.958+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcee7fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009550 tx=0x7f9dc4009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef10bb80aae974a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.958+0000 7f9db3fff700 1 -- 172.21.15.45:0/3925989835 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9dbc0ccc60 con 0x7f9dcc100d60 2023-10-17T01:15:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.958+0000 7f9db17fa700 1 -- 172.21.15.45:0/2018535632 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4015680 con 0x5641dcee7ba0 2023-10-17T01:15:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.958+0000 7f9db17fa700 1 -- 172.21.15.45:0/2018535632 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc400f040 con 0x5641dcee7ba0 2023-10-17T01:15:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.958+0000 7f9db17fa700 1 -- 172.21.15.45:0/2018535632 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40149b0 con 0x5641dcee7ba0 2023-10-17T01:15:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9db40a0b20 0x7f9db40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/2018535632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 msgr2=0x5641dcee7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcee7fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009550 tx=0x7f9dc4009230 comp rx=0 tx=0).stop 2023-10-17T01:15:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/2018535632 shutdown_connections 2023-10-17T01:15:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dcee9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcee7fc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/2018535632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dcee8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.959+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/2018535632 >> 172.21.15.45:0/2018535632 conn(0x5641dce470d0 msgr2=0x5641dcee2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.960+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9db40a0b20 0x7f9db40a2fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc096040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.960+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/2018535632 shutdown_connections 2023-10-17T01:15:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.960+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/2018535632 wait complete. 2023-10-17T01:15:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.961+0000 7f9de4fa4b80 1 -- start start 2023-10-17T01:15:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.962+0000 7f9de4fa4b80 1 Processor -- start 2023-10-17T01:15:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.965+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.965+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcef1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.965+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee92b0 0x5641dcef8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcdd5510 con 0x5641dcee8500 2023-10-17T01:15:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcdd51b0 con 0x5641dcee92b0 2023-10-17T01:15:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dccbc920 con 0x5641dcee7ba0 2023-10-17T01:15:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9de4fa4b80 10 client.0 _ll_register_callbacks cb 0x5641dcd22a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9dca7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcef1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43668/0 (socket says 172.21.15.45:43668) 2023-10-17T01:15:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.966+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/623366767 learned_addr learned my addr 172.21.15.45:0/623366767 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.967+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee92b0 0x5641dcef8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.967+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee92b0 msgr2=0x5641dcef8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.967+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee92b0 0x5641dcef8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.967+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/623366767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 msgr2=0x5641dcef1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.967+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcef1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.967+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/623366767 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc4009a50 con 0x5641dcee7ba0 2023-10-17T01:15:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.968+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcef1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.968+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef1910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009710 tx=0x7f9dc4005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3074f8d3dd5d05b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.969+0000 7f9daeff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.969+0000 7f9daeff5700 1 -- 172.21.15.45:0/623366767 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40242f0 con 0x5641dcee7ba0 2023-10-17T01:15:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.969+0000 7f9daeff5700 1 -- 172.21.15.45:0/623366767 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9dc4003c00 con 0x5641dcee7ba0 2023-10-17T01:15:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.969+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641dce63d10 con 0x5641dcee7ba0 2023-10-17T01:15:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.969+0000 7f9daeff5700 1 -- 172.21.15.45:0/623366767 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc4003dc0 con 0x5641dcee7ba0 2023-10-17T01:15:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.970+0000 7f9de4fa4b80 10 client.25522 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.970+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5641dcc4fc60 con 0x5641dcee7ba0 2023-10-17T01:15:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.970+0000 7f9daeff5700 1 -- 172.21.15.45:0/623366767 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641dce63d10 con 0x5641dcee7ba0 2023-10-17T01:15:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.970+0000 7f9daeff5700 1 -- 172.21.15.45:0/623366767 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9dc4004320 con 0x5641dcee7ba0 2023-10-17T01:15:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.971+0000 7f9de4fa4b80 10 client.25522 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.971+0000 7f9de4fa4b80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.971+0000 7f9de4fa4b80 20 client.25522 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.971+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcefd610 0x5641dceffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.972+0000 7f9de4fa4b80 4 client.25522 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.972+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641dceb6d30 con 0x5641dcefd610 2023-10-17T01:15:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.972+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcefd610 0x5641dceffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.974+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcefd610 0x5641dceffa30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4524d0b75373476b server_cookie=9ecc4b6871a6b423 in_seq=0 out_seq=0 2023-10-17T01:15:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.974+0000 7f9daeff5700 10 client.25522 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.974+0000 7f9daeff5700 1 -- 172.21.15.45:0/623366767 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5641dceb6d30 con 0x5641dcefd610 2023-10-17T01:15:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.974+0000 7f9daeff5700 10 client.25522 handle_command_reply: tid=0 2023-10-17T01:15:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.974+0000 7f9de4fa4b80 1 client.25522 shutdown 2023-10-17T01:15:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.976+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 msgr2=0x5641dcef1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.976+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef1910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009710 tx=0x7f9dc4005ae0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 shutdown_connections 2023-10-17T01:15:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcefd610 0x5641dceffa30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee92b0 0x5641dcef8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef1910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/623366767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcef1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 >> 172.21.15.45:0/623366767 conn(0x5641dce470d0 msgr2=0x5641dcee2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.977+0000 7f9daeff5700 0 client.25522 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.978+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 shutdown_connections 2023-10-17T01:15:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.978+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/623366767 wait complete. 2023-10-17T01:15:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:24.979+0000 7f9de4fa4b80 20 client.25522 trim_cache size 0 max 16384 2023-10-17T01:15:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.002+0000 7f9de4fa4b80 1 Processor -- start 2023-10-17T01:15:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.003+0000 7f9de4fa4b80 1 -- start start 2023-10-17T01:15:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dceef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dceefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcef14e0 con 0x5641dcee92b0 2023-10-17T01:15:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcee8b80 con 0x5641dcee8500 2023-10-17T01:15:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcedf420 con 0x5641dcee7ba0 2023-10-17T01:15:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9dca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dceef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9dca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dceef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54980/0 (socket says 172.21.15.45:54980) 2023-10-17T01:15:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.004+0000 7f9dca7fc700 1 -- 172.21.15.45:0/52105063 learned_addr learned my addr 172.21.15.45:0/52105063 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.005+0000 7f9dca7fc700 1 -- 172.21.15.45:0/52105063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 msgr2=0x5641dcef9780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.005+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef9780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.005+0000 7f9dca7fc700 1 -- 172.21.15.45:0/52105063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 msgr2=0x5641dceefbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.005+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dceefbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.005+0000 7f9dca7fc700 1 -- 172.21.15.45:0/52105063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641dcc8d250 con 0x5641dcee8500 2023-10-17T01:15:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.005+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dceef6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005d90 tx=0x7f9dc0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfa5c12a77f789e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.006+0000 7f9db1ffb700 1 -- 172.21.15.45:0/52105063 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0016040 con 0x5641dcee8500 2023-10-17T01:15:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.006+0000 7f9db1ffb700 1 -- 172.21.15.45:0/52105063 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc0017040 con 0x5641dcee8500 2023-10-17T01:15:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.006+0000 7f9db1ffb700 1 -- 172.21.15.45:0/52105063 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc001ee10 con 0x5641dcee8500 2023-10-17T01:15:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.007+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/52105063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 msgr2=0x5641dceef6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.007+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dceef6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005d90 tx=0x7f9dc0009260 comp rx=0 tx=0).stop 2023-10-17T01:15:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/52105063 shutdown_connections 2023-10-17T01:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee92b0 0x5641dceefbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee7ba0 0x5641dcef9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/52105063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee8500 0x5641dceef6b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/52105063 >> 172.21.15.45:0/52105063 conn(0x5641dce470d0 msgr2=0x5641dcef05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/52105063 shutdown_connections 2023-10-17T01:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/52105063 wait complete. 2023-10-17T01:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.008+0000 7f9de4fa4b80 1 -- start start 2023-10-17T01:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.009+0000 7f9de4fa4b80 1 Processor -- start 2023-10-17T01:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.010+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 0x5641dcf4a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.010+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcf4ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9de4fa4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee92b0 0x5641dcf4be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 0x5641dcf4a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcef6e40 con 0x5641dcee8500 2023-10-17T01:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcef6fb0 con 0x5641dcee7ba0 2023-10-17T01:15:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9de4fa4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dcef7170 con 0x5641dcee92b0 2023-10-17T01:15:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9de4fa4b80 10 client.0 _ll_register_callbacks cb 0x7f9dbc0d05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.011+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 0x5641dcf4a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54986/0 (socket says 172.21.15.45:54986) 2023-10-17T01:15:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.012+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/248377124 learned_addr learned my addr 172.21.15.45:0/248377124 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.012+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/248377124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee92b0 msgr2=0x5641dcf4be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.012+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee92b0 0x5641dcf4be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.012+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcf4ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.012+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/248377124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 msgr2=0x5641dcf4ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcf4ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/248377124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc0009e90 con 0x5641dcee7ba0 2023-10-17T01:15:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 0x5641dcf4a8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009270 tx=0x7f9dc40090b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975c1edb646da4c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9daeff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcf4ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9daeff5700 1 -- 172.21.15.45:0/248377124 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40145f0 con 0x5641dcee7ba0 2023-10-17T01:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.013+0000 7f9daeff5700 1 -- 172.21.15.45:0/248377124 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9dc4004bf0 con 0x5641dcee7ba0 2023-10-17T01:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.014+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dc400f040 con 0x5641dcee7ba0 2023-10-17T01:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.014+0000 7f9daeff5700 1 -- 172.21.15.45:0/248377124 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc40242f0 con 0x5641dcee7ba0 2023-10-17T01:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.015+0000 7f9de4fa4b80 10 client.25538 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.015+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5641dcc652b0 con 0x5641dcee7ba0 2023-10-17T01:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.016+0000 7f9daeff5700 1 -- 172.21.15.45:0/248377124 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc403e020 con 0x5641dcee7ba0 2023-10-17T01:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9daeff5700 1 -- 172.21.15.45:0/248377124 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9dc4038020 con 0x5641dcee7ba0 2023-10-17T01:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9de4fa4b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9de4fa4b80 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9de4fa4b80 20 client.25538 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcf608f0 0x5641dcf62d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9de4fa4b80 4 client.25538 mds_command: new command op to 24587 tid=0[{"prefix": "damage ls"}] 2023-10-17T01:15:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9dc400f040 con 0x5641dcf608f0 2023-10-17T01:15:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.017+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcf608f0 0x5641dcf62d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.018+0000 7f9daeff5700 10 client.25538 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.018+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcf608f0 0x5641dcf62d10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf87b10cecd63a1f server_cookie=c009959b6a56e546 in_seq=0 out_seq=0 2023-10-17T01:15:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.018+0000 7f9daeff5700 1 -- 172.21.15.45:0/248377124 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9dc400f040 con 0x5641dcf608f0 2023-10-17T01:15:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.018+0000 7f9daeff5700 10 client.25538 handle_command_reply: tid=0 2023-10-17T01:15:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.018+0000 7f9de4fa4b80 1 client.25538 shutdown 2023-10-17T01:15:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.019+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 msgr2=0x5641dcf4a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.019+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 0x5641dcf4a8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009270 tx=0x7f9dc40090b0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 shutdown_connections 2023-10-17T01:15:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5641dcee92b0 0x5641dcf4be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5641dcf608f0 0x5641dcf62d10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5641dcee7ba0 0x5641dcf4a8f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 --2- 172.21.15.45:0/248377124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641dcee8500 0x5641dcf4ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9daeff5700 0 client.25538 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 >> 172.21.15.45:0/248377124 conn(0x5641dce470d0 msgr2=0x5641dcef9780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.020+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 shutdown_connections 2023-10-17T01:15:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.021+0000 7f9de4fa4b80 1 -- 172.21.15.45:0/248377124 wait complete. 2023-10-17T01:15:25.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.021+0000 7f9de4fa4b80 20 client.25538 trim_cache size 0 max 16384 2023-10-17T01:15:25.026 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2023-10-17T01:15:25.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9db40a0b20 msgr2=0x7f9db40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9db40a0b20 0x7f9db40a2fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc096040 comp rx=0 tx=0).stop 2023-10-17T01:15:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 msgr2=0x7f9dcc074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc074b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0058e0 tx=0x7f9dbc00f690 comp rx=0 tx=0).stop 2023-10-17T01:15:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 shutdown_connections 2023-10-17T01:15:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9db40a0b20 0x7f9db40a2fe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc1028f0 0x7f9dcc073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc100d60 0x7f9dcc074b00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 --2- 172.21.15.45:0/3925989835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc101f50 0x7f9dcc073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:25.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 >> 172.21.15.45:0/3925989835 conn(0x7f9dcc0fc2d0 msgr2=0x7f9dcc105480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:25.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 shutdown_connections 2023-10-17T01:15:25.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:25.022+0000 7f9dd15ff700 1 -- 172.21.15.45:0/3925989835 wait complete. 2023-10-17T01:15:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 15.0 s, 2181.2 tps, lat 47.210 ms stddev 65.096 2023-10-17T01:15:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:25 smithi045 ceph-mon[98474]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 81 MiB/s wr, 964 op/s 2023-10-17T01:15:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:25 smithi060 ceph-mon[134136]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 81 MiB/s wr, 964 op/s 2023-10-17T01:15:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:25 smithi181 ceph-mon[134520]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 81 MiB/s wr, 964 op/s 2023-10-17T01:15:26.059 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-17T01:15:26.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-17T01:15:26.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 -- 172.21.15.45:0/2074479478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c0107d50 msgr2=0x7f00c0108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2074479478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c0107d50 0x7f00c0108170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009990 tx=0x7f00bc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 -- 172.21.15.45:0/2074479478 shutdown_connections 2023-10-17T01:15:26.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2074479478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c010f5d0 0x7f00c01119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2074479478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c0107d50 0x7f00c0108170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2074479478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c01086b0 0x7f00c010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 -- 172.21.15.45:0/2074479478 >> 172.21.15.45:0/2074479478 conn(0x7f00c006db00 msgr2=0x7f00c006ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 -- 172.21.15.45:0/2074479478 shutdown_connections 2023-10-17T01:15:26.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.447+0000 7f00c7dff700 1 -- 172.21.15.45:0/2074479478 wait complete. 2023-10-17T01:15:26.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.450+0000 7f00c7dff700 1 Processor -- start 2023-10-17T01:15:26.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.451+0000 7f00c7dff700 1 -- start start 2023-10-17T01:15:26.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c7dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0107d50 0x7f00c01a8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c7dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 0x7f00c01aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c7dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c010f5d0 0x7f00c01acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c7dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01ad4b0 con 0x7f00c0107d50 2023-10-17T01:15:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c7dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01ad5f0 con 0x7f00c010f5d0 2023-10-17T01:15:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c7dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01ad7b0 con 0x7f00c01086b0 2023-10-17T01:15:26.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c539a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 0x7f00c01aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c539a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 0x7f00c01aa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43688/0 (socket says 172.21.15.45:43688) 2023-10-17T01:15:26.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c539a700 1 -- 172.21.15.45:0/2932808664 learned_addr learned my addr 172.21.15.45:0/2932808664 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.452+0000 7f00c639c700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c010f5d0 0x7f00c01acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0107d50 0x7f00c01a8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c539a700 1 -- 172.21.15.45:0/2932808664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c010f5d0 msgr2=0x7f00c01acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c539a700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c010f5d0 0x7f00c01acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c539a700 1 -- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0107d50 msgr2=0x7f00c01a8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c539a700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0107d50 0x7f00c01a8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c539a700 1 -- 172.21.15.45:0/2932808664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0095f0 con 0x7f00c01086b0 2023-10-17T01:15:26.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0107d50 0x7f00c01a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.453+0000 7f00c539a700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 0x7f00c01aa990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f00b000d980 tx=0x7f00b000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.454+0000 7f00b6ffd700 1 -- 172.21.15.45:0/2932808664 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b000de60 con 0x7f00c01086b0 2023-10-17T01:15:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.454+0000 7f00b6ffd700 1 -- 172.21.15.45:0/2932808664 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b00044e0 con 0x7f00c01086b0 2023-10-17T01:15:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.454+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c01ada90 con 0x7f00c01086b0 2023-10-17T01:15:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.454+0000 7f00b6ffd700 1 -- 172.21.15.45:0/2932808664 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0003be0 con 0x7f00c01086b0 2023-10-17T01:15:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.455+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c01adfa0 con 0x7f00c01086b0 2023-10-17T01:15:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.461+0000 7f00b6ffd700 1 -- 172.21.15.45:0/2932808664 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f00b0009bb0 con 0x7f00c01086b0 2023-10-17T01:15:26.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.463+0000 7f00b6ffd700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f00ac0a0cf0 0x7f00ac0a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.463+0000 7f00b6ffd700 1 -- 172.21.15.45:0/2932808664 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f00b00cb920 con 0x7f00c01086b0 2023-10-17T01:15:26.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.463+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f00ac0a0cf0 0x7f00ac0a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.464+0000 7f00db800b80 1 Processor -- start 2023-10-17T01:15:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.465+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f00ac0a0cf0 0x7f00ac0a31b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00b5c0 tx=0x7f00bc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 -- start start 2023-10-17T01:15:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d326040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d326da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d3277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d27da90 con 0x564f2d326580 2023-10-17T01:15:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d27b450 con 0x564f2d327330 2023-10-17T01:15:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.466+0000 7f00db800b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d276d40 con 0x564f2d325c20 2023-10-17T01:15:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.467+0000 7f00c539a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d326da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.467+0000 7f00c639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d3277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.467+0000 7f00c639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d3277b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:55020/0 (socket says 172.21.15.45:55020) 2023-10-17T01:15:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.467+0000 7f00c5b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d326040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.467+0000 7f00c639c700 1 -- 172.21.15.45:0/3142428368 learned_addr learned my addr 172.21.15.45:0/3142428368 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.468+0000 7f00c539a700 1 -- 172.21.15.45:0/3142428368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 msgr2=0x564f2d326040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.468+0000 7f00c539a700 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d326040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.468+0000 7f00c539a700 1 -- 172.21.15.45:0/3142428368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 msgr2=0x564f2d3277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.468+0000 7f00c539a700 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d3277b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.468+0000 7f00c539a700 1 -- 172.21.15.45:0/3142428368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2d0a4e60 con 0x564f2d326580 2023-10-17T01:15:26.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.468+0000 7f00c639c700 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d3277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.469+0000 7f00c539a700 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d326da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f00b00c93f0 tx=0x7f00b0098c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181c918b96bccf35 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.470+0000 7f00abfff700 1 -- 172.21.15.45:0/3142428368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00996c0 con 0x564f2d326580 2023-10-17T01:15:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.470+0000 7f00abfff700 1 -- 172.21.15.45:0/3142428368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b0098fc0 con 0x564f2d326580 2023-10-17T01:15:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.470+0000 7f00abfff700 1 -- 172.21.15.45:0/3142428368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00986f0 con 0x564f2d326580 2023-10-17T01:15:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 -- 172.21.15.45:0/3142428368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 msgr2=0x564f2d326da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d326da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f00b00c93f0 tx=0x7f00b0098c00 comp rx=0 tx=0).stop 2023-10-17T01:15:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 -- 172.21.15.45:0/3142428368 shutdown_connections 2023-10-17T01:15:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d3277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d326040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 --2- 172.21.15.45:0/3142428368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d326da0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.473+0000 7f00db800b80 1 -- 172.21.15.45:0/3142428368 >> 172.21.15.45:0/3142428368 conn(0x564f2d285150 msgr2=0x564f2d3209d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.474+0000 7f00db800b80 1 -- 172.21.15.45:0/3142428368 shutdown_connections 2023-10-17T01:15:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.474+0000 7f00db800b80 1 -- 172.21.15.45:0/3142428368 wait complete. 2023-10-17T01:15:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.475+0000 7f00db800b80 1 -- start start 2023-10-17T01:15:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.475+0000 7f00db800b80 1 Processor -- start 2023-10-17T01:15:26.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.476+0000 7f00db800b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00db800b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d32ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00db800b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d327330 0x564f2d336fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00db800b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d213590 con 0x564f2d326580 2023-10-17T01:15:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00c639c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d327330 0x564f2d336fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00c639c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d327330 0x564f2d336fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43700/0 (socket says 172.21.15.45:43700) 2023-10-17T01:15:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00c639c700 1 -- 172.21.15.45:0/2599822036 learned_addr learned my addr 172.21.15.45:0/2599822036 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00c5b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d213230 con 0x564f2d325c20 2023-10-17T01:15:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d0fa9a0 con 0x564f2d327330 2023-10-17T01:15:26.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00db800b80 10 client.0 _ll_register_callbacks cb 0x564f2d160b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.477+0000 7f00c539a700 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d32ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00c5b9b700 1 -- 172.21.15.45:0/2599822036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d327330 msgr2=0x564f2d336fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d327330 0x564f2d336fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00c5b9b700 1 -- 172.21.15.45:0/2599822036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 msgr2=0x564f2d32ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d32ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00c5b9b700 1 -- 172.21.15.45:0/2599822036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b8009c10 con 0x564f2d325c20 2023-10-17T01:15:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.478+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32fa20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000c00 tx=0x7f00bc01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad69174f67512a8c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.479+0000 7f00a97fa700 1 -- 172.21.15.45:0/2599822036 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc02b040 con 0x564f2d325c20 2023-10-17T01:15:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.479+0000 7f00a97fa700 1 -- 172.21.15.45:0/2599822036 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f00bc030430 con 0x564f2d325c20 2023-10-17T01:15:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.479+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f2d2a1d90 con 0x564f2d325c20 2023-10-17T01:15:26.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.479+0000 7f00a97fa700 1 -- 172.21.15.45:0/2599822036 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00bc01d070 con 0x564f2d325c20 2023-10-17T01:15:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.480+0000 7f00db800b80 10 client.25550 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.480+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f00c01adfa0 con 0x564f2d325c20 2023-10-17T01:15:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.480+0000 7f00a97fa700 1 -- 172.21.15.45:0/2599822036 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f2d2a1d90 con 0x564f2d325c20 2023-10-17T01:15:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.480+0000 7f00a97fa700 1 -- 172.21.15.45:0/2599822036 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00bc03a6c0 con 0x564f2d325c20 2023-10-17T01:15:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.481+0000 7f00db800b80 10 client.25550 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.481+0000 7f00db800b80 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.481+0000 7f00db800b80 20 client.25550 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.481+0000 7f00db800b80 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d33b750 0x564f2d33db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.481+0000 7f00db800b80 4 client.25550 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.481+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f2d2f4db0 con 0x564f2d33b750 2023-10-17T01:15:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.482+0000 7f00c539a700 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d33b750 0x564f2d33db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.483+0000 7f00a97fa700 10 client.25550 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.483+0000 7f00c539a700 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d33b750 0x564f2d33db70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d277c2801fd2e5 server_cookie=bf2215d9b10bb043 in_seq=0 out_seq=0 2023-10-17T01:15:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.484+0000 7f00a97fa700 1 -- 172.21.15.45:0/2599822036 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x564f2d2f4db0 con 0x564f2d33b750 2023-10-17T01:15:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.484+0000 7f00a97fa700 10 client.25550 handle_command_reply: tid=0 2023-10-17T01:15:26.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.484+0000 7f00db800b80 1 client.25550 shutdown 2023-10-17T01:15:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.486+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 msgr2=0x564f2d32fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.486+0000 7f00db800b80 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32fa20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000c00 tx=0x7f00bc01d000 comp rx=0 tx=0).stop 2023-10-17T01:15:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.486+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 shutdown_connections 2023-10-17T01:15:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d327330 0x564f2d336fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d33b750 0x564f2d33db70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32fa20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 --2- 172.21.15.45:0/2599822036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d32ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00a97fa700 0 client.25550 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 >> 172.21.15.45:0/2599822036 conn(0x564f2d285150 msgr2=0x564f2d3200a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 shutdown_connections 2023-10-17T01:15:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.487+0000 7f00db800b80 1 -- 172.21.15.45:0/2599822036 wait complete. 2023-10-17T01:15:26.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.488+0000 7f00db800b80 20 client.25550 trim_cache size 0 max 16384 2023-10-17T01:15:26.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 Processor -- start 2023-10-17T01:15:26.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 -- start start 2023-10-17T01:15:26.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d326580 0x564f2d32d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d327330 0x564f2d32f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d32f5f0 con 0x564f2d327330 2023-10-17T01:15:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d326c30 con 0x564f2d325c20 2023-10-17T01:15:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.526+0000 7f00db800b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d31af30 con 0x564f2d326580 2023-10-17T01:15:26.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c5b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c5b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:55046/0 (socket says 172.21.15.45:55046) 2023-10-17T01:15:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c539a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d326580 0x564f2d32d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c5b9b700 1 -- 172.21.15.45:0/611387398 learned_addr learned my addr 172.21.15.45:0/611387398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c539a700 1 -- 172.21.15.45:0/611387398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 msgr2=0x564f2d32e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c539a700 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c539a700 1 -- 172.21.15.45:0/611387398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d327330 msgr2=0x564f2d32f7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c539a700 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d327330 0x564f2d32f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.527+0000 7f00c539a700 1 -- 172.21.15.45:0/611387398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2d320c70 con 0x564f2d326580 2023-10-17T01:15:26.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.528+0000 7f00c539a700 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d326580 0x564f2d32d810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x564f2d326980 tx=0x7f00b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a0a7a5ce3cc8e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.528+0000 7f00b4bff700 1 -- 172.21.15.45:0/611387398 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0095300 con 0x564f2d326580 2023-10-17T01:15:26.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.528+0000 7f00b4bff700 1 -- 172.21.15.45:0/611387398 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b00a1030 con 0x564f2d326580 2023-10-17T01:15:26.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.528+0000 7f00b4bff700 1 -- 172.21.15.45:0/611387398 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0093e30 con 0x564f2d326580 2023-10-17T01:15:26.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 -- 172.21.15.45:0/611387398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d326580 msgr2=0x564f2d32d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d326580 0x564f2d32d810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x564f2d326980 tx=0x7f00b0000bc0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 -- 172.21.15.45:0/611387398 shutdown_connections 2023-10-17T01:15:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d327330 0x564f2d32f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d325c20 0x564f2d32e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 --2- 172.21.15.45:0/611387398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d326580 0x564f2d32d810 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.529+0000 7f00db800b80 1 -- 172.21.15.45:0/611387398 >> 172.21.15.45:0/611387398 conn(0x564f2d285150 msgr2=0x564f2d337890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.530+0000 7f00db800b80 1 -- 172.21.15.45:0/611387398 shutdown_connections 2023-10-17T01:15:26.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.530+0000 7f00db800b80 1 -- 172.21.15.45:0/611387398 wait complete. 2023-10-17T01:15:26.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.530+0000 7f00db800b80 1 -- start start 2023-10-17T01:15:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.530+0000 7f00db800b80 1 Processor -- start 2023-10-17T01:15:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d385ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d3863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d38d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d38d8f0 con 0x564f2d326580 2023-10-17T01:15:26.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d38da30 con 0x564f2d327330 2023-10-17T01:15:26.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2d38dbc0 con 0x564f2d325c20 2023-10-17T01:15:26.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.532+0000 7f00db800b80 10 client.0 _ll_register_callbacks cb 0x7f00bc003960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:26.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.533+0000 7f00c539a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d3863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.533+0000 7f00c539a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d3863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35860/0 (socket says 172.21.15.45:35860) 2023-10-17T01:15:26.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.533+0000 7f00c539a700 1 -- 172.21.15.45:0/731352128 learned_addr learned my addr 172.21.15.45:0/731352128 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c639c700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d38d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d385ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c539a700 1 -- 172.21.15.45:0/731352128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 msgr2=0x564f2d385ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c539a700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d385ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c539a700 1 -- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 msgr2=0x564f2d38d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c539a700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d38d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c539a700 1 -- 172.21.15.45:0/731352128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f2d0a4e60 con 0x564f2d326580 2023-10-17T01:15:26.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d385ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:26.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.534+0000 7f00c639c700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d38d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:26.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.535+0000 7f00c539a700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d3863e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f00b0098f50 tx=0x7f00b00c9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c7551c642ce619 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.535+0000 7f00a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:15:26.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.536+0000 7f00a97fa700 1 -- 172.21.15.45:0/731352128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00aa070 con 0x564f2d326580 2023-10-17T01:15:26.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.536+0000 7f00a97fa700 1 -- 172.21.15.45:0/731352128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f00b00984b0 con 0x564f2d326580 2023-10-17T01:15:26.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.536+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00b0098fc0 con 0x564f2d326580 2023-10-17T01:15:26.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.537+0000 7f00a97fa700 1 -- 172.21.15.45:0/731352128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00b009cdf0 con 0x564f2d326580 2023-10-17T01:15:26.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.537+0000 7f00db800b80 10 client.15471 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:26.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.537+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564f2d08dce0 con 0x564f2d326580 2023-10-17T01:15:26.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.538+0000 7f00a97fa700 1 -- 172.21.15.45:0/731352128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b0098fc0 con 0x564f2d326580 2023-10-17T01:15:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.538+0000 7f00a97fa700 1 -- 172.21.15.45:0/731352128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00b0094db0 con 0x564f2d326580 2023-10-17T01:15:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.539+0000 7f00db800b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.539+0000 7f00db800b80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.539+0000 7f00db800b80 20 client.15471 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.539+0000 7f00db800b80 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d38f060 0x564f2d391480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.539+0000 7f00db800b80 4 client.15471 mds_command: new command op to 24587 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-17T01:15:26.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.539+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564f2d26fdc0 con 0x564f2d38f060 2023-10-17T01:15:26.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.540+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d38f060 0x564f2d391480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.540+0000 7f00c5b9b700 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d38f060 0x564f2d391480 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe49aa2eb397dc0 server_cookie=dcb38663a9ad77c8 in_seq=0 out_seq=0 2023-10-17T01:15:26.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.541+0000 7f00a97fa700 10 client.15471 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:26.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.541+0000 7f00a97fa700 1 -- 172.21.15.45:0/731352128 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564f2d26fdc0 con 0x564f2d38f060 2023-10-17T01:15:26.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.541+0000 7f00a97fa700 10 client.15471 handle_command_reply: tid=0 2023-10-17T01:15:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.541+0000 7f00db800b80 1 client.15471 shutdown 2023-10-17T01:15:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.542+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 msgr2=0x564f2d3863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.542+0000 7f00db800b80 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d3863e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f00b0098f50 tx=0x7f00b00c9500 comp rx=0 tx=0).stop 2023-10-17T01:15:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 shutdown_connections 2023-10-17T01:15:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564f2d38f060 0x564f2d391480 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564f2d327330 0x564f2d38d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564f2d325c20 0x564f2d385ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00a97fa700 0 client.15471 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:26.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 --2- 172.21.15.45:0/731352128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f2d326580 0x564f2d3863e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 >> 172.21.15.45:0/731352128 conn(0x564f2d285150 msgr2=0x564f2d32d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 shutdown_connections 2023-10-17T01:15:26.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 1 -- 172.21.15.45:0/731352128 wait complete. 2023-10-17T01:15:26.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.543+0000 7f00db800b80 20 client.15471 trim_cache size 0 max 16384 2023-10-17T01:15:26.548 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:15:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2023-10-17T01:15:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:15:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2023-10-17T01:15:26.549 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:15:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.544+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f00ac0a0cf0 msgr2=0x7f00ac0a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.544+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f00ac0a0cf0 0x7f00ac0a31b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00b5c0 tx=0x7f00bc005fd0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.544+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 msgr2=0x7f00c01aa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.544+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 0x7f00c01aa990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f00b000d980 tx=0x7f00b000dd40 comp rx=0 tx=0).stop 2023-10-17T01:15:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 shutdown_connections 2023-10-17T01:15:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f00ac0a0cf0 0x7f00ac0a31b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00c010f5d0 0x7f00c01acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0107d50 0x7f00c01a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 --2- 172.21.15.45:0/2932808664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c01086b0 0x7f00c01aa990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 >> 172.21.15.45:0/2932808664 conn(0x7f00c006db00 msgr2=0x7f00c006e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 shutdown_connections 2023-10-17T01:15:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.545+0000 7f00c7dff700 1 -- 172.21.15.45:0/2932808664 wait complete. 2023-10-17T01:15:26.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:15:26.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.928+0000 7fb38ddff700 1 -- 172.21.15.45:0/462131033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb388106a20 msgr2=0x7fb388108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.928+0000 7fb38ddff700 1 --2- 172.21.15.45:0/462131033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb388106a20 0x7fb388108e90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b0a0 tx=0x7fb38000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.929+0000 7fb38ddff700 1 -- 172.21.15.45:0/462131033 shutdown_connections 2023-10-17T01:15:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.929+0000 7fb38ddff700 1 --2- 172.21.15.45:0/462131033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb388106a20 0x7fb388108e90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.929+0000 7fb38ddff700 1 --2- 172.21.15.45:0/462131033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3881017c0 0x7fb388103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.929+0000 7fb38ddff700 1 --2- 172.21.15.45:0/462131033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3881040f0 0x7fb3881064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.929+0000 7fb38ddff700 1 -- 172.21.15.45:0/462131033 >> 172.21.15.45:0/462131033 conn(0x7fb3880fb1b0 msgr2=0x7fb3880fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.930+0000 7fb38ddff700 1 -- 172.21.15.45:0/462131033 shutdown_connections 2023-10-17T01:15:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.930+0000 7fb38ddff700 1 -- 172.21.15.45:0/462131033 wait complete. 2023-10-17T01:15:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.931+0000 7fb38ddff700 1 Processor -- start 2023-10-17T01:15:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.931+0000 7fb38ddff700 1 -- start start 2023-10-17T01:15:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb38ddff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3881017c0 0x7fb38819d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb38ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 0x7fb38819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb38ddff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb388106a20 0x7fb3881a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb38ddff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881a4cc0 con 0x7fb3881017c0 2023-10-17T01:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb38ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881a4e00 con 0x7fb3881040f0 2023-10-17T01:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb38ddff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881a4f90 con 0x7fb388106a20 2023-10-17T01:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb386ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 0x7fb38819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.932+0000 7fb386ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 0x7fb38819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:55084/0 (socket says 172.21.15.45:55084) 2023-10-17T01:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 -- 172.21.15.45:0/1138671672 learned_addr learned my addr 172.21.15.45:0/1138671672 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb387fff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb388106a20 0x7fb3881a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb3877fe700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3881017c0 0x7fb38819d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 -- 172.21.15.45:0/1138671672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb388106a20 msgr2=0x7fb3881a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb388106a20 0x7fb3881a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 -- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3881017c0 msgr2=0x7fb38819d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3881017c0 0x7fb38819d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 -- 172.21.15.45:0/1138671672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb380009d00 con 0x7fb3881040f0 2023-10-17T01:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb387fff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb388106a20 0x7fb3881a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.933+0000 7fb386ffd700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 0x7fb38819d680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c920 tx=0x7fb37c00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.934+0000 7fb384ff9700 1 -- 172.21.15.45:0/1138671672 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c007b00 con 0x7fb3881040f0 2023-10-17T01:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.934+0000 7fb384ff9700 1 -- 172.21.15.45:0/1138671672 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37c00ce70 con 0x7fb3881040f0 2023-10-17T01:15:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.934+0000 7fb384ff9700 1 -- 172.21.15.45:0/1138671672 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c005580 con 0x7fb3881040f0 2023-10-17T01:15:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.934+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3881a5270 con 0x7fb3881040f0 2023-10-17T01:15:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.934+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3881a5730 con 0x7fb3881040f0 2023-10-17T01:15:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.939+0000 7fb3a17aeb80 1 Processor -- start 2023-10-17T01:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 -- start start 2023-10-17T01:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb57ba0 0x5580fdb57fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 0x5580fdb58d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 0x5580fdb59730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdaafa10 con 0x5580fdb592b0 2023-10-17T01:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdaad3d0 con 0x5580fdb57ba0 2023-10-17T01:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.941+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdaa8cc0 con 0x5580fdb58500 2023-10-17T01:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 0x5580fdb58d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 0x5580fdb58d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43744/0 (socket says 172.21.15.45:43744) 2023-10-17T01:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 -- 172.21.15.45:0/3218891068 learned_addr learned my addr 172.21.15.45:0/3218891068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 -- 172.21.15.45:0/3218891068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb57ba0 msgr2=0x5580fdb57fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb57ba0 0x5580fdb57fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 -- 172.21.15.45:0/3218891068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 msgr2=0x5580fdb59730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 0x5580fdb59730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 -- 172.21.15.45:0/3218891068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580fd8d6de0 con 0x5580fdb58500 2023-10-17T01:15:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.942+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 0x5580fdb58d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb37800bd70 tx=0x7fb37800f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0eccb3430710b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.943+0000 7fb36dffb700 1 -- 172.21.15.45:0/3218891068 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800fe50 con 0x5580fdb58500 2023-10-17T01:15:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.943+0000 7fb36dffb700 1 -- 172.21.15.45:0/3218891068 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb378011900 con 0x5580fdb58500 2023-10-17T01:15:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.943+0000 7fb384ff9700 1 -- 172.21.15.45:0/1138671672 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb37c004750 con 0x7fb3881040f0 2023-10-17T01:15:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.944+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3218891068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 msgr2=0x5580fdb58d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.944+0000 7fb36dffb700 1 -- 172.21.15.45:0/3218891068 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800fe50 con 0x5580fdb58500 2023-10-17T01:15:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.944+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 0x5580fdb58d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb37800bd70 tx=0x7fb37800f640 comp rx=0 tx=0).stop 2023-10-17T01:15:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.944+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3218891068 shutdown_connections 2023-10-17T01:15:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.944+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 0x5580fdb59730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.944+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb57ba0 0x5580fdb57fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.945+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3218891068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb58500 0x5580fdb58d20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.945+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3218891068 >> 172.21.15.45:0/3218891068 conn(0x5580fdab70d0 msgr2=0x5580fdb52950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.945+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3218891068 shutdown_connections 2023-10-17T01:15:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.945+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3218891068 wait complete. 2023-10-17T01:15:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.946+0000 7fb384ff9700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3700a0a60 0x7fb3700a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.946+0000 7fb384ff9700 1 -- 172.21.15.45:0/1138671672 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb37c0cbc70 con 0x7fb3881040f0 2023-10-17T01:15:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.946+0000 7fb387fff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3700a0a60 0x7fb3700a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.946+0000 7fb3a17aeb80 1 -- start start 2023-10-17T01:15:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.947+0000 7fb3a17aeb80 1 Processor -- start 2023-10-17T01:15:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.947+0000 7fb387fff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3700a0a60 0x7fb3700a2f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b830 tx=0x7fb3800079b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb57ba0 0x5580fdb61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb61dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb592b0 0x5580fdb68e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fda45510 con 0x5580fdb57ba0 2023-10-17T01:15:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fda451b0 con 0x5580fdb58500 2023-10-17T01:15:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fd92c920 con 0x5580fdb592b0 2023-10-17T01:15:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3a17aeb80 10 client.0 _ll_register_callbacks cb 0x5580fd992a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb386ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb61dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb386ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb61dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:55102/0 (socket says 172.21.15.45:55102) 2023-10-17T01:15:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb386ffd700 1 -- 172.21.15.45:0/3273695633 learned_addr learned my addr 172.21.15.45:0/3273695633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb387fff700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb592b0 0x5580fdb68e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb57ba0 0x5580fdb61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb387fff700 1 -- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 msgr2=0x5580fdb61dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb387fff700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb61dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.948+0000 7fb387fff700 1 -- 172.21.15.45:0/3273695633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb57ba0 msgr2=0x5580fdb61880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.949+0000 7fb387fff700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb57ba0 0x5580fdb61880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.949+0000 7fb387fff700 1 -- 172.21.15.45:0/3273695633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37800b9a0 con 0x5580fdb592b0 2023-10-17T01:15:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.949+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb57ba0 0x5580fdb61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.949+0000 7fb387fff700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb592b0 0x5580fdb68e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb380009fd0 tx=0x7fb38000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b2cf1045a556384 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.949+0000 7fb366ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.950+0000 7fb366ffd700 1 -- 172.21.15.45:0/3273695633 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000e040 con 0x5580fdb592b0 2023-10-17T01:15:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.950+0000 7fb366ffd700 1 -- 172.21.15.45:0/3273695633 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb380024040 con 0x5580fdb592b0 2023-10-17T01:15:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.950+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580fdad3d10 con 0x5580fdb592b0 2023-10-17T01:15:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.950+0000 7fb366ffd700 1 -- 172.21.15.45:0/3273695633 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38001de40 con 0x5580fdb592b0 2023-10-17T01:15:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.951+0000 7fb3a17aeb80 10 client.25546 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.951+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580fd8bfc60 con 0x5580fdb592b0 2023-10-17T01:15:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.951+0000 7fb366ffd700 1 -- 172.21.15.45:0/3273695633 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580fdad3d10 con 0x5580fdb592b0 2023-10-17T01:15:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb366ffd700 1 -- 172.21.15.45:0/3273695633 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3800050d0 con 0x5580fdb592b0 2023-10-17T01:15:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3a17aeb80 10 client.25546 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3a17aeb80 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3a17aeb80 20 client.25546 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdb6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3a17aeb80 4 client.25546 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580fdb26d30 con 0x5580fdb6d6b0 2023-10-17T01:15:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.952+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdb6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.953+0000 7fb3877fe700 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdb6fad0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe266b6d20427ea server_cookie=7bed68d94f4962bc in_seq=0 out_seq=0 2023-10-17T01:15:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.953+0000 7fb366ffd700 10 client.25546 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.954+0000 7fb366ffd700 1 -- 172.21.15.45:0/3273695633 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5580fdb26d30 con 0x5580fdb6d6b0 2023-10-17T01:15:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.954+0000 7fb366ffd700 10 client.25546 handle_command_reply: tid=0 2023-10-17T01:15:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.954+0000 7fb3a17aeb80 1 client.25546 shutdown 2023-10-17T01:15:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb592b0 msgr2=0x5580fdb68e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb592b0 0x5580fdb68e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb380009fd0 tx=0x7fb38000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 shutdown_connections 2023-10-17T01:15:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb592b0 0x5580fdb68e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb57ba0 0x5580fdb61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdb6fad0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/3273695633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb61dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb366ffd700 0 client.25546 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:26.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.955+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 >> 172.21.15.45:0/3273695633 conn(0x5580fdab70d0 msgr2=0x5580fdb52020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.956+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 shutdown_connections 2023-10-17T01:15:26.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.956+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/3273695633 wait complete. 2023-10-17T01:15:26.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.957+0000 7fb3a17aeb80 20 client.25546 trim_cache size 0 max 16384 2023-10-17T01:15:26.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.988+0000 7fb3a17aeb80 1 Processor -- start 2023-10-17T01:15:26.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 -- start start 2023-10-17T01:15:26.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 0x5580fdb61630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:26.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdb69220 con 0x5580fdb592b0 2023-10-17T01:15:26.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdb6aa90 con 0x5580fdb58500 2023-10-17T01:15:26.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.989+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdb60430 con 0x5580fdb57ba0 2023-10-17T01:15:26.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.990+0000 7fb3877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.990+0000 7fb3877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43764/0 (socket says 172.21.15.45:43764) 2023-10-17T01:15:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.990+0000 7fb3877fe700 1 -- 172.21.15.45:0/2689081161 learned_addr learned my addr 172.21.15.45:0/2689081161 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.990+0000 7fb386ffd700 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.991+0000 7fb386ffd700 1 -- 172.21.15.45:0/2689081161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 msgr2=0x5580fdb696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.991+0000 7fb386ffd700 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.991+0000 7fb386ffd700 1 -- 172.21.15.45:0/2689081161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 msgr2=0x5580fdb61630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.991+0000 7fb386ffd700 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 0x5580fdb61630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.991+0000 7fb386ffd700 1 -- 172.21.15.45:0/2689081161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580fd8fd250 con 0x5580fdb58500 2023-10-17T01:15:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.991+0000 7fb386ffd700 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb5f5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0074a0 tx=0x7fb37c0991c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1a3944bee4963 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.992+0000 7fb36e7fc700 1 -- 172.21.15.45:0/2689081161 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0992f0 con 0x5580fdb58500 2023-10-17T01:15:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.992+0000 7fb36e7fc700 1 -- 172.21.15.45:0/2689081161 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37c09d120 con 0x5580fdb58500 2023-10-17T01:15:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.992+0000 7fb36e7fc700 1 -- 172.21.15.45:0/2689081161 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c099550 con 0x5580fdb58500 2023-10-17T01:15:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.993+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/2689081161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 msgr2=0x5580fdb5f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.993+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb5f5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0074a0 tx=0x7fb37c0991c0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.993+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/2689081161 shutdown_connections 2023-10-17T01:15:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.993+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb592b0 0x5580fdb61630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.994+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.994+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/2689081161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb58500 0x5580fdb5f5d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.994+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/2689081161 >> 172.21.15.45:0/2689081161 conn(0x5580fdab70d0 msgr2=0x5580fdb60690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.994+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/2689081161 shutdown_connections 2023-10-17T01:15:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.994+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/2689081161 wait complete. 2023-10-17T01:15:26.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.995+0000 7fb3a17aeb80 1 -- start start 2023-10-17T01:15:27.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.995+0000 7fb3a17aeb80 1 Processor -- start 2023-10-17T01:15:27.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:27.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb58500 0x5580fdb659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:27.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb592b0 0x5580fdb65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:27.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdbbf000 con 0x5580fdb58500 2023-10-17T01:15:27.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdbbf140 con 0x5580fdb592b0 2023-10-17T01:15:27.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580fdbbf280 con 0x5580fdb57ba0 2023-10-17T01:15:27.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3a17aeb80 10 client.0 _ll_register_callbacks cb 0x5580fd992a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:27.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.996+0000 7fb3877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:27.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.997+0000 7fb3877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43776/0 (socket says 172.21.15.45:43776) 2023-10-17T01:15:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.997+0000 7fb3877fe700 1 -- 172.21.15.45:0/1191398973 learned_addr learned my addr 172.21.15.45:0/1191398973 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.997+0000 7fb387fff700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb592b0 0x5580fdb65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.997+0000 7fb3877fe700 1 -- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb592b0 msgr2=0x5580fdb65f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.997+0000 7fb3877fe700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb592b0 0x5580fdb65f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.997+0000 7fb3877fe700 1 -- 172.21.15.45:0/1191398973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb58500 msgr2=0x5580fdb659e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.998+0000 7fb386ffd700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb58500 0x5580fdb659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.998+0000 7fb3877fe700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb58500 0x5580fdb659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.998+0000 7fb3877fe700 1 -- 172.21.15.45:0/1191398973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3881a5730 con 0x5580fdb57ba0 2023-10-17T01:15:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.998+0000 7fb387fff700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb592b0 0x5580fdb65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.998+0000 7fb366ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.998+0000 7fb3877fe700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb654a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5580fdb58900 tx=0x7fb37800b3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f6e39d5dd8ce16 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:27.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.999+0000 7fb366ffd700 1 -- 172.21.15.45:0/1191398973 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37801e040 con 0x5580fdb57ba0 2023-10-17T01:15:27.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.999+0000 7fb366ffd700 1 -- 172.21.15.45:0/1191398973 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb378011af0 con 0x5580fdb57ba0 2023-10-17T01:15:27.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:26.999+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb378011900 con 0x5580fdb57ba0 2023-10-17T01:15:27.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.000+0000 7fb366ffd700 1 -- 172.21.15.45:0/1191398973 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb378011cd0 con 0x5580fdb57ba0 2023-10-17T01:15:27.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.000+0000 7fb3a17aeb80 10 client.25552 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:27.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.000+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580fd8d52b0 con 0x5580fdb57ba0 2023-10-17T01:15:27.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.000+0000 7fb366ffd700 1 -- 172.21.15.45:0/1191398973 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb378011900 con 0x5580fdb57ba0 2023-10-17T01:15:27.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.000+0000 7fb366ffd700 1 -- 172.21.15.45:0/1191398973 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb378005040 con 0x5580fdb57ba0 2023-10-17T01:15:27.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb3a17aeb80 10 client.25552 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:27.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb3a17aeb80 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:27.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb3a17aeb80 20 client.25552 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:27.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdbc2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:27.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb3a17aeb80 4 client.25552 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:15:27.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5580fdaa1d40 con 0x5580fdb6d6b0 2023-10-17T01:15:27.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.001+0000 7fb386ffd700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdbc2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:27.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.002+0000 7fb386ffd700 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdbc2c90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac804c0684000b83 server_cookie=e19995dfce1a6aa6 in_seq=0 out_seq=0 2023-10-17T01:15:27.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.002+0000 7fb366ffd700 10 client.25552 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:27.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.002+0000 7fb366ffd700 1 -- 172.21.15.45:0/1191398973 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5580fdaa1d40 con 0x5580fdb6d6b0 2023-10-17T01:15:27.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.002+0000 7fb366ffd700 10 client.25552 handle_command_reply: tid=0 2023-10-17T01:15:27.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.002+0000 7fb3a17aeb80 1 client.25552 shutdown 2023-10-17T01:15:27.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 msgr2=0x5580fdb654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:27.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb654a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5580fdb58900 tx=0x7fb37800b3a0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 shutdown_connections 2023-10-17T01:15:27.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5580fdb6d6b0 0x5580fdbc2c90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580fdb592b0 0x5580fdb65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5580fdb57ba0 0x5580fdb654a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 --2- 172.21.15.45:0/1191398973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580fdb58500 0x5580fdb659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb366ffd700 0 client.25552 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:27.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 >> 172.21.15.45:0/1191398973 conn(0x5580fdab70d0 msgr2=0x5580fdb5f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:27.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 shutdown_connections 2023-10-17T01:15:27.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 1 -- 172.21.15.45:0/1191398973 wait complete. 2023-10-17T01:15:27.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.003+0000 7fb3a17aeb80 20 client.25552 trim_cache size 0 max 16384 2023-10-17T01:15:27.012 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:15:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:15:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:15:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:15:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:15:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:15:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:15:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:15:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:15:27.014 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:15:27.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.004+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3700a0a60 msgr2=0x7fb3700a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:27.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.004+0000 7fb38ddff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3700a0a60 0x7fb3700a2f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b830 tx=0x7fb3800079b0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.004+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 msgr2=0x7fb38819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:27.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.004+0000 7fb38ddff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 0x7fb38819d680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c920 tx=0x7fb37c00cc30 comp rx=0 tx=0).stop 2023-10-17T01:15:27.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.005+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 shutdown_connections 2023-10-17T01:15:27.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.005+0000 7fb38ddff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb3700a0a60 0x7fb3700a2f20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.005+0000 7fb38ddff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb388106a20 0x7fb3881a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.005+0000 7fb38ddff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3881017c0 0x7fb38819d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.005+0000 7fb38ddff700 1 --2- 172.21.15.45:0/1138671672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3881040f0 0x7fb38819d680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:27.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.005+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 >> 172.21.15.45:0/1138671672 conn(0x7fb3880fb1b0 msgr2=0x7fb388102430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:27.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.006+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 shutdown_connections 2023-10-17T01:15:27.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:27.006+0000 7fb38ddff700 1 -- 172.21.15.45:0/1138671672 wait complete. 2023-10-17T01:15:27.041 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:15:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 70 MiB/s wr, 947 op/s 2023-10-17T01:15:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub queued for path: / 2023-10-17T01:15:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:15:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:15:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:15:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:15:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:15:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:27 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 70 MiB/s wr, 947 op/s 2023-10-17T01:15:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub queued for path: / 2023-10-17T01:15:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:15:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:15:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:15:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:15:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:15:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:27 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 70 MiB/s wr, 947 op/s 2023-10-17T01:15:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub queued for path: / 2023-10-17T01:15:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:15:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:15:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:15:27.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:15:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:15:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:27.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:27 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:28.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:28 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:15:28.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:28 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:28.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:28 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:15:28.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:28 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:28 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:15:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:28 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:15:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:29 smithi045 ceph-mon[98474]: pgmap v343: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 MiB/s wr, 1.06k op/s 2023-10-17T01:15:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:29 smithi060 ceph-mon[134136]: pgmap v343: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 MiB/s wr, 1.06k op/s 2023-10-17T01:15:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:29 smithi181 ceph-mon[134520]: pgmap v343: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 MiB/s wr, 1.06k op/s 2023-10-17T01:15:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 20.0 s, 2851.2 tps, lat 34.981 ms stddev 41.117 2023-10-17T01:15:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 -- 172.21.15.45:0/1593275069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea41017d0 msgr2=0x7efea4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 --2- 172.21.15.45:0/1593275069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea41017d0 0x7efea4103bc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efea0009990 tx=0x7efea0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 -- 172.21.15.45:0/1593275069 shutdown_connections 2023-10-17T01:15:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 --2- 172.21.15.45:0/1593275069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea4106a30 0x7efea4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 --2- 172.21.15.45:0/1593275069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea41017d0 0x7efea4103bc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 --2- 172.21.15.45:0/1593275069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 0x7efea41064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 -- 172.21.15.45:0/1593275069 >> 172.21.15.45:0/1593275069 conn(0x7efea40fb1a0 msgr2=0x7efea40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.912+0000 7efeac3ff700 1 -- 172.21.15.45:0/1593275069 shutdown_connections 2023-10-17T01:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.913+0000 7efeac3ff700 1 -- 172.21.15.45:0/1593275069 wait complete. 2023-10-17T01:15:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.914+0000 7efeac3ff700 1 Processor -- start 2023-10-17T01:15:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.919+0000 7efeac3ff700 1 -- start start 2023-10-17T01:15:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.921+0000 7efeac3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 0x7efea4194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.921+0000 7efeac3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 0x7efea4194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.921+0000 7efeac3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea4106a30 0x7efea41a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.921+0000 7efeac3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41a4de0 con 0x7efea41017d0 2023-10-17T01:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.922+0000 7efeac3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41a4f50 con 0x7efea4104100 2023-10-17T01:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.922+0000 7efeac3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41a5110 con 0x7efea4106a30 2023-10-17T01:15:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.922+0000 7efea999a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 0x7efea4194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.922+0000 7efeaa19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 0x7efea4194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.922+0000 7efeaa19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 0x7efea4194870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56342/0 (socket says 172.21.15.45:56342) 2023-10-17T01:15:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.922+0000 7efeaa19b700 1 -- 172.21.15.45:0/471249479 learned_addr learned my addr 172.21.15.45:0/471249479 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa19b700 1 -- 172.21.15.45:0/471249479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea4106a30 msgr2=0x7efea41a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa99c700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea4106a30 0x7efea41a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa19b700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea4106a30 0x7efea41a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa19b700 1 -- 172.21.15.45:0/471249479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 msgr2=0x7efea4194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa19b700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 0x7efea4194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa19b700 1 -- 172.21.15.45:0/471249479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efea41017d0 2023-10-17T01:15:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efea999a700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 0x7efea4194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa99c700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea4106a30 0x7efea41a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efeaa19b700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 0x7efea4194870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efea000b680 tx=0x7efea000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efe9b7fe700 1 -- 172.21.15.45:0/471249479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea001d070 con 0x7efea41017d0 2023-10-17T01:15:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efe9b7fe700 1 -- 172.21.15.45:0/471249479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea000fc30 con 0x7efea41017d0 2023-10-17T01:15:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.923+0000 7efe9b7fe700 1 -- 172.21.15.45:0/471249479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000e640 con 0x7efea41017d0 2023-10-17T01:15:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.924+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea41a5390 con 0x7efea41017d0 2023-10-17T01:15:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.925+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea41a57f0 con 0x7efea41017d0 2023-10-17T01:15:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.928+0000 7efe9b7fe700 1 -- 172.21.15.45:0/471249479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7efea00173f0 con 0x7efea41017d0 2023-10-17T01:15:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.930+0000 7efe9b7fe700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efe900a0bf0 0x7efe900a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.931+0000 7efe9b7fe700 1 -- 172.21.15.45:0/471249479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efea00cd9b0 con 0x7efea41017d0 2023-10-17T01:15:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.931+0000 7efea999a700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efe900a0bf0 0x7efe900a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.932+0000 7efebfcc5b80 1 Processor -- start 2023-10-17T01:15:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.933+0000 7efebfcc5b80 1 -- start start 2023-10-17T01:15:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.933+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df31c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.933+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df32790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.933+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df33210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.933+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2de899b0 con 0x564a2df32190 2023-10-17T01:15:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.933+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2de87370 con 0x564a2df32d90 2023-10-17T01:15:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2de82c60 con 0x564a2df31830 2023-10-17T01:15:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df31c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efea999a700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efe900a0bf0 0x7efe900a30b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe94009cb0 tx=0x7efe94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df31c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56922/0 (socket says 172.21.15.45:56922) 2023-10-17T01:15:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 -- 172.21.15.45:0/3641525812 learned_addr learned my addr 172.21.15.45:0/3641525812 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 -- 172.21.15.45:0/3641525812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 msgr2=0x564a2df33210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df33210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 -- 172.21.15.45:0/3641525812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 msgr2=0x564a2df32790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df32790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 -- 172.21.15.45:0/3641525812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a2dcb0d80 con 0x564a2df31830 2023-10-17T01:15:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.934+0000 7efeaa19b700 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df31c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efea009aff0 tx=0x7efea0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa39680d34dacf35 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.935+0000 7efe98bfe700 1 -- 172.21.15.45:0/3641525812 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00a3040 con 0x564a2df31830 2023-10-17T01:15:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.935+0000 7efe98bfe700 1 -- 172.21.15.45:0/3641525812 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea000fda0 con 0x564a2df31830 2023-10-17T01:15:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.935+0000 7efe98bfe700 1 -- 172.21.15.45:0/3641525812 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea009b520 con 0x564a2df31830 2023-10-17T01:15:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.937+0000 7efebfcc5b80 1 -- 172.21.15.45:0/3641525812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 msgr2=0x564a2df31c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.937+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df31c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efea009aff0 tx=0x7efea0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.937+0000 7efebfcc5b80 1 -- 172.21.15.45:0/3641525812 shutdown_connections 2023-10-17T01:15:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.937+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df33210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.937+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df31c50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.938+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/3641525812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df32790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.938+0000 7efebfcc5b80 1 -- 172.21.15.45:0/3641525812 >> 172.21.15.45:0/3641525812 conn(0x564a2de91070 msgr2=0x564a2df2c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.938+0000 7efebfcc5b80 1 -- 172.21.15.45:0/3641525812 shutdown_connections 2023-10-17T01:15:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.938+0000 7efebfcc5b80 1 -- 172.21.15.45:0/3641525812 wait complete. 2023-10-17T01:15:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.939+0000 7efebfcc5b80 1 -- start start 2023-10-17T01:15:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.939+0000 7efebfcc5b80 1 Processor -- start 2023-10-17T01:15:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.941+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df31830 0x564a2df4c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.941+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df32190 0x564a2df4c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.941+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df53670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.941+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2de1f4b0 con 0x564a2df31830 2023-10-17T01:15:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.941+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2de1f150 con 0x564a2df32d90 2023-10-17T01:15:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.941+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2dd068c0 con 0x564a2df32190 2023-10-17T01:15:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.942+0000 7efebfcc5b80 10 client.0 _ll_register_callbacks cb 0x564a2dd6ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.942+0000 7efeaa99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df53670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.942+0000 7efeaa99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df53670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43316/0 (socket says 172.21.15.45:43316) 2023-10-17T01:15:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.942+0000 7efeaa99c700 1 -- 172.21.15.45:0/410113617 learned_addr learned my addr 172.21.15.45:0/410113617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.942+0000 7efea999a700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df32190 0x564a2df4c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa19b700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df31830 0x564a2df4c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa99c700 1 -- 172.21.15.45:0/410113617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df32190 msgr2=0x564a2df4c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa99c700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df32190 0x564a2df4c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa99c700 1 -- 172.21.15.45:0/410113617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df31830 msgr2=0x564a2df4c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa99c700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df31830 0x564a2df4c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa99c700 1 -- 172.21.15.45:0/410113617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea41a57f0 con 0x564a2df32d90 2023-10-17T01:15:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efeaa99c700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df53670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0078a0 tx=0x7efe9c007c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da873aa65e4afd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efe8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.943+0000 7efea999a700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df32190 0x564a2df4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.944+0000 7efe8dffb700 1 -- 172.21.15.45:0/410113617 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c007ea0 con 0x564a2df32d90 2023-10-17T01:15:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.944+0000 7efe8dffb700 1 -- 172.21.15.45:0/410113617 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efe9c00e9e0 con 0x564a2df32d90 2023-10-17T01:15:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.944+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a2deadcb0 con 0x564a2df32d90 2023-10-17T01:15:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.944+0000 7efe8dffb700 1 -- 172.21.15.45:0/410113617 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9c00eba0 con 0x564a2df32d90 2023-10-17T01:15:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.944+0000 7efebfcc5b80 10 client.25580 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.944+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564a2dc99c00 con 0x564a2df32d90 2023-10-17T01:15:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.945+0000 7efe8dffb700 1 -- 172.21.15.45:0/410113617 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a2deadcb0 con 0x564a2df32d90 2023-10-17T01:15:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.945+0000 7efe8dffb700 1 -- 172.21.15.45:0/410113617 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe9c01d3d0 con 0x564a2df32d90 2023-10-17T01:15:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.945+0000 7efebfcc5b80 10 client.25580 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.945+0000 7efebfcc5b80 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.945+0000 7efebfcc5b80 20 client.25580 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.945+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2df57fb0 0x564a2df5a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.946+0000 7efebfcc5b80 4 client.25580 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.946+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a2df00cd0 con 0x564a2df57fb0 2023-10-17T01:15:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.946+0000 7efeaa19b700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2df57fb0 0x564a2df5a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.947+0000 7efe8dffb700 10 client.25580 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.947+0000 7efeaa19b700 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2df57fb0 0x564a2df5a3d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3d1a1d6c0965f8 server_cookie=dd0fd5bc80f6b13b in_seq=0 out_seq=0 2023-10-17T01:15:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.947+0000 7efe8dffb700 1 -- 172.21.15.45:0/410113617 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x564a2df00cd0 con 0x564a2df57fb0 2023-10-17T01:15:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.947+0000 7efe8dffb700 10 client.25580 handle_command_reply: tid=0 2023-10-17T01:15:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 client.25580 shutdown 2023-10-17T01:15:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 msgr2=0x564a2df53670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df53670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0078a0 tx=0x7efe9c007c60 comp rx=0 tx=0).stop 2023-10-17T01:15:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 shutdown_connections 2023-10-17T01:15:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2df57fb0 0x564a2df5a3d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df53670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df31830 0x564a2df4c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/410113617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df32190 0x564a2df4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efe8dffb700 0 client.25580 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 >> 172.21.15.45:0/410113617 conn(0x564a2de91070 msgr2=0x564a2df2ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 shutdown_connections 2023-10-17T01:15:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.948+0000 7efebfcc5b80 1 -- 172.21.15.45:0/410113617 wait complete. 2023-10-17T01:15:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.949+0000 7efebfcc5b80 20 client.25580 trim_cache size 0 max 16384 2023-10-17T01:15:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.971+0000 7efebfcc5b80 1 Processor -- start 2023-10-17T01:15:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.971+0000 7efebfcc5b80 1 -- start start 2023-10-17T01:15:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df2ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2df57dc0 con 0x564a2df32190 2023-10-17T01:15:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2df2d750 con 0x564a2df32d90 2023-10-17T01:15:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2df4bc60 con 0x564a2df31830 2023-10-17T01:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efeaa19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efeaa19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df5a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56944/0 (socket says 172.21.15.45:56944) 2023-10-17T01:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efeaa19b700 1 -- 172.21.15.45:0/634497915 learned_addr learned my addr 172.21.15.45:0/634497915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.972+0000 7efeaa99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efea999a700 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df2ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efeaa19b700 1 -- 172.21.15.45:0/634497915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 msgr2=0x564a2df39240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efeaa19b700 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df39240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efeaa19b700 1 -- 172.21.15.45:0/634497915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 msgr2=0x564a2df2ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efeaa19b700 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df2ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efeaa19b700 1 -- 172.21.15.45:0/634497915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a2dcd71f0 con 0x564a2df31830 2023-10-17T01:15:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.973+0000 7efeaa19b700 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df5a3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efea009ad60 tx=0x7efea0017c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd24b13fb3526fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.974+0000 7efe993ff700 1 -- 172.21.15.45:0/634497915 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00a3040 con 0x564a2df31830 2023-10-17T01:15:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.974+0000 7efe993ff700 1 -- 172.21.15.45:0/634497915 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea0099400 con 0x564a2df31830 2023-10-17T01:15:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.974+0000 7efe993ff700 1 -- 172.21.15.45:0/634497915 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea009fbd0 con 0x564a2df31830 2023-10-17T01:15:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 -- 172.21.15.45:0/634497915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 msgr2=0x564a2df5a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df5a3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efea009ad60 tx=0x7efea0017c50 comp rx=0 tx=0).stop 2023-10-17T01:15:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 -- 172.21.15.45:0/634497915 shutdown_connections 2023-10-17T01:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2df39240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df5a3a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/634497915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df2ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.975+0000 7efebfcc5b80 1 -- 172.21.15.45:0/634497915 >> 172.21.15.45:0/634497915 conn(0x564a2de91070 msgr2=0x564a2df28460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.976+0000 7efebfcc5b80 1 -- 172.21.15.45:0/634497915 shutdown_connections 2023-10-17T01:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.976+0000 7efebfcc5b80 1 -- 172.21.15.45:0/634497915 wait complete. 2023-10-17T01:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.976+0000 7efebfcc5b80 1 -- start start 2023-10-17T01:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.977+0000 7efebfcc5b80 1 Processor -- start 2023-10-17T01:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df51040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2dfa7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2dfa83a0 con 0x564a2df32190 2023-10-17T01:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2dfa84e0 con 0x564a2df32d90 2023-10-17T01:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a2dfa8670 con 0x564a2df31830 2023-10-17T01:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.978+0000 7efebfcc5b80 10 client.0 _ll_register_callbacks cb 0x7efe9c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2dfa7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2dfa7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43334/0 (socket says 172.21.15.45:43334) 2023-10-17T01:15:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 -- 172.21.15.45:0/2114071307 learned_addr learned my addr 172.21.15.45:0/2114071307 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa19b700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 -- 172.21.15.45:0/2114071307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 msgr2=0x564a2df50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efea999a700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df51040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 -- 172.21.15.45:0/2114071307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 msgr2=0x564a2df51040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df51040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa99c700 1 -- 172.21.15.45:0/2114071307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a2dcb0d80 con 0x564a2df32d90 2023-10-17T01:15:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efeaa19b700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.979+0000 7efea999a700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df51040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.980+0000 7efeaa99c700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2dfa7e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007540 tx=0x7efe9c00e4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d357bbfcfca6bb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.980+0000 7efe8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.980+0000 7efe8dffb700 1 -- 172.21.15.45:0/2114071307 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c01da90 con 0x564a2df32d90 2023-10-17T01:15:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.980+0000 7efe8dffb700 1 -- 172.21.15.45:0/2114071307 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efe9c004a30 con 0x564a2df32d90 2023-10-17T01:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.980+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea000fda0 con 0x564a2df32d90 2023-10-17T01:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efe8dffb700 1 -- 172.21.15.45:0/2114071307 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9c004bf0 con 0x564a2df32d90 2023-10-17T01:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 10 client.25586 fetch_fsmap learned FSMap version 30 2023-10-17T01:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564a2dcaf250 con 0x564a2df32d90 2023-10-17T01:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efe8dffb700 1 -- 172.21.15.45:0/2114071307 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9c038020 con 0x564a2df32d90 2023-10-17T01:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efe8dffb700 1 -- 172.21.15.45:0/2114071307 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe9c032020 con 0x564a2df32d90 2023-10-17T01:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 10 client.25586 fetch_fsmap finished waiting for FSMap version 30 2023-10-17T01:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 20 client.25586 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2dfaa180 0x564a2dfac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 4 client.25586 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.981+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a2de7bce0 con 0x564a2dfaa180 2023-10-17T01:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.982+0000 7efeaa19b700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2dfaa180 0x564a2dfac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.983+0000 7efeaa19b700 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2dfaa180 0x564a2dfac5a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aa07982d0939305 server_cookie=4a65ca7d180bd20b in_seq=0 out_seq=0 2023-10-17T01:15:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:30.983+0000 7efe8dffb700 10 client.25586 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.002+0000 7efe8dffb700 1 -- 172.21.15.45:0/2114071307 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a2de7bce0 con 0x564a2dfaa180 2023-10-17T01:15:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.002+0000 7efe8dffb700 10 client.25586 handle_command_reply: tid=0 2023-10-17T01:15:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.002+0000 7efebfcc5b80 1 client.25586 shutdown 2023-10-17T01:15:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.002+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 msgr2=0x564a2dfa7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.002+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2dfa7e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007540 tx=0x7efe9c00e4f0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 shutdown_connections 2023-10-17T01:15:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564a2dfaa180 0x564a2dfac5a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564a2df32d90 0x564a2dfa7e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564a2df31830 0x564a2df50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 --2- 172.21.15.45:0/2114071307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a2df32190 0x564a2df51040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 >> 172.21.15.45:0/2114071307 conn(0x564a2de91070 msgr2=0x564a2df2af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efe8dffb700 0 client.25586 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.003+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 shutdown_connections 2023-10-17T01:15:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.004+0000 7efebfcc5b80 1 -- 172.21.15.45:0/2114071307 wait complete. 2023-10-17T01:15:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.004+0000 7efebfcc5b80 20 client.25586 trim_cache size 0 max 16384 2023-10-17T01:15:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:15:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:15:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:15:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:15:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efe900a0bf0 msgr2=0x7efe900a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efe900a0bf0 0x7efe900a30b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe94009cb0 tx=0x7efe94009260 comp rx=0 tx=0).stop 2023-10-17T01:15:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 msgr2=0x7efea4194870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 0x7efea4194870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efea000b680 tx=0x7efea000f460 comp rx=0 tx=0).stop 2023-10-17T01:15:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 shutdown_connections 2023-10-17T01:15:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efe900a0bf0 0x7efe900a30b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efea4106a30 0x7efea41a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea41017d0 0x7efea4194870 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 --2- 172.21.15.45:0/471249479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efea4104100 0x7efea4194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.005+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 >> 172.21.15.45:0/471249479 conn(0x7efea40fb1a0 msgr2=0x7efea4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.006+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 shutdown_connections 2023-10-17T01:15:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:31.006+0000 7efeac3ff700 1 -- 172.21.15.45:0/471249479 wait complete. 2023-10-17T01:15:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:31 smithi045 ceph-mon[98474]: pgmap v344: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 67 MiB/s wr, 1.10k op/s 2023-10-17T01:15:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:31 smithi060 ceph-mon[134136]: pgmap v344: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 67 MiB/s wr, 1.10k op/s 2023-10-17T01:15:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:31 smithi181 ceph-mon[134520]: pgmap v344: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 67 MiB/s wr, 1.10k op/s 2023-10-17T01:15:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:33 smithi045 ceph-mon[98474]: pgmap v345: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 64 MiB/s wr, 1.11k op/s 2023-10-17T01:15:33.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:33 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:33 smithi060 ceph-mon[134136]: pgmap v345: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 64 MiB/s wr, 1.11k op/s 2023-10-17T01:15:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:33 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:33 smithi181 ceph-mon[134520]: pgmap v345: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 64 MiB/s wr, 1.11k op/s 2023-10-17T01:15:33.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:33 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 25.0 s, 2989.0 tps, lat 33.454 ms stddev 38.931 2023-10-17T01:15:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:35 smithi045 ceph-mon[98474]: pgmap v346: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 60 MiB/s wr, 1.18k op/s 2023-10-17T01:15:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:35 smithi060 ceph-mon[134136]: pgmap v346: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 60 MiB/s wr, 1.18k op/s 2023-10-17T01:15:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:35 smithi181 ceph-mon[134520]: pgmap v346: 97 pgs: 97 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 60 MiB/s wr, 1.18k op/s 2023-10-17T01:15:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:36 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:36.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:36 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:36.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:36 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:37 smithi045 ceph-mon[98474]: pgmap v347: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 33 MiB/s wr, 896 op/s 2023-10-17T01:15:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:37 smithi060 ceph-mon[134136]: pgmap v347: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 33 MiB/s wr, 896 op/s 2023-10-17T01:15:37.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:37 smithi181 ceph-mon[134520]: pgmap v347: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 33 MiB/s wr, 896 op/s 2023-10-17T01:15:38.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:38 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:38 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:38.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:38 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:15:39.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:39 smithi045 ceph-mon[98474]: pgmap v348: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 31 MiB/s wr, 924 op/s 2023-10-17T01:15:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:39 smithi060 ceph-mon[134136]: pgmap v348: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 31 MiB/s wr, 924 op/s 2023-10-17T01:15:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:39 smithi181 ceph-mon[134520]: pgmap v348: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 31 MiB/s wr, 924 op/s 2023-10-17T01:15:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 30.0 s, 2768.0 tps, lat 36.027 ms stddev 45.526 2023-10-17T01:15:41.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:41 smithi045 ceph-mon[98474]: pgmap v349: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 30 MiB/s wr, 984 op/s 2023-10-17T01:15:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:41 smithi060 ceph-mon[134136]: pgmap v349: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 30 MiB/s wr, 984 op/s 2023-10-17T01:15:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:41 smithi181 ceph-mon[134520]: pgmap v349: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 30 MiB/s wr, 984 op/s 2023-10-17T01:15:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:43 smithi045 ceph-mon[98474]: pgmap v350: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 999 op/s 2023-10-17T01:15:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:43 smithi060 ceph-mon[134136]: pgmap v350: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 999 op/s 2023-10-17T01:15:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:43 smithi181 ceph-mon[134520]: pgmap v350: 97 pgs: 97 active+clean; 1.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 999 op/s 2023-10-17T01:15:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/2956850677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4102920 msgr2=0x7f2aa41091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/2956850677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4102920 0x7f2aa41091d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b0d0 tx=0x7f2a9c00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/2956850677 shutdown_connections 2023-10-17T01:15:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/2956850677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4102920 0x7f2aa41091d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/2956850677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4100d90 0x7f2aa41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/2956850677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa4101f80 0x7f2aa41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/2956850677 >> 172.21.15.45:0/2956850677 conn(0x7f2aa40fc300 msgr2=0x7f2aa40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/2956850677 shutdown_connections 2023-10-17T01:15:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.267+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/2956850677 wait complete. 2023-10-17T01:15:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.268+0000 7f2aaa3ff700 1 Processor -- start 2023-10-17T01:15:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 -- start start 2023-10-17T01:15:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 0x7f2aa4196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4101f80 0x7f2aa4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa4102920 0x7f2aa4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41986a0 con 0x7f2aa4100d90 2023-10-17T01:15:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa4198810 con 0x7f2aa4102920 2023-10-17T01:15:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aaa3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41989d0 con 0x7f2aa4101f80 2023-10-17T01:15:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 0x7f2aa4196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4101f80 0x7f2aa4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 0x7f2aa4196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47540/0 (socket says 172.21.15.45:47540) 2023-10-17T01:15:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aa3fff700 1 -- 172.21.15.45:0/499043940 learned_addr learned my addr 172.21.15.45:0/499043940 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.269+0000 7f2aa899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa4102920 0x7f2aa4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa37fe700 1 -- 172.21.15.45:0/499043940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa4102920 msgr2=0x7f2aa4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa4102920 0x7f2aa4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa37fe700 1 -- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 msgr2=0x7f2aa4196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 0x7f2aa4196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa37fe700 1 -- 172.21.15.45:0/499043940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c009d00 con 0x7f2aa4101f80 2023-10-17T01:15:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 0x7f2aa4196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4101f80 0x7f2aa4197260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800b450 tx=0x7f2a9800b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa17fa700 1 -- 172.21.15.45:0/499043940 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98011830 con 0x7f2aa4101f80 2023-10-17T01:15:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa17fa700 1 -- 172.21.15.45:0/499043940 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a9800be60 con 0x7f2aa4101f80 2023-10-17T01:15:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aa17fa700 1 -- 172.21.15.45:0/499043940 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98010d20 con 0x7f2aa4101f80 2023-10-17T01:15:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa4198cb0 con 0x7f2aa4101f80 2023-10-17T01:15:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.270+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa40731e0 con 0x7f2aa4101f80 2023-10-17T01:15:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.274+0000 7f2aa17fa700 1 -- 172.21.15.45:0/499043940 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2a98010380 con 0x7f2aa4101f80 2023-10-17T01:15:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.276+0000 7f2abddf2b80 1 Processor -- start 2023-10-17T01:15:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 -- start start 2023-10-17T01:15:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa17fa700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2a8c0a0b20 0x7f2a8c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b2040 0x56025e4b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 0x56025e4b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4019b0 con 0x56025e4b2040 2023-10-17T01:15:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e3ff370 con 0x56025e4b35a0 2023-10-17T01:15:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e3fac60 con 0x56025e4b29a0 2023-10-17T01:15:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 0x56025e4b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 0x56025e4b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54660/0 (socket says 172.21.15.45:54660) 2023-10-17T01:15:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa37fe700 1 -- 172.21.15.45:0/282751482 learned_addr learned my addr 172.21.15.45:0/282751482 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa17fa700 1 -- 172.21.15.45:0/499043940 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2a980cbb20 con 0x7f2aa4101f80 2023-10-17T01:15:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa899c700 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.277+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b2040 0x56025e4b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa899c700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2a8c0a0b20 0x7f2a8c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa37fe700 1 -- 172.21.15.45:0/282751482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 msgr2=0x56025e4b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa37fe700 1 -- 172.21.15.45:0/282751482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b2040 msgr2=0x56025e4b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b2040 0x56025e4b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa37fe700 1 -- 172.21.15.45:0/282751482 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56025e228d80 con 0x56025e4b29a0 2023-10-17T01:15:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa899c700 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 0x56025e4b2fa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800bfd0 tx=0x7f2a98098d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a555c4faaf5759 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2a8a7fc700 1 -- 172.21.15.45:0/282751482 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98022bf0 con 0x56025e4b29a0 2023-10-17T01:15:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2a8a7fc700 1 -- 172.21.15.45:0/282751482 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a980174a0 con 0x56025e4b29a0 2023-10-17T01:15:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.278+0000 7f2a8a7fc700 1 -- 172.21.15.45:0/282751482 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9809cfe0 con 0x56025e4b29a0 2023-10-17T01:15:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 -- 172.21.15.45:0/282751482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 msgr2=0x56025e4b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 0x56025e4b2fa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800bfd0 tx=0x7f2a98098d50 comp rx=0 tx=0).stop 2023-10-17T01:15:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2aa899c700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2a8c0a0b20 0x7f2a8c0a2fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009cd0 tx=0x7f2a9c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 -- 172.21.15.45:0/282751482 shutdown_connections 2023-10-17T01:15:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b2040 0x56025e4b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/282751482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b29a0 0x56025e4b2fa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 -- 172.21.15.45:0/282751482 >> 172.21.15.45:0/282751482 conn(0x56025e409070 msgr2=0x56025e4a42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 -- 172.21.15.45:0/282751482 shutdown_connections 2023-10-17T01:15:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.279+0000 7f2abddf2b80 1 -- 172.21.15.45:0/282751482 wait complete. 2023-10-17T01:15:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 -- start start 2023-10-17T01:15:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 Processor -- start 2023-10-17T01:15:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4bbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4bc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e3974b0 con 0x56025e4b29a0 2023-10-17T01:15:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e397150 con 0x56025e4b35a0 2023-10-17T01:15:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.280+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e27e8c0 con 0x56025e4b2040 2023-10-17T01:15:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2abddf2b80 10 client.0 _ll_register_callbacks cb 0x56025e2e4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4bbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4bbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54676/0 (socket says 172.21.15.45:54676) 2023-10-17T01:15:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 -- 172.21.15.45:0/266505321 learned_addr learned my addr 172.21.15.45:0/266505321 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa899c700 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 -- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 msgr2=0x56025e4c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 -- 172.21.15.45:0/266505321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 msgr2=0x56025e4bc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4bc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 -- 172.21.15.45:0/266505321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa40731e0 con 0x56025e4b2040 2023-10-17T01:15:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4bbbc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a94012810 tx=0x7f2a94012b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56a0721d24f8f2b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.281+0000 7f2a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2a837fe700 1 -- 172.21.15.45:0/266505321 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94004a30 con 0x56025e4b2040 2023-10-17T01:15:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2a837fe700 1 -- 172.21.15.45:0/266505321 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2a94004b90 con 0x56025e4b2040 2023-10-17T01:15:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56025e425cb0 con 0x56025e4b2040 2023-10-17T01:15:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2a837fe700 1 -- 172.21.15.45:0/266505321 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a94004d50 con 0x56025e4b2040 2023-10-17T01:15:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 10 client.25573 fetch_fsmap learned FSMap version 33 2023-10-17T01:15:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56025e211c00 con 0x56025e4b2040 2023-10-17T01:15:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2a837fe700 1 -- 172.21.15.45:0/266505321 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56025e425cb0 con 0x56025e4b2040 2023-10-17T01:15:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2a837fe700 1 -- 172.21.15.45:0/266505321 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2a94025050 con 0x56025e4b2040 2023-10-17T01:15:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 10 client.25573 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:15:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 20 client.25573 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e4c7af0 0x56025e4c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 4 client.25573 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.282+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56025e478cd0 con 0x56025e4c7af0 2023-10-17T01:15:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.283+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e4c7af0 0x56025e4c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.283+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e4c7af0 0x56025e4c9f10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4340d2031c358973 server_cookie=8d56138066dec12a in_seq=0 out_seq=0 2023-10-17T01:15:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.283+0000 7f2a837fe700 10 client.25573 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.284+0000 7f2a837fe700 1 -- 172.21.15.45:0/266505321 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x56025e478cd0 con 0x56025e4c7af0 2023-10-17T01:15:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.284+0000 7f2a837fe700 10 client.25573 handle_command_reply: tid=0 2023-10-17T01:15:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.284+0000 7f2abddf2b80 1 client.25573 shutdown 2023-10-17T01:15:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.284+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 msgr2=0x56025e4bbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.284+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4bbbc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a94012810 tx=0x7f2a94012b20 comp rx=0 tx=0).stop 2023-10-17T01:15:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 shutdown_connections 2023-10-17T01:15:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e4c7af0 0x56025e4c9f10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4bbbc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2a837fe700 0 client.25573 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/266505321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4bc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 >> 172.21.15.45:0/266505321 conn(0x56025e409070 msgr2=0x56025e4a3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 shutdown_connections 2023-10-17T01:15:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 1 -- 172.21.15.45:0/266505321 wait complete. 2023-10-17T01:15:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.285+0000 7f2abddf2b80 20 client.25573 trim_cache size 0 max 16384 2023-10-17T01:15:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.315+0000 7f2abddf2b80 1 Processor -- start 2023-10-17T01:15:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 -- start start 2023-10-17T01:15:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4a4db0 con 0x56025e4b29a0 2023-10-17T01:15:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4bb070 con 0x56025e4b35a0 2023-10-17T01:15:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4b3430 con 0x56025e4b2040 2023-10-17T01:15:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.316+0000 7f2aa899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.317+0000 7f2aa899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54712/0 (socket says 172.21.15.45:54712) 2023-10-17T01:15:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.317+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.317+0000 7f2aa899c700 1 -- 172.21.15.45:0/253460122 learned_addr learned my addr 172.21.15.45:0/253460122 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.317+0000 7f2aa899c700 1 -- 172.21.15.45:0/253460122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 msgr2=0x56025e4ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.317+0000 7f2aa899c700 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.317+0000 7f2aa899c700 1 -- 172.21.15.45:0/253460122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 msgr2=0x56025e4a4130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.318+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.319+0000 7f2aa899c700 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.319+0000 7f2aa899c700 1 -- 172.21.15.45:0/253460122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56025e24f1f0 con 0x56025e4b35a0 2023-10-17T01:15:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.319+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.320+0000 7f2aa899c700 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b99d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b0d0 tx=0x7f2a9c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db17db8a8a40c6c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.320+0000 7f2a8affd700 1 -- 172.21.15.45:0/253460122 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00e070 con 0x56025e4b35a0 2023-10-17T01:15:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.320+0000 7f2a8affd700 1 -- 172.21.15.45:0/253460122 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a9c003dc0 con 0x56025e4b35a0 2023-10-17T01:15:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.320+0000 7f2a8affd700 1 -- 172.21.15.45:0/253460122 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00e070 con 0x56025e4b35a0 2023-10-17T01:15:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 -- 172.21.15.45:0/253460122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 msgr2=0x56025e4b99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b99d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b0d0 tx=0x7f2a9c0099e0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 -- 172.21.15.45:0/253460122 shutdown_connections 2023-10-17T01:15:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b35a0 0x56025e4b99d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b2040 0x56025e4ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/253460122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.322+0000 7f2abddf2b80 1 -- 172.21.15.45:0/253460122 >> 172.21.15.45:0/253460122 conn(0x56025e409070 msgr2=0x56025e4a3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.323+0000 7f2abddf2b80 1 -- 172.21.15.45:0/253460122 shutdown_connections 2023-10-17T01:15:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.323+0000 7f2abddf2b80 1 -- 172.21.15.45:0/253460122 wait complete. 2023-10-17T01:15:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.323+0000 7f2abddf2b80 1 -- start start 2023-10-17T01:15:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.324+0000 7f2abddf2b80 1 Processor -- start 2023-10-17T01:15:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 0x56025e4c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b35a0 0x56025e4bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4bdf20 con 0x56025e4b29a0 2023-10-17T01:15:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4be090 con 0x56025e4b2040 2023-10-17T01:15:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e4be250 con 0x56025e4b35a0 2023-10-17T01:15:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2abddf2b80 10 client.0 _ll_register_callbacks cb 0x7f2a9400beb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 0x56025e4c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 0x56025e4c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54726/0 (socket says 172.21.15.45:54726) 2023-10-17T01:15:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2aa3fff700 1 -- 172.21.15.45:0/986391087 learned_addr learned my addr 172.21.15.45:0/986391087 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.325+0000 7f2aa899c700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b35a0 0x56025e4bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.326+0000 7f2aa3fff700 1 -- 172.21.15.45:0/986391087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b35a0 msgr2=0x56025e4bd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.326+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b35a0 0x56025e4bd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.326+0000 7f2aa3fff700 1 -- 172.21.15.45:0/986391087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 msgr2=0x56025e4c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.326+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.326+0000 7f2aa3fff700 1 -- 172.21.15.45:0/986391087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa4109b20 con 0x56025e4b2040 2023-10-17T01:15:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.326+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.327+0000 7f2aa3fff700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 0x56025e4c0b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b2e0 tx=0x7f2a94010ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88477985f810a2a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.327+0000 7f2a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.327+0000 7f2a837fe700 1 -- 172.21.15.45:0/986391087 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94005ee0 con 0x56025e4b2040 2023-10-17T01:15:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.327+0000 7f2a837fe700 1 -- 172.21.15.45:0/986391087 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2a94000e40 con 0x56025e4b2040 2023-10-17T01:15:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.328+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a980174a0 con 0x56025e4b2040 2023-10-17T01:15:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.328+0000 7f2a837fe700 1 -- 172.21.15.45:0/986391087 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a94027420 con 0x56025e4b2040 2023-10-17T01:15:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.328+0000 7f2abddf2b80 10 client.25601 fetch_fsmap learned FSMap version 33 2023-10-17T01:15:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.328+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56025e227250 con 0x56025e4b2040 2023-10-17T01:15:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.328+0000 7f2a837fe700 1 -- 172.21.15.45:0/986391087 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a980174a0 con 0x56025e4b2040 2023-10-17T01:15:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2a837fe700 1 -- 172.21.15.45:0/986391087 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2a94037050 con 0x56025e4b2040 2023-10-17T01:15:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2abddf2b80 10 client.25601 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:15:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2abddf2b80 10 client.25601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2abddf2b80 20 client.25601 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e52bb50 0x56025e52df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2abddf2b80 4 client.25601 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:15:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.329+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56025e3f3ce0 con 0x56025e52bb50 2023-10-17T01:15:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.330+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e52bb50 0x56025e52df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.331+0000 7f2aa37fe700 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e52bb50 0x56025e52df70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4541dd04b9567ff server_cookie=c13a996cd776358e in_seq=0 out_seq=0 2023-10-17T01:15:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.331+0000 7f2a837fe700 10 client.25601 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.349+0000 7f2a837fe700 1 -- 172.21.15.45:0/986391087 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56025e3f3ce0 con 0x56025e52bb50 2023-10-17T01:15:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.350+0000 7f2a837fe700 10 client.25601 handle_command_reply: tid=0 2023-10-17T01:15:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.350+0000 7f2abddf2b80 1 client.25601 shutdown 2023-10-17T01:15:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 msgr2=0x56025e4c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 0x56025e4c0b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b2e0 tx=0x7f2a94010ef0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 shutdown_connections 2023-10-17T01:15:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56025e4b35a0 0x56025e4bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56025e52bb50 0x56025e52df70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2a837fe700 0 client.25601 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56025e4b2040 0x56025e4c0b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 --2- 172.21.15.45:0/986391087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56025e4b29a0 0x56025e4c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 >> 172.21.15.45:0/986391087 conn(0x56025e409070 msgr2=0x56025e4b99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.351+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 shutdown_connections 2023-10-17T01:15:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.352+0000 7f2abddf2b80 1 -- 172.21.15.45:0/986391087 wait complete. 2023-10-17T01:15:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.352+0000 7f2abddf2b80 20 client.25601 trim_cache size 0 max 16384 2023-10-17T01:15:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:15:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:15:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:15:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:15:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2a8c0a0b20 msgr2=0x7f2a8c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2a8c0a0b20 0x7f2a8c0a2fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009cd0 tx=0x7f2a9c005040 comp rx=0 tx=0).stop 2023-10-17T01:15:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4101f80 msgr2=0x7f2aa4197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4101f80 0x7f2aa4197260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800b450 tx=0x7f2a9800b760 comp rx=0 tx=0).stop 2023-10-17T01:15:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 shutdown_connections 2023-10-17T01:15:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2a8c0a0b20 0x7f2a8c0a2fe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa4102920 0x7f2aa4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa4100d90 0x7f2aa4196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 --2- 172.21.15.45:0/499043940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2aa4101f80 0x7f2aa4197260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 >> 172.21.15.45:0/499043940 conn(0x7f2aa40fc300 msgr2=0x7f2aa40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 shutdown_connections 2023-10-17T01:15:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:44.353+0000 7f2aaa3ff700 1 -- 172.21.15.45:0/499043940 wait complete. 2023-10-17T01:15:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 35.0 s, 2207.0 tps, lat 45.327 ms stddev 63.286 2023-10-17T01:15:45.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:45 smithi045 ceph-mon[98474]: pgmap v351: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 1.25k op/s 2023-10-17T01:15:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:45 smithi060 ceph-mon[134136]: pgmap v351: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 1.25k op/s 2023-10-17T01:15:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:45 smithi181 ceph-mon[134520]: pgmap v351: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 1.25k op/s 2023-10-17T01:15:47.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:47 smithi045 ceph-mon[98474]: pgmap v352: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 MiB/s wr, 1.04k op/s 2023-10-17T01:15:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:47 smithi060 ceph-mon[134136]: pgmap v352: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 MiB/s wr, 1.04k op/s 2023-10-17T01:15:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:47 smithi181 ceph-mon[134520]: pgmap v352: 97 pgs: 97 active+clean; 1.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 MiB/s wr, 1.04k op/s 2023-10-17T01:15:49.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:49 smithi045 ceph-mon[98474]: pgmap v353: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 MiB/s wr, 1.09k op/s 2023-10-17T01:15:49.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:49 smithi060 ceph-mon[134136]: pgmap v353: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 MiB/s wr, 1.09k op/s 2023-10-17T01:15:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:49 smithi181 ceph-mon[134520]: pgmap v353: 97 pgs: 97 active+clean; 1.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 MiB/s wr, 1.09k op/s 2023-10-17T01:15:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 40.0 s, 2100.2 tps, lat 45.776 ms stddev 62.242 2023-10-17T01:15:51.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:51 smithi045 ceph-mon[98474]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 49 MiB/s wr, 1.18k op/s 2023-10-17T01:15:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:51 smithi060 ceph-mon[134136]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 49 MiB/s wr, 1.18k op/s 2023-10-17T01:15:51.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:51 smithi181 ceph-mon[134520]: pgmap v354: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 49 MiB/s wr, 1.18k op/s 2023-10-17T01:15:53.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:53 smithi045 ceph-mon[98474]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 54 MiB/s wr, 1.50k op/s 2023-10-17T01:15:53.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:53 smithi060 ceph-mon[134136]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 54 MiB/s wr, 1.50k op/s 2023-10-17T01:15:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:53 smithi181 ceph-mon[134520]: pgmap v355: 97 pgs: 97 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 54 MiB/s wr, 1.50k op/s 2023-10-17T01:15:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 45.0 s, 3031.8 tps, lat 34.351 ms stddev 48.819 2023-10-17T01:15:55.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:55 smithi045 ceph-mon[98474]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 69 MiB/s wr, 2.35k op/s 2023-10-17T01:15:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:55 smithi060 ceph-mon[134136]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 69 MiB/s wr, 2.35k op/s 2023-10-17T01:15:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:55 smithi181 ceph-mon[134520]: pgmap v356: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 69 MiB/s wr, 2.35k op/s 2023-10-17T01:15:57.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:15:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.391+0000 7f578e5ff700 1 -- 172.21.15.45:0/848203654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108560 msgr2=0x7f57881089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.391+0000 7f578e5ff700 1 --2- 172.21.15.45:0/848203654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108560 0x7f57881089c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5770009920 tx=0x7f5770009c30 comp rx=0 tx=0).stop 2023-10-17T01:15:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.392+0000 7f578e5ff700 1 -- 172.21.15.45:0/848203654 shutdown_connections 2023-10-17T01:15:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.392+0000 7f578e5ff700 1 --2- 172.21.15.45:0/848203654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788108f00 0x7f578810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.392+0000 7f578e5ff700 1 --2- 172.21.15.45:0/848203654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5788107370 0x7f5788107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.392+0000 7f578e5ff700 1 --2- 172.21.15.45:0/848203654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108560 0x7f57881089c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.392+0000 7f578e5ff700 1 -- 172.21.15.45:0/848203654 >> 172.21.15.45:0/848203654 conn(0x7f57880762f0 msgr2=0x7f5788078750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.394+0000 7f578e5ff700 1 -- 172.21.15.45:0/848203654 shutdown_connections 2023-10-17T01:15:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.394+0000 7f578e5ff700 1 -- 172.21.15.45:0/848203654 wait complete. 2023-10-17T01:15:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.395+0000 7f578e5ff700 1 Processor -- start 2023-10-17T01:15:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.396+0000 7f578e5ff700 1 -- start start 2023-10-17T01:15:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.398+0000 7f578e5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788107370 0x7f578819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.398+0000 7f578e5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5788108560 0x7f578819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.398+0000 7f578e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 0x7f57881a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.398+0000 7f578e5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a4c70 con 0x7f5788108560 2023-10-17T01:15:57.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.398+0000 7f578e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a4db0 con 0x7f5788108f00 2023-10-17T01:15:57.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.398+0000 7f578e5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a4f40 con 0x7f5788107370 2023-10-17T01:15:57.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 0x7f57881a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 0x7f57881a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42854/0 (socket says 172.21.15.45:42854) 2023-10-17T01:15:57.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 -- 172.21.15.45:0/942644322 learned_addr learned my addr 172.21.15.45:0/942644322 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f577f7fe700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5788108560 0x7f578819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 -- 172.21.15.45:0/942644322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788107370 msgr2=0x7f578819d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f5787fff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788107370 0x7f578819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788107370 0x7f578819d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 -- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5788108560 msgr2=0x7f578819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5788108560 0x7f578819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f578cb9c700 1 -- 172.21.15.45:0/942644322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770009520 con 0x7f5788108f00 2023-10-17T01:15:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.399+0000 7f5787fff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788107370 0x7f578819d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.400+0000 7f578cb9c700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 0x7f57881a46a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f578000e970 tx=0x7f578000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.400+0000 7f5785ffb700 1 -- 172.21.15.45:0/942644322 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578000cbd0 con 0x7f5788108f00 2023-10-17T01:15:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.400+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57881a5220 con 0x7f5788108f00 2023-10-17T01:15:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.400+0000 7f5785ffb700 1 -- 172.21.15.45:0/942644322 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f578000ee50 con 0x7f5788108f00 2023-10-17T01:15:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.400+0000 7f5785ffb700 1 -- 172.21.15.45:0/942644322 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780005690 con 0x7f5788108f00 2023-10-17T01:15:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.400+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57881a5730 con 0x7f5788108f00 2023-10-17T01:15:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.404+0000 7f5785ffb700 1 -- 172.21.15.45:0/942644322 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f5780004750 con 0x7f5788108f00 2023-10-17T01:15:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.406+0000 7f5785ffb700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f57740a0c70 0x7f57740a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.407+0000 7f5785ffb700 1 -- 172.21.15.45:0/942644322 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5780014070 con 0x7f5788108f00 2023-10-17T01:15:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.407+0000 7f5787fff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f57740a0c70 0x7f57740a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.409+0000 7f5787fff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f57740a0c70 0x7f57740a3130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5778009690 tx=0x7f5778009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.415+0000 7f57a1e58b80 1 Processor -- start 2023-10-17T01:15:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.415+0000 7f57a1e58b80 1 -- start start 2023-10-17T01:15:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd328fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd329d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd32a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd280a10 con 0x5621cd32a2b0 2023-10-17T01:15:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd27e3d0 con 0x5621cd329500 2023-10-17T01:15:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd279cc0 con 0x5621cd328ba0 2023-10-17T01:15:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f577f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd329d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f577f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd329d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42866/0 (socket says 172.21.15.45:42866) 2023-10-17T01:15:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.416+0000 7f577f7fe700 1 -- 172.21.15.45:0/2334673838 learned_addr learned my addr 172.21.15.45:0/2334673838 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f578cb9c700 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd32a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577f7fe700 1 -- 172.21.15.45:0/2334673838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 msgr2=0x5621cd328fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:15:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577f7fe700 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd328fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577f7fe700 1 -- 172.21.15.45:0/2334673838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 msgr2=0x5621cd32a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577f7fe700 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd32a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577f7fe700 1 -- 172.21.15.45:0/2334673838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621cd0a7de0 con 0x5621cd329500 2023-10-17T01:15:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577f7fe700 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd329d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5770009230 tx=0x7f5770009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef757babd6428b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.417+0000 7f577e7fc700 1 -- 172.21.15.45:0/2334673838 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577001c070 con 0x5621cd329500 2023-10-17T01:15:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.418+0000 7f577e7fc700 1 -- 172.21.15.45:0/2334673838 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f577000ba20 con 0x5621cd329500 2023-10-17T01:15:57.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.418+0000 7f577e7fc700 1 -- 172.21.15.45:0/2334673838 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770005430 con 0x5621cd329500 2023-10-17T01:15:57.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.419+0000 7f57a1e58b80 1 -- 172.21.15.45:0/2334673838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 msgr2=0x5621cd329d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.419+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd329d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5770009230 tx=0x7f5770009440 comp rx=0 tx=0).stop 2023-10-17T01:15:57.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.420+0000 7f57a1e58b80 1 -- 172.21.15.45:0/2334673838 shutdown_connections 2023-10-17T01:15:57.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.420+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd32a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.420+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd328fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.420+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/2334673838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd329d20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.420+0000 7f57a1e58b80 1 -- 172.21.15.45:0/2334673838 >> 172.21.15.45:0/2334673838 conn(0x5621cd2880d0 msgr2=0x5621cd323950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.421+0000 7f57a1e58b80 1 -- 172.21.15.45:0/2334673838 shutdown_connections 2023-10-17T01:15:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.421+0000 7f57a1e58b80 1 -- 172.21.15.45:0/2334673838 wait complete. 2023-10-17T01:15:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.423+0000 7f57a1e58b80 1 -- start start 2023-10-17T01:15:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.423+0000 7f57a1e58b80 1 Processor -- start 2023-10-17T01:15:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd328ba0 0x5621cd332880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd332dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd339e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd216510 con 0x5621cd328ba0 2023-10-17T01:15:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd2161b0 con 0x5621cd329500 2023-10-17T01:15:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd0fd920 con 0x5621cd32a2b0 2023-10-17T01:15:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.425+0000 7f57a1e58b80 10 client.0 _ll_register_callbacks cb 0x5621cd163a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd339e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd339e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52652/0 (socket says 172.21.15.45:52652) 2023-10-17T01:15:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 -- 172.21.15.45:0/4162290906 learned_addr learned my addr 172.21.15.45:0/4162290906 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f577f7fe700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd332dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f5787fff700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd328ba0 0x5621cd332880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 -- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 msgr2=0x5621cd332dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd332dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 -- 172.21.15.45:0/4162290906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd328ba0 msgr2=0x5621cd332880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd328ba0 0x5621cd332880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 -- 172.21.15.45:0/4162290906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5788110100 con 0x5621cd32a2b0 2023-10-17T01:15:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f577f7fe700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd332dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f5787fff700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd328ba0 0x5621cd332880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:15:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.426+0000 7f578cb9c700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd339e00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f57800a1580 tx=0x7f578000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520cf13d57318f0a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.427+0000 7f57677fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:15:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.427+0000 7f57677fe700 1 -- 172.21.15.45:0/4162290906 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780099ea0 con 0x5621cd32a2b0 2023-10-17T01:15:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.427+0000 7f57677fe700 1 -- 172.21.15.45:0/4162290906 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f57800a1ab0 con 0x5621cd32a2b0 2023-10-17T01:15:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.427+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621cd2a4d10 con 0x5621cd32a2b0 2023-10-17T01:15:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.428+0000 7f57677fe700 1 -- 172.21.15.45:0/4162290906 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5780098f30 con 0x5621cd32a2b0 2023-10-17T01:15:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.428+0000 7f57a1e58b80 10 client.25585 fetch_fsmap learned FSMap version 33 2023-10-17T01:15:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.428+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5621cd090c60 con 0x5621cd32a2b0 2023-10-17T01:15:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.429+0000 7f57677fe700 1 -- 172.21.15.45:0/4162290906 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621cd2a4d10 con 0x5621cd32a2b0 2023-10-17T01:15:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.429+0000 7f57677fe700 1 -- 172.21.15.45:0/4162290906 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f57800989a0 con 0x5621cd32a2b0 2023-10-17T01:15:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f57a1e58b80 10 client.25585 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:15:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f57a1e58b80 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f57a1e58b80 20 client.25585 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd34f6d0 0x5621cd351af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f57a1e58b80 4 client.25585 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621cd2f7d30 con 0x5621cd34f6d0 2023-10-17T01:15:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.430+0000 7f5787fff700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd34f6d0 0x5621cd351af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.432+0000 7f5787fff700 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd34f6d0 0x5621cd351af0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219e09fe96ccc4d9 server_cookie=bbe80cfc0fbb82c7 in_seq=0 out_seq=0 2023-10-17T01:15:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.432+0000 7f57677fe700 10 client.25585 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.433+0000 7f57677fe700 1 -- 172.21.15.45:0/4162290906 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5621cd2f7d30 con 0x5621cd34f6d0 2023-10-17T01:15:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.433+0000 7f57677fe700 10 client.25585 handle_command_reply: tid=0 2023-10-17T01:15:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.433+0000 7f57a1e58b80 1 client.25585 shutdown 2023-10-17T01:15:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 msgr2=0x5621cd339e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd339e00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f57800a1580 tx=0x7f578000cde0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 shutdown_connections 2023-10-17T01:15:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd339e00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd328ba0 0x5621cd332880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd34f6d0 0x5621cd351af0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/4162290906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd332dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 >> 172.21.15.45:0/4162290906 conn(0x5621cd2880d0 msgr2=0x5621cd323020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.434+0000 7f57677fe700 0 client.25585 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.435+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 shutdown_connections 2023-10-17T01:15:57.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.435+0000 7f57a1e58b80 1 -- 172.21.15.45:0/4162290906 wait complete. 2023-10-17T01:15:57.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.435+0000 7f57a1e58b80 20 client.25585 trim_cache size 0 max 16384 2023-10-17T01:15:57.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.472+0000 7f57a1e58b80 1 Processor -- start 2023-10-17T01:15:57.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.473+0000 7f57a1e58b80 1 -- start start 2023-10-17T01:15:57.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.473+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd324650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.473+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd323020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.473+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd3306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.473+0000 7f577f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd323020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f577f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd323020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42886/0 (socket says 172.21.15.45:42886) 2023-10-17T01:15:57.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f5787fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd324650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd331d30 con 0x5621cd32a2b0 2023-10-17T01:15:57.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd3323f0 con 0x5621cd329500 2023-10-17T01:15:57.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd320420 con 0x5621cd328ba0 2023-10-17T01:15:57.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f577f7fe700 1 -- 172.21.15.45:0/1784066695 learned_addr learned my addr 172.21.15.45:0/1784066695 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.474+0000 7f5787fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd324650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52660/0 (socket says 172.21.15.45:52660) 2023-10-17T01:15:57.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f578cb9c700 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd3306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f5787fff700 1 -- 172.21.15.45:0/1784066695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 msgr2=0x5621cd323020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f5787fff700 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd323020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f5787fff700 1 -- 172.21.15.45:0/1784066695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 msgr2=0x5621cd3306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f5787fff700 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd3306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f5787fff700 1 -- 172.21.15.45:0/1784066695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621cd0ce250 con 0x5621cd328ba0 2023-10-17T01:15:57.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.475+0000 7f578cb9c700 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd3306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:57.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.476+0000 7f5787fff700 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd324650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5780096460 tx=0x7f5778011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d54c1a5b99a06da server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.476+0000 7f577effd700 1 -- 172.21.15.45:0/1784066695 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778024da0 con 0x5621cd328ba0 2023-10-17T01:15:57.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.476+0000 7f577effd700 1 -- 172.21.15.45:0/1784066695 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5778016960 con 0x5621cd328ba0 2023-10-17T01:15:57.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.476+0000 7f577effd700 1 -- 172.21.15.45:0/1784066695 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577802d6f0 con 0x5621cd328ba0 2023-10-17T01:15:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.477+0000 7f57a1e58b80 1 -- 172.21.15.45:0/1784066695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 msgr2=0x5621cd324650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.477+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd324650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5780096460 tx=0x7f5778011000 comp rx=0 tx=0).stop 2023-10-17T01:15:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.478+0000 7f57a1e58b80 1 -- 172.21.15.45:0/1784066695 shutdown_connections 2023-10-17T01:15:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.478+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd32a2b0 0x5621cd3306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.478+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd328ba0 0x5621cd324650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.478+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/1784066695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd329500 0x5621cd323020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.478+0000 7f57a1e58b80 1 -- 172.21.15.45:0/1784066695 >> 172.21.15.45:0/1784066695 conn(0x5621cd2880d0 msgr2=0x5621cd3238c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.481+0000 7f57a1e58b80 1 -- 172.21.15.45:0/1784066695 shutdown_connections 2023-10-17T01:15:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.481+0000 7f57a1e58b80 1 -- 172.21.15.45:0/1784066695 wait complete. 2023-10-17T01:15:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.481+0000 7f57a1e58b80 1 -- start start 2023-10-17T01:15:57.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.482+0000 7f57a1e58b80 1 Processor -- start 2023-10-17T01:15:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 0x5621cd337c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd329500 0x5621cd336230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd336770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd336e50 con 0x5621cd329500 2023-10-17T01:15:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd336fc0 con 0x5621cd328ba0 2023-10-17T01:15:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cd3a1060 con 0x5621cd32a2b0 2023-10-17T01:15:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.484+0000 7f57a1e58b80 10 client.0 _ll_register_callbacks cb 0x7f57800993a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.485+0000 7f5787fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 0x5621cd337c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.485+0000 7f5787fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 0x5621cd337c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42902/0 (socket says 172.21.15.45:42902) 2023-10-17T01:15:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.485+0000 7f5787fff700 1 -- 172.21.15.45:0/819678263 learned_addr learned my addr 172.21.15.45:0/819678263 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.486+0000 7f577f7fe700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd329500 0x5621cd336230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.486+0000 7f578cb9c700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd336770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.486+0000 7f5787fff700 1 -- 172.21.15.45:0/819678263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 msgr2=0x5621cd336770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.486+0000 7f5787fff700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd336770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.486+0000 7f5787fff700 1 -- 172.21.15.45:0/819678263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd329500 msgr2=0x5621cd336230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.486+0000 7f5787fff700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd329500 0x5621cd336230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.488+0000 7f5787fff700 1 -- 172.21.15.45:0/819678263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778007d50 con 0x5621cd328ba0 2023-10-17T01:15:57.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.488+0000 7f5787fff700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 0x5621cd337c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5778024e50 tx=0x7f5778024e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0951e99f9f39b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.489+0000 7f57677fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:57.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.489+0000 7f57677fe700 1 -- 172.21.15.45:0/819678263 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778031040 con 0x5621cd328ba0 2023-10-17T01:15:57.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.489+0000 7f57677fe700 1 -- 172.21.15.45:0/819678263 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5778016cd0 con 0x5621cd328ba0 2023-10-17T01:15:57.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.490+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f577000ba20 con 0x5621cd328ba0 2023-10-17T01:15:57.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.490+0000 7f57677fe700 1 -- 172.21.15.45:0/819678263 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5778016e90 con 0x5621cd328ba0 2023-10-17T01:15:57.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.491+0000 7f57a1e58b80 10 client.25619 fetch_fsmap learned FSMap version 33 2023-10-17T01:15:57.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.491+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5621cd0a62b0 con 0x5621cd328ba0 2023-10-17T01:15:57.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57677fe700 1 -- 172.21.15.45:0/819678263 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f577000ba20 con 0x5621cd328ba0 2023-10-17T01:15:57.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57677fe700 1 -- 172.21.15.45:0/819678263 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f577803d070 con 0x5621cd328ba0 2023-10-17T01:15:57.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57a1e58b80 10 client.25619 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:15:57.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57a1e58b80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:57.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57a1e58b80 20 client.25619 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:57.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd3a2970 0x5621cd3a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57a1e58b80 4 client.25619 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:15:57.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.492+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5621cd272d40 con 0x5621cd3a2970 2023-10-17T01:15:57.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.493+0000 7f577f7fe700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd3a2970 0x5621cd3a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.495+0000 7f577f7fe700 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd3a2970 0x5621cd3a4d90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9b567f4e2cb9e6 server_cookie=c96ab23bd20f56ca in_seq=0 out_seq=0 2023-10-17T01:15:57.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.495+0000 7f57677fe700 10 client.25619 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.495+0000 7f57677fe700 1 -- 172.21.15.45:0/819678263 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5621cd272d40 con 0x5621cd3a2970 2023-10-17T01:15:57.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.495+0000 7f57677fe700 10 client.25619 handle_command_reply: tid=0 2023-10-17T01:15:57.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.496+0000 7f57a1e58b80 1 client.25619 shutdown 2023-10-17T01:15:57.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.497+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 msgr2=0x5621cd337c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.497+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 0x5621cd337c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5778024e50 tx=0x7f5778024e80 comp rx=0 tx=0).stop 2023-10-17T01:15:57.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 shutdown_connections 2023-10-17T01:15:57.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5621cd32a2b0 0x5621cd336770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5621cd3a2970 0x5621cd3a4d90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621cd328ba0 0x5621cd337c20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 --2- 172.21.15.45:0/819678263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621cd329500 0x5621cd336230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 >> 172.21.15.45:0/819678263 conn(0x5621cd2880d0 msgr2=0x5621cd3306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57677fe700 0 client.25619 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 shutdown_connections 2023-10-17T01:15:57.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.498+0000 7f57a1e58b80 1 -- 172.21.15.45:0/819678263 wait complete. 2023-10-17T01:15:57.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.499+0000 7f57a1e58b80 20 client.25619 trim_cache size 0 max 16384 2023-10-17T01:15:57.503 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:15:57.503 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:15:57.504 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:15:57.504 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:15:57.504 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:15:57.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:15:57.504 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:15:57.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:15:57.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:15:57.505 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:15:57.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.503+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f57740a0c70 msgr2=0x7f57740a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.503+0000 7f578e5ff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f57740a0c70 0x7f57740a3130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5778009690 tx=0x7f5778009040 comp rx=0 tx=0).stop 2023-10-17T01:15:57.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.503+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 msgr2=0x7f57881a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.503+0000 7f578e5ff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 0x7f57881a46a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f578000e970 tx=0x7f578000ed30 comp rx=0 tx=0).stop 2023-10-17T01:15:57.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.504+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 shutdown_connections 2023-10-17T01:15:57.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.504+0000 7f578e5ff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f57740a0c70 0x7f57740a3130 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.504+0000 7f578e5ff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5788108f00 0x7f57881a46a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.504+0000 7f578e5ff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5788107370 0x7f578819d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.504+0000 7f578e5ff700 1 --2- 172.21.15.45:0/942644322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5788108560 0x7f578819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.504+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 >> 172.21.15.45:0/942644322 conn(0x7f57880762f0 msgr2=0x7f578810bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.507+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 shutdown_connections 2023-10-17T01:15:57.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:15:57.507+0000 7f578e5ff700 1 -- 172.21.15.45:0/942644322 wait complete. 2023-10-17T01:15:57.566 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:15:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.661+0000 7f47b159d700 1 -- 172.21.15.45:0/2666668575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac075710 msgr2=0x7f47ac075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.661+0000 7f47b159d700 1 --2- 172.21.15.45:0/2666668575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac075710 0x7f47ac075b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f479c0098f0 tx=0x7f479c009c00 comp rx=0 tx=0).stop 2023-10-17T01:15:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.661+0000 7f47b159d700 1 -- 172.21.15.45:0/2666668575 shutdown_connections 2023-10-17T01:15:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.662+0000 7f47b159d700 1 --2- 172.21.15.45:0/2666668575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 0x7f47ac077b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.662+0000 7f47b159d700 1 --2- 172.21.15.45:0/2666668575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac075710 0x7f47ac075b30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.662+0000 7f47b159d700 1 --2- 172.21.15.45:0/2666668575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ac076950 0x7f47ac076dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.662+0000 7f47b159d700 1 -- 172.21.15.45:0/2666668575 >> 172.21.15.45:0/2666668575 conn(0x7f47ac0fe320 msgr2=0x7f47ac100760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.663+0000 7f47b159d700 1 -- 172.21.15.45:0/2666668575 shutdown_connections 2023-10-17T01:15:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.663+0000 7f47b159d700 1 -- 172.21.15.45:0/2666668575 wait complete. 2023-10-17T01:15:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.664+0000 7f47b159d700 1 Processor -- start 2023-10-17T01:15:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.665+0000 7f47b159d700 1 -- start start 2023-10-17T01:15:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.665+0000 7f47b159d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ac075710 0x7f47ac19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b159d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 0x7f47ac19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b159d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 0x7f47ac1a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b159d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1a4a00 con 0x7f47ac075710 2023-10-17T01:15:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b159d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1a4b40 con 0x7f47ac076950 2023-10-17T01:15:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b159d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1a4cd0 con 0x7f47ac077310 2023-10-17T01:15:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 0x7f47ac19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 0x7f47ac1a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 0x7f47ac1a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52694/0 (socket says 172.21.15.45:52694) 2023-10-17T01:15:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 0x7f47ac19d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42924/0 (socket says 172.21.15.45:42924) 2023-10-17T01:15:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47b0d9c700 1 -- 172.21.15.45:0/1145530411 learned_addr learned my addr 172.21.15.45:0/1145530411 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47abfff700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ac075710 0x7f47ac19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 -- 172.21.15.45:0/1145530411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 msgr2=0x7f47ac1a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 0x7f47ac1a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 -- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ac075710 msgr2=0x7f47ac19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ac075710 0x7f47ac19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.666+0000 7f47ab7fe700 1 -- 172.21.15.45:0/1145530411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009520 con 0x7f47ac076950 2023-10-17T01:15:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.667+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 0x7f47ac19d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47a000c970 tx=0x7f47a000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.668+0000 7f47a97fa700 1 -- 172.21.15.45:0/1145530411 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0004440 con 0x7f47ac076950 2023-10-17T01:15:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.668+0000 7f47a97fa700 1 -- 172.21.15.45:0/1145530411 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a0003b40 con 0x7f47ac076950 2023-10-17T01:15:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.668+0000 7f47a97fa700 1 -- 172.21.15.45:0/1145530411 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0007bc0 con 0x7f47ac076950 2023-10-17T01:15:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.669+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac1a4fb0 con 0x7f47ac076950 2023-10-17T01:15:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.669+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac1a53e0 con 0x7f47ac076950 2023-10-17T01:15:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.679+0000 7f47a97fa700 1 -- 172.21.15.45:0/1145530411 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f47a0003cb0 con 0x7f47ac076950 2023-10-17T01:15:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.681+0000 7f47c618bb80 1 Processor -- start 2023-10-17T01:15:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.681+0000 7f47c618bb80 1 -- start start 2023-10-17T01:15:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4828b90 0x55fcc4828fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4829980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 0x55fcc482a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47c618bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc47809b0 con 0x55fcc4828b90 2023-10-17T01:15:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47c618bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc477e370 con 0x55fcc4829f10 2023-10-17T01:15:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47c618bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc4779c60 con 0x55fcc48294f0 2023-10-17T01:15:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.682+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 0x55fcc482a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 0x55fcc482a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42928/0 (socket says 172.21.15.45:42928) 2023-10-17T01:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47ab7fe700 1 -- 172.21.15.45:0/4104101558 learned_addr learned my addr 172.21.15.45:0/4104101558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47abfff700 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4829980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47ab7fe700 1 -- 172.21.15.45:0/4104101558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 msgr2=0x55fcc4829980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4829980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47ab7fe700 1 -- 172.21.15.45:0/4104101558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4828b90 msgr2=0x55fcc4828fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.683+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4828b90 0x55fcc4828fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.684+0000 7f47ab7fe700 1 -- 172.21.15.45:0/4104101558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc45a7d80 con 0x55fcc4829f10 2023-10-17T01:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.684+0000 7f47abfff700 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4829980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.684+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 0x55fcc482a3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47a00cfe20 tx=0x7f47a00d43c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e766a00e6798830 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.684+0000 7f47a97fa700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47940a0c20 0x7f47940a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.685+0000 7f47a97fa700 1 -- 172.21.15.45:0/1145530411 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47a00cbfd0 con 0x7f47ac076950 2023-10-17T01:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.685+0000 7f47927fc700 1 -- 172.21.15.45:0/4104101558 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0095930 con 0x55fcc4829f10 2023-10-17T01:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.685+0000 7f47927fc700 1 -- 172.21.15.45:0/4104101558 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a00993c0 con 0x55fcc4829f10 2023-10-17T01:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.685+0000 7f47927fc700 1 -- 172.21.15.45:0/4104101558 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0098f50 con 0x55fcc4829f10 2023-10-17T01:15:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.686+0000 7f47b0d9c700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47940a0c20 0x7f47940a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.687+0000 7f47b0d9c700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47940a0c20 0x7f47940a30e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55fcc4828930 tx=0x7f47a40099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 -- 172.21.15.45:0/4104101558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 msgr2=0x55fcc482a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 0x55fcc482a3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47a00cfe20 tx=0x7f47a00d43c0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 -- 172.21.15.45:0/4104101558 shutdown_connections 2023-10-17T01:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4829f10 0x55fcc482a3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4828b90 0x55fcc4828fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 --2- 172.21.15.45:0/4104101558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4829980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.688+0000 7f47c618bb80 1 -- 172.21.15.45:0/4104101558 >> 172.21.15.45:0/4104101558 conn(0x55fcc4788070 msgr2=0x55fcc4823660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.689+0000 7f47c618bb80 1 -- 172.21.15.45:0/4104101558 shutdown_connections 2023-10-17T01:15:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.689+0000 7f47c618bb80 1 -- 172.21.15.45:0/4104101558 wait complete. 2023-10-17T01:15:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.690+0000 7f47c618bb80 1 -- start start 2023-10-17T01:15:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.690+0000 7f47c618bb80 1 Processor -- start 2023-10-17T01:15:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4832b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4829f10 0x55fcc4839b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc47164b0 con 0x55fcc4829f10 2023-10-17T01:15:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc4716150 con 0x55fcc4828b90 2023-10-17T01:15:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc45fd8c0 con 0x55fcc48294f0 2023-10-17T01:15:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.693+0000 7f47c618bb80 10 client.0 _ll_register_callbacks cb 0x55fcc4663a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47abfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4832b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4829f10 0x55fcc4839b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47abfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42944/0 (socket says 172.21.15.45:42944) 2023-10-17T01:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47abfff700 1 -- 172.21.15.45:0/3741065348 learned_addr learned my addr 172.21.15.45:0/3741065348 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47abfff700 1 -- 172.21.15.45:0/3741065348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 msgr2=0x55fcc4832b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47abfff700 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4832b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.694+0000 7f47abfff700 1 -- 172.21.15.45:0/3741065348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4829f10 msgr2=0x55fcc4839b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.695+0000 7f47abfff700 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4829f10 0x55fcc4839b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.695+0000 7f47abfff700 1 -- 172.21.15.45:0/3741065348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac1a53e0 con 0x55fcc4828b90 2023-10-17T01:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.695+0000 7f47abfff700 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48325c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f479c00fa40 tx=0x7f479c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450c1520244ec8ac server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.695+0000 7f478b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:15:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.695+0000 7f478b7fe700 1 -- 172.21.15.45:0/3741065348 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c025070 con 0x55fcc4828b90 2023-10-17T01:15:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.696+0000 7f478b7fe700 1 -- 172.21.15.45:0/3741065348 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f479c005040 con 0x55fcc4828b90 2023-10-17T01:15:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.696+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcc47a4cb0 con 0x55fcc4828b90 2023-10-17T01:15:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.696+0000 7f478b7fe700 1 -- 172.21.15.45:0/3741065348 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479c01c070 con 0x55fcc4828b90 2023-10-17T01:15:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.697+0000 7f47c618bb80 10 client.25631 fetch_fsmap learned FSMap version 33 2023-10-17T01:15:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.697+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fcc4590c00 con 0x55fcc4828b90 2023-10-17T01:15:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.698+0000 7f478b7fe700 1 -- 172.21.15.45:0/3741065348 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcc47a4cb0 con 0x55fcc4828b90 2023-10-17T01:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.699+0000 7f478b7fe700 1 -- 172.21.15.45:0/3741065348 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f479c028030 con 0x55fcc4828b90 2023-10-17T01:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.699+0000 7f47c618bb80 10 client.25631 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.699+0000 7f47c618bb80 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.700+0000 7f47c618bb80 20 client.25631 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.700+0000 7f47c618bb80 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc483e520 0x55fcc4840940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.700+0000 7f47c618bb80 4 client.25631 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:15:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.700+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcc47f7cd0 con 0x55fcc483e520 2023-10-17T01:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.700+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc483e520 0x55fcc4840940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.701+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc483e520 0x55fcc4840940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3507b7404af55b server_cookie=df80fade9bac1736 in_seq=0 out_seq=0 2023-10-17T01:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.701+0000 7f478b7fe700 10 client.25631 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.702+0000 7f478b7fe700 1 -- 172.21.15.45:0/3741065348 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55fcc47f7cd0 con 0x55fcc483e520 2023-10-17T01:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.702+0000 7f478b7fe700 10 client.25631 handle_command_reply: tid=0 2023-10-17T01:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.702+0000 7f47c618bb80 1 client.25631 shutdown 2023-10-17T01:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.703+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 msgr2=0x55fcc48325c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.703+0000 7f47c618bb80 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48325c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f479c00fa40 tx=0x7f479c00b790 comp rx=0 tx=0).stop 2023-10-17T01:15:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.703+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 shutdown_connections 2023-10-17T01:15:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.703+0000 7f47c618bb80 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc4829f10 0x55fcc4839b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.703+0000 7f47c618bb80 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc483e520 0x55fcc4840940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.704+0000 7f47c618bb80 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48325c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.704+0000 7f47c618bb80 1 --2- 172.21.15.45:0/3741065348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc48294f0 0x55fcc4832b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.704+0000 7f478b7fe700 0 client.25631 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.704+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 >> 172.21.15.45:0/3741065348 conn(0x55fcc4788070 msgr2=0x55fcc4822db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.705+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 shutdown_connections 2023-10-17T01:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.705+0000 7f47c618bb80 1 -- 172.21.15.45:0/3741065348 wait complete. 2023-10-17T01:15:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.706+0000 7f47c618bb80 20 client.25631 trim_cache size 0 max 16384 2023-10-17T01:15:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 Processor -- start 2023-10-17T01:15:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 -- start start 2023-10-17T01:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc482a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc482ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc482d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc4833610 con 0x55fcc48294f0 2023-10-17T01:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc48203c0 con 0x55fcc4828b90 2023-10-17T01:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.757+0000 7f47c618bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc481b450 con 0x55fcc4829f10 2023-10-17T01:15:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.758+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc482d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.758+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc482d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52722/0 (socket says 172.21.15.45:52722) 2023-10-17T01:15:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.758+0000 7f47b0d9c700 1 -- 172.21.15.45:0/2875430896 learned_addr learned my addr 172.21.15.45:0/2875430896 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.759+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc482ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.759+0000 7f47abfff700 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc482a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.760+0000 7f47b0d9c700 1 -- 172.21.15.45:0/2875430896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 msgr2=0x55fcc482a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.760+0000 7f47b0d9c700 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc482a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.760+0000 7f47b0d9c700 1 -- 172.21.15.45:0/2875430896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 msgr2=0x55fcc482ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.760+0000 7f47b0d9c700 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc482ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.760+0000 7f47b0d9c700 1 -- 172.21.15.45:0/2875430896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc45ce1f0 con 0x55fcc4829f10 2023-10-17T01:15:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.762+0000 7f47b0d9c700 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc482d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47a40179e0 tx=0x7f47a4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe0c70c1cd4ff68 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.763+0000 7f4792ffd700 1 -- 172.21.15.45:0/2875430896 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40159a0 con 0x55fcc4829f10 2023-10-17T01:15:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.763+0000 7f4792ffd700 1 -- 172.21.15.45:0/2875430896 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a400f040 con 0x55fcc4829f10 2023-10-17T01:15:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.763+0000 7f4792ffd700 1 -- 172.21.15.45:0/2875430896 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40159a0 con 0x55fcc4829f10 2023-10-17T01:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.764+0000 7f47c618bb80 1 -- 172.21.15.45:0/2875430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 msgr2=0x55fcc482d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.764+0000 7f47c618bb80 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc482d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47a40179e0 tx=0x7f47a4014000 comp rx=0 tx=0).stop 2023-10-17T01:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.765+0000 7f47c618bb80 1 -- 172.21.15.45:0/2875430896 shutdown_connections 2023-10-17T01:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.765+0000 7f47c618bb80 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc482d3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.765+0000 7f47c618bb80 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc482a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.765+0000 7f47c618bb80 1 --2- 172.21.15.45:0/2875430896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc482ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.765+0000 7f47c618bb80 1 -- 172.21.15.45:0/2875430896 >> 172.21.15.45:0/2875430896 conn(0x55fcc4788070 msgr2=0x55fcc4830370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.766+0000 7f47c618bb80 1 -- 172.21.15.45:0/2875430896 shutdown_connections 2023-10-17T01:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.766+0000 7f47c618bb80 1 -- 172.21.15.45:0/2875430896 wait complete. 2023-10-17T01:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.767+0000 7f47c618bb80 1 -- start start 2023-10-17T01:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.767+0000 7f47c618bb80 1 Processor -- start 2023-10-17T01:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.769+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.769+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc4835cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.769+0000 7f47c618bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc48361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.769+0000 7f47c618bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc4836730 con 0x55fcc48294f0 2023-10-17T01:15:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.769+0000 7f47c618bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc48368a0 con 0x55fcc4828b90 2023-10-17T01:15:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.769+0000 7f47c618bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc4836a60 con 0x55fcc4829f10 2023-10-17T01:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.770+0000 7f47c618bb80 10 client.0 _ll_register_callbacks cb 0x7f479c00f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.770+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc48361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.770+0000 7f47b0d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc48361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52726/0 (socket says 172.21.15.45:52726) 2023-10-17T01:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.770+0000 7f47b0d9c700 1 -- 172.21.15.45:0/140831676 learned_addr learned my addr 172.21.15.45:0/140831676 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.770+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc4835cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.770+0000 7f47abfff700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.771+0000 7f47ab7fe700 1 -- 172.21.15.45:0/140831676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 msgr2=0x55fcc48361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.771+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc48361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.771+0000 7f47ab7fe700 1 -- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 msgr2=0x55fcc48376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.771+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48376a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.771+0000 7f47ab7fe700 1 -- 172.21.15.45:0/140831676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a4017660 con 0x55fcc48294f0 2023-10-17T01:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.771+0000 7f47b0d9c700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc48361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.772+0000 7f47ab7fe700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc4835cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47a00cfe20 tx=0x7f47a000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb740b2d914757b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.772+0000 7f478b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.773+0000 7f478b7fe700 1 -- 172.21.15.45:0/140831676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a00cf6c0 con 0x55fcc48294f0 2023-10-17T01:15:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.773+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47a00993c0 con 0x55fcc48294f0 2023-10-17T01:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.773+0000 7f478b7fe700 1 -- 172.21.15.45:0/140831676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47a0082e40 con 0x55fcc48294f0 2023-10-17T01:15:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.774+0000 7f478b7fe700 1 -- 172.21.15.45:0/140831676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a00cf8d0 con 0x55fcc48294f0 2023-10-17T01:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.775+0000 7f47c618bb80 10 client.15522 fetch_fsmap learned FSMap version 33 2023-10-17T01:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.775+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fcc45a6250 con 0x55fcc48294f0 2023-10-17T01:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f478b7fe700 1 -- 172.21.15.45:0/140831676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a00993c0 con 0x55fcc48294f0 2023-10-17T01:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f478b7fe700 1 -- 172.21.15.45:0/140831676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f47a0007d20 con 0x55fcc48294f0 2023-10-17T01:15:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f47c618bb80 10 client.15522 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:15:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f47c618bb80 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:15:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f47c618bb80 20 client.15522 populate_metadata read hostname 'smithi045' 2023-10-17T01:15:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f47c618bb80 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc48a21a0 0x55fcc48a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:15:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f47c618bb80 4 client.15522 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:15:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.776+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fcc4772ce0 con 0x55fcc48a21a0 2023-10-17T01:15:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.778+0000 7f47abfff700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc48a21a0 0x55fcc48a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:15:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.779+0000 7f47abfff700 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc48a21a0 0x55fcc48a45c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f438128049b0e74a server_cookie=ff8fb8d7eb25ceb7 in_seq=0 out_seq=0 2023-10-17T01:15:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.779+0000 7f478b7fe700 10 client.15522 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:57 smithi045 ceph-mon[98474]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 MiB/s wr, 2.05k op/s 2023-10-17T01:15:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.809+0000 7f478b7fe700 1 -- 172.21.15.45:0/140831676 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fcc4772ce0 con 0x55fcc48a21a0 2023-10-17T01:15:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.809+0000 7f478b7fe700 10 client.15522 handle_command_reply: tid=0 2023-10-17T01:15:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 client.15522 shutdown 2023-10-17T01:15:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 msgr2=0x55fcc4835cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc4835cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47a00cfe20 tx=0x7f47a000a560 comp rx=0 tx=0).stop 2023-10-17T01:15:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 shutdown_connections 2023-10-17T01:15:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcc4829f10 0x55fcc48361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fcc48a21a0 0x55fcc48a45c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f478b7fe700 0 client.15522 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:15:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fcc4828b90 0x55fcc48376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 --2- 172.21.15.45:0/140831676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcc48294f0 0x55fcc4835cb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.810+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 >> 172.21.15.45:0/140831676 conn(0x55fcc4788070 msgr2=0x55fcc482ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.811+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 shutdown_connections 2023-10-17T01:15:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.811+0000 7f47c618bb80 1 -- 172.21.15.45:0/140831676 wait complete. 2023-10-17T01:15:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.812+0000 7f47c618bb80 20 client.15522 trim_cache size 0 max 16384 2023-10-17T01:15:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:15:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:15:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:15:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:15:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.813+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47940a0c20 msgr2=0x7f47940a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.813+0000 7f47b159d700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47940a0c20 0x7f47940a30e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55fcc4828930 tx=0x7f47a40099e0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.813+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 msgr2=0x7f47ac19d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:15:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.813+0000 7f47b159d700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 0x7f47ac19d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47a000c970 tx=0x7f47a000cc80 comp rx=0 tx=0).stop 2023-10-17T01:15:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 shutdown_connections 2023-10-17T01:15:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f47940a0c20 0x7f47940a30e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47ac077310 0x7f47ac1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ac075710 0x7f47ac19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 --2- 172.21.15.45:0/1145530411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47ac076950 0x7f47ac19d3c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:15:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 >> 172.21.15.45:0/1145530411 conn(0x7f47ac0fe320 msgr2=0x7f47ac100060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:15:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 shutdown_connections 2023-10-17T01:15:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:15:57.814+0000 7f47b159d700 1 -- 172.21.15.45:0/1145530411 wait complete. 2023-10-17T01:15:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:57 smithi060 ceph-mon[134136]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 MiB/s wr, 2.05k op/s 2023-10-17T01:15:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:57 smithi181 ceph-mon[134520]: pgmap v357: 97 pgs: 97 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 MiB/s wr, 2.05k op/s 2023-10-17T01:15:59.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:15:59 smithi045 ceph-mon[98474]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 MiB/s wr, 2.08k op/s 2023-10-17T01:15:59.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:15:59 smithi060 ceph-mon[134136]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 MiB/s wr, 2.08k op/s 2023-10-17T01:15:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:15:59 smithi181 ceph-mon[134520]: pgmap v358: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 MiB/s wr, 2.08k op/s 2023-10-17T01:16:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 50.0 s, 3358.4 tps, lat 29.804 ms stddev 31.354 2023-10-17T01:16:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:01 smithi045 ceph-mon[98474]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 MiB/s wr, 2.07k op/s 2023-10-17T01:16:01.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:01 smithi060 ceph-mon[134136]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 MiB/s wr, 2.07k op/s 2023-10-17T01:16:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:01 smithi181 ceph-mon[134520]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 MiB/s wr, 2.07k op/s 2023-10-17T01:16:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:03 smithi045 ceph-mon[98474]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-10-17T01:16:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:03 smithi060 ceph-mon[134136]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-10-17T01:16:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:03 smithi181 ceph-mon[134520]: pgmap v360: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-10-17T01:16:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 55.0 s, 3325.8 tps, lat 30.104 ms stddev 30.996 2023-10-17T01:16:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:05 smithi045 ceph-mon[98474]: pgmap v361: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 37 MiB/s wr, 1.70k op/s 2023-10-17T01:16:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:05 smithi060 ceph-mon[134136]: pgmap v361: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 37 MiB/s wr, 1.70k op/s 2023-10-17T01:16:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:05 smithi181 ceph-mon[134520]: pgmap v361: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 37 MiB/s wr, 1.70k op/s 2023-10-17T01:16:07.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:07 smithi045 ceph-mon[98474]: pgmap v362: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 18 MiB/s wr, 748 op/s 2023-10-17T01:16:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:07 smithi060 ceph-mon[134136]: pgmap v362: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 18 MiB/s wr, 748 op/s 2023-10-17T01:16:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:07 smithi181 ceph-mon[134520]: pgmap v362: 97 pgs: 97 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 18 MiB/s wr, 748 op/s 2023-10-17T01:16:09.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:09 smithi045 ceph-mon[98474]: pgmap v363: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 19 MiB/s wr, 842 op/s 2023-10-17T01:16:09.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:09 smithi060 ceph-mon[134136]: pgmap v363: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 19 MiB/s wr, 842 op/s 2023-10-17T01:16:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:09 smithi181 ceph-mon[134520]: pgmap v363: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 19 MiB/s wr, 842 op/s 2023-10-17T01:16:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 60.0 s, 2556.1 tps, lat 39.000 ms stddev 43.844 2023-10-17T01:16:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 -- 172.21.15.45:0/2895508627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00108f40 msgr2=0x7faa0010f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 --2- 172.21.15.45:0/2895508627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00108f40 0x7faa0010f7f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b0a0 tx=0x7fa9f800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 -- 172.21.15.45:0/2895508627 shutdown_connections 2023-10-17T01:16:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 --2- 172.21.15.45:0/2895508627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa00108f40 0x7faa0010f7f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 --2- 172.21.15.45:0/2895508627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa001073b0 0x7faa001077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 --2- 172.21.15.45:0/2895508627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa001085a0 0x7faa00108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.078+0000 7faa057ff700 1 -- 172.21.15.45:0/2895508627 >> 172.21.15.45:0/2895508627 conn(0x7faa00076330 msgr2=0x7faa00078790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.079+0000 7faa057ff700 1 -- 172.21.15.45:0/2895508627 shutdown_connections 2023-10-17T01:16:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.079+0000 7faa057ff700 1 -- 172.21.15.45:0/2895508627 wait complete. 2023-10-17T01:16:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.081+0000 7faa057ff700 1 Processor -- start 2023-10-17T01:16:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.081+0000 7faa057ff700 1 -- start start 2023-10-17T01:16:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.081+0000 7faa057ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa001073b0 0x7faa0006f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.081+0000 7faa057ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 0x7faa0006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.081+0000 7faa057ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa00108f40 0x7faa00070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 0x7faa0006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 0x7faa0006fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:48964/0 (socket says 172.21.15.45:48964) 2023-10-17T01:16:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7faa057ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000710e0 con 0x7faa001073b0 2023-10-17T01:16:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7faa057ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00071250 con 0x7faa00108f40 2023-10-17T01:16:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7faa057ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00071410 con 0x7faa001085a0 2023-10-17T01:16:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3674719782 learned_addr learned my addr 172.21.15.45:0/3674719782 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9feffd700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa001073b0 0x7faa0006f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9ff7fe700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa00108f40 0x7faa00070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3674719782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa00108f40 msgr2=0x7faa00070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa00108f40 0x7faa00070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa001073b0 msgr2=0x7faa0006f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa001073b0 0x7faa0006f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.082+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3674719782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8009d00 con 0x7faa001085a0 2023-10-17T01:16:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.083+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 0x7faa0006fbb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b550 tx=0x7fa9e800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.083+0000 7fa9fcff9700 1 -- 172.21.15.45:0/3674719782 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011830 con 0x7faa001085a0 2023-10-17T01:16:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.083+0000 7fa9fcff9700 1 -- 172.21.15.45:0/3674719782 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9e8011e50 con 0x7faa001085a0 2023-10-17T01:16:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.083+0000 7fa9fcff9700 1 -- 172.21.15.45:0/3674719782 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8010dc0 con 0x7faa001085a0 2023-10-17T01:16:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.083+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa000716f0 con 0x7faa001085a0 2023-10-17T01:16:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.084+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa00071f20 con 0x7faa001085a0 2023-10-17T01:16:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.088+0000 7fa9fcff9700 1 -- 172.21.15.45:0/3674719782 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa9e8010380 con 0x7faa001085a0 2023-10-17T01:16:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.090+0000 7fa9fcff9700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9ec0a0ba0 0x7fa9ec0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.091+0000 7fa9feffd700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9ec0a0ba0 0x7fa9ec0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.091+0000 7fa9fcff9700 1 -- 172.21.15.45:0/3674719782 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9e80cbae0 con 0x7faa001085a0 2023-10-17T01:16:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.092+0000 7fa9feffd700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9ec0a0ba0 0x7fa9ec0a3060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa0006f3e0 tx=0x7fa9f0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.097+0000 7faa19100b80 1 Processor -- start 2023-10-17T01:16:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.098+0000 7faa19100b80 1 -- start start 2023-10-17T01:16:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7faa19100b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6276830 0x55f2f6276c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7faa19100b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 0x55f2f6277790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7faa19100b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6278210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7faa19100b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f61ce9b0 con 0x55f2f6277d90 2023-10-17T01:16:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7faa19100b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f61cc370 con 0x55f2f6276830 2023-10-17T01:16:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7faa19100b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f61c7c60 con 0x55f2f6277190 2023-10-17T01:16:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7fa9feffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6276830 0x55f2f6276c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.100+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 0x55f2f6277790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 0x55f2f6277790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:48974/0 (socket says 172.21.15.45:48974) 2023-10-17T01:16:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3218218206 learned_addr learned my addr 172.21.15.45:0/3218218206 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9ff7fe700 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6278210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3218218206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6276830 msgr2=0x55f2f6276c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6276830 0x55f2f6276c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3218218206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 msgr2=0x55f2f6278210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6278210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.101+0000 7fa9f67fc700 1 -- 172.21.15.45:0/3218218206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f5ff5d80 con 0x55f2f6277190 2023-10-17T01:16:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.102+0000 7fa9ff7fe700 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6278210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:16:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.103+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 0x55f2f6277790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8098e60 tx=0x7fa9e8099030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9334001c29451bd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.105+0000 7fa9f57fa700 1 -- 172.21.15.45:0/3218218206 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8099160 con 0x55f2f6277190 2023-10-17T01:16:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.105+0000 7fa9f57fa700 1 -- 172.21.15.45:0/3218218206 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9e809cf90 con 0x55f2f6277190 2023-10-17T01:16:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.105+0000 7fa9f57fa700 1 -- 172.21.15.45:0/3218218206 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8098650 con 0x55f2f6277190 2023-10-17T01:16:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.105+0000 7faa19100b80 1 -- 172.21.15.45:0/3218218206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 msgr2=0x55f2f6277790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.105+0000 7faa19100b80 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 0x55f2f6277790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8098e60 tx=0x7fa9e8099030 comp rx=0 tx=0).stop 2023-10-17T01:16:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 -- 172.21.15.45:0/3218218206 shutdown_connections 2023-10-17T01:16:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6278210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6276830 0x55f2f6276c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 --2- 172.21.15.45:0/3218218206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6277190 0x55f2f6277790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 -- 172.21.15.45:0/3218218206 >> 172.21.15.45:0/3218218206 conn(0x55f2f61d6070 msgr2=0x55f2f62712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 -- 172.21.15.45:0/3218218206 shutdown_connections 2023-10-17T01:16:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.106+0000 7faa19100b80 1 -- 172.21.15.45:0/3218218206 wait complete. 2023-10-17T01:16:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.107+0000 7faa19100b80 1 -- start start 2023-10-17T01:16:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.108+0000 7faa19100b80 1 Processor -- start 2023-10-17T01:16:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f6280410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 0x55f2f6280950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6287990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f61644b0 con 0x55f2f6277d90 2023-10-17T01:16:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f6164150 con 0x55f2f6277190 2023-10-17T01:16:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f604b8c0 con 0x55f2f6276830 2023-10-17T01:16:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7faa19100b80 10 client.0 _ll_register_callbacks cb 0x55f2f60b1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7fa9feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f6280410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.109+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 0x55f2f6280950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 0x55f2f6280950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60002/0 (socket says 172.21.15.45:60002) 2023-10-17T01:16:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 -- 172.21.15.45:0/300062961 learned_addr learned my addr 172.21.15.45:0/300062961 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9ff7fe700 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6287990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 -- 172.21.15.45:0/300062961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 msgr2=0x55f2f6280410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f6280410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 -- 172.21.15.45:0/300062961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 msgr2=0x55f2f6287990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6287990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.110+0000 7fa9f67fc700 1 -- 172.21.15.45:0/300062961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa00071f20 con 0x55f2f6277190 2023-10-17T01:16:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.111+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 0x55f2f6280950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8095660 tx=0x7fa9e8099740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e540419e262a8ef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.111+0000 7fa9e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:16:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.111+0000 7fa9e67fc700 1 -- 172.21.15.45:0/300062961 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8099160 con 0x55f2f6277190 2023-10-17T01:16:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.111+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2f61f2cb0 con 0x55f2f6277190 2023-10-17T01:16:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.111+0000 7fa9e67fc700 1 -- 172.21.15.45:0/300062961 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9e80972a0 con 0x55f2f6277190 2023-10-17T01:16:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.112+0000 7fa9e67fc700 1 -- 172.21.15.45:0/300062961 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9e80b72c0 con 0x55f2f6277190 2023-10-17T01:16:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.112+0000 7faa19100b80 10 client.25646 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.112+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f2f5fdec00 con 0x55f2f6277190 2023-10-17T01:16:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.113+0000 7fa9e67fc700 1 -- 172.21.15.45:0/300062961 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9e80a5080 con 0x55f2f6277190 2023-10-17T01:16:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.113+0000 7fa9e67fc700 1 -- 172.21.15.45:0/300062961 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa9e807e030 con 0x55f2f6277190 2023-10-17T01:16:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.116+0000 7faa19100b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.116+0000 7faa19100b80 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.116+0000 7faa19100b80 20 client.25646 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.116+0000 7faa19100b80 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f628bdf0 0x55f2f628e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.116+0000 7faa19100b80 4 client.25646 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:16:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.116+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2f6245cd0 con 0x55f2f628bdf0 2023-10-17T01:16:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.117+0000 7fa9feffd700 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f628bdf0 0x55f2f628e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.119+0000 7fa9e67fc700 10 client.25646 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.119+0000 7fa9feffd700 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f628bdf0 0x55f2f628e210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b667505517fb7fdc server_cookie=e87988f6643b7b5a in_seq=0 out_seq=0 2023-10-17T01:16:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.120+0000 7fa9e67fc700 1 -- 172.21.15.45:0/300062961 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55f2f6245cd0 con 0x55f2f628bdf0 2023-10-17T01:16:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.120+0000 7fa9e67fc700 10 client.25646 handle_command_reply: tid=0 2023-10-17T01:16:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.120+0000 7faa19100b80 1 client.25646 shutdown 2023-10-17T01:16:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 msgr2=0x55f2f6280950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 0x55f2f6280950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8095660 tx=0x7fa9e8099740 comp rx=0 tx=0).stop 2023-10-17T01:16:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 shutdown_connections 2023-10-17T01:16:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f6287990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f6280410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f628bdf0 0x55f2f628e210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 --2- 172.21.15.45:0/300062961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277190 0x55f2f6280950 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8095660 tx=0x7fa9e8099740 comp rx=0 tx=0).stop 2023-10-17T01:16:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7fa9e67fc700 0 client.25646 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 >> 172.21.15.45:0/300062961 conn(0x55f2f61d6070 msgr2=0x55f2f6270a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 shutdown_connections 2023-10-17T01:16:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.122+0000 7faa19100b80 1 -- 172.21.15.45:0/300062961 wait complete. 2023-10-17T01:16:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.123+0000 7faa19100b80 20 client.25646 trim_cache size 0 max 16384 2023-10-17T01:16:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.153+0000 7faa19100b80 1 Processor -- start 2023-10-17T01:16:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 -- start start 2023-10-17T01:16:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f627eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277d90 0x55f2f6280410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6285940 0x55f2f6280950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f628bc00 con 0x55f2f6285940 2023-10-17T01:16:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f627f8c0 con 0x55f2f6277d90 2023-10-17T01:16:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.154+0000 7faa19100b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f627ff80 con 0x55f2f6276830 2023-10-17T01:16:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277d90 0x55f2f6280410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f627eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9f67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277d90 0x55f2f6280410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60016/0 (socket says 172.21.15.45:60016) 2023-10-17T01:16:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9f67fc700 1 -- 172.21.15.45:0/1651731615 learned_addr learned my addr 172.21.15.45:0/1651731615 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9feffd700 1 -- 172.21.15.45:0/1651731615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277d90 msgr2=0x55f2f6280410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9feffd700 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277d90 0x55f2f6280410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9feffd700 1 -- 172.21.15.45:0/1651731615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6285940 msgr2=0x55f2f6280950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9feffd700 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6285940 0x55f2f6280950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.155+0000 7fa9feffd700 1 -- 172.21.15.45:0/1651731615 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f601c1f0 con 0x55f2f6276830 2023-10-17T01:16:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.156+0000 7fa9feffd700 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f627eff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55f2f628e7f0 tx=0x7fa9f0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8128edb4477a220f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.156+0000 7fa9f5ffb700 1 -- 172.21.15.45:0/1651731615 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0027070 con 0x55f2f6276830 2023-10-17T01:16:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.156+0000 7fa9f5ffb700 1 -- 172.21.15.45:0/1651731615 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f000ecf0 con 0x55f2f6276830 2023-10-17T01:16:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.156+0000 7fa9f5ffb700 1 -- 172.21.15.45:0/1651731615 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0015e90 con 0x55f2f6276830 2023-10-17T01:16:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 -- 172.21.15.45:0/1651731615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 msgr2=0x55f2f627eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f627eff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55f2f628e7f0 tx=0x7fa9f0011000 comp rx=0 tx=0).stop 2023-10-17T01:16:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 -- 172.21.15.45:0/1651731615 shutdown_connections 2023-10-17T01:16:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6285940 0x55f2f6280950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f627eff0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 --2- 172.21.15.45:0/1651731615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6277d90 0x55f2f6280410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.158+0000 7faa19100b80 1 -- 172.21.15.45:0/1651731615 >> 172.21.15.45:0/1651731615 conn(0x55f2f61d6070 msgr2=0x55f2f627e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.159+0000 7faa19100b80 1 -- 172.21.15.45:0/1651731615 shutdown_connections 2023-10-17T01:16:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.159+0000 7faa19100b80 1 -- 172.21.15.45:0/1651731615 wait complete. 2023-10-17T01:16:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.160+0000 7faa19100b80 1 -- start start 2023-10-17T01:16:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.160+0000 7faa19100b80 1 Processor -- start 2023-10-17T01:16:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.162+0000 7faa19100b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f62d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7faa19100b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f62d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7faa19100b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 0x55f2f62dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7fa9feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f62d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7fa9feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f62d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:49004/0 (socket says 172.21.15.45:49004) 2023-10-17T01:16:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7faa19100b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f62dd0c0 con 0x55f2f6277d90 2023-10-17T01:16:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7faa19100b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f62dd200 con 0x55f2f6285940 2023-10-17T01:16:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7faa19100b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f62dd3c0 con 0x55f2f6276830 2023-10-17T01:16:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7fa9ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 0x55f2f62dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7faa19100b80 10 client.0 _ll_register_callbacks cb 0x7fa9e8098f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7fa9ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 0x55f2f62dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60030/0 (socket says 172.21.15.45:60030) 2023-10-17T01:16:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.163+0000 7fa9ff7fe700 1 -- 172.21.15.45:0/586879937 learned_addr learned my addr 172.21.15.45:0/586879937 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f62d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9feffd700 1 -- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 msgr2=0x55f2f62dcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9feffd700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 0x55f2f62dcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9feffd700 1 -- 172.21.15.45:0/586879937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 msgr2=0x55f2f62d5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9feffd700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f62d5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9feffd700 1 -- 172.21.15.45:0/586879937 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f001a040 con 0x55f2f6276830 2023-10-17T01:16:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9ff7fe700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 0x55f2f62dcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.164+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f62d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:16:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.165+0000 7fa9feffd700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f62d5990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f0015ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82fcd906c0276148 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.165+0000 7fa9e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:16:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.166+0000 7fa9e67fc700 1 -- 172.21.15.45:0/586879937 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0027040 con 0x55f2f6276830 2023-10-17T01:16:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.166+0000 7fa9e67fc700 1 -- 172.21.15.45:0/586879937 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9f0015850 con 0x55f2f6276830 2023-10-17T01:16:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.166+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9e809cf90 con 0x55f2f6276830 2023-10-17T01:16:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7fa9e67fc700 1 -- 172.21.15.45:0/586879937 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f0015a10 con 0x55f2f6276830 2023-10-17T01:16:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 10 client.25627 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f2f5ff4250 con 0x55f2f6276830 2023-10-17T01:16:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7fa9e67fc700 1 -- 172.21.15.45:0/586879937 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f003e030 con 0x55f2f6276830 2023-10-17T01:16:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7fa9e67fc700 1 -- 172.21.15.45:0/586879937 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa9f0031480 con 0x55f2f6276830 2023-10-17T01:16:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 20 client.25627 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f62dee70 0x55f2f62e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 4 client.25627 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.168+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2f61c0ce0 con 0x55f2f62dee70 2023-10-17T01:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.170+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f62dee70 0x55f2f62e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.171+0000 7fa9f67fc700 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f62dee70 0x55f2f62e1290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701fd7efb4b5b4fc server_cookie=12db2c091ae3aa98 in_seq=0 out_seq=0 2023-10-17T01:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.172+0000 7fa9e67fc700 10 client.25627 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.216+0000 7fa9e67fc700 1 -- 172.21.15.45:0/586879937 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2f61c0ce0 con 0x55f2f62dee70 2023-10-17T01:16:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.216+0000 7fa9e67fc700 10 client.25627 handle_command_reply: tid=0 2023-10-17T01:16:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.217+0000 7faa19100b80 1 client.25627 shutdown 2023-10-17T01:16:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 msgr2=0x55f2f62d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f62d5990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f0015ea0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 shutdown_connections 2023-10-17T01:16:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f2f62dee70 0x55f2f62e1290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f2f6285940 0x55f2f62dcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f2f6276830 0x55f2f62d5990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 --2- 172.21.15.45:0/586879937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2f6277d90 0x55f2f62d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 >> 172.21.15.45:0/586879937 conn(0x55f2f61d6070 msgr2=0x55f2f627eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.218+0000 7fa9e67fc700 0 client.25627 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.219+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 shutdown_connections 2023-10-17T01:16:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.219+0000 7faa19100b80 1 -- 172.21.15.45:0/586879937 wait complete. 2023-10-17T01:16:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.220+0000 7faa19100b80 20 client.25627 trim_cache size 0 max 16384 2023-10-17T01:16:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:16:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:16:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:16:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.221+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9ec0a0ba0 msgr2=0x7fa9ec0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.221+0000 7faa057ff700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9ec0a0ba0 0x7fa9ec0a3060 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa0006f3e0 tx=0x7fa9f0009190 comp rx=0 tx=0).stop 2023-10-17T01:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 msgr2=0x7faa0006fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 0x7faa0006fbb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b550 tx=0x7fa9e800b860 comp rx=0 tx=0).stop 2023-10-17T01:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 shutdown_connections 2023-10-17T01:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9ec0a0ba0 0x7fa9ec0a3060 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa00108f40 0x7faa00070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa001073b0 0x7faa0006f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 --2- 172.21.15.45:0/3674719782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa001085a0 0x7faa0006fbb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 >> 172.21.15.45:0/3674719782 conn(0x7faa00076330 msgr2=0x7faa0010ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 shutdown_connections 2023-10-17T01:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:11.222+0000 7faa057ff700 1 -- 172.21.15.45:0/3674719782 wait complete. 2023-10-17T01:16:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:11 smithi045 ceph-mon[98474]: pgmap v364: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 19 MiB/s wr, 943 op/s 2023-10-17T01:16:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:11 smithi060 ceph-mon[134136]: pgmap v364: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 19 MiB/s wr, 943 op/s 2023-10-17T01:16:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:11 smithi181 ceph-mon[134520]: pgmap v364: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 19 MiB/s wr, 943 op/s 2023-10-17T01:16:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:13 smithi045 ceph-mon[98474]: pgmap v365: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 23 MiB/s wr, 982 op/s 2023-10-17T01:16:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:13 smithi060 ceph-mon[134136]: pgmap v365: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 23 MiB/s wr, 982 op/s 2023-10-17T01:16:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:13 smithi181 ceph-mon[134520]: pgmap v365: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 23 MiB/s wr, 982 op/s 2023-10-17T01:16:14.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:16:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:16:14.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:16:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 65.0 s, 2754.5 tps, lat 36.284 ms stddev 41.583 2023-10-17T01:16:15.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:15 smithi045 ceph-mon[98474]: pgmap v366: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-10-17T01:16:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:15 smithi060 ceph-mon[134136]: pgmap v366: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-10-17T01:16:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:15 smithi181 ceph-mon[134520]: pgmap v366: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-10-17T01:16:16.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:16:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:16:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:16:16.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:16:16.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:16:16.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:16:16.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:16:16.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:16:16.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:16:17.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:17 smithi045 ceph-mon[98474]: pgmap v367: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 974 op/s 2023-10-17T01:16:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:17 smithi060 ceph-mon[134136]: pgmap v367: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 974 op/s 2023-10-17T01:16:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:17 smithi181 ceph-mon[134520]: pgmap v367: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 974 op/s 2023-10-17T01:16:19.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:19 smithi060 ceph-mon[134136]: pgmap v368: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 28 MiB/s wr, 1.20k op/s 2023-10-17T01:16:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:19 smithi181 ceph-mon[134520]: pgmap v368: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 28 MiB/s wr, 1.20k op/s 2023-10-17T01:16:20.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:19 smithi045 ceph-mon[98474]: pgmap v368: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 28 MiB/s wr, 1.20k op/s 2023-10-17T01:16:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 70.0 s, 2255.4 tps, lat 44.452 ms stddev 79.890 2023-10-17T01:16:21.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:21 smithi060 ceph-mon[134136]: pgmap v369: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 1.68k op/s 2023-10-17T01:16:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:21 smithi181 ceph-mon[134520]: pgmap v369: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 1.68k op/s 2023-10-17T01:16:22.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:21 smithi045 ceph-mon[98474]: pgmap v369: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 1.68k op/s 2023-10-17T01:16:23.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:23 smithi060 ceph-mon[134136]: pgmap v370: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 1.95k op/s 2023-10-17T01:16:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:23 smithi181 ceph-mon[134520]: pgmap v370: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 1.95k op/s 2023-10-17T01:16:24.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:23 smithi045 ceph-mon[98474]: pgmap v370: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 1.95k op/s 2023-10-17T01:16:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.474+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3720008782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 msgr2=0x7f08bc10a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.474+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3720008782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc10a140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08b8008030 tx=0x7f08b8008340 comp rx=0 tx=0).stop 2023-10-17T01:16:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3720008782 shutdown_connections 2023-10-17T01:16:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3720008782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08bc10d230 0x7f08bc10f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3720008782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc10a140 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3720008782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc10a710 0x7f08bc10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3720008782 >> 172.21.15.45:0/3720008782 conn(0x7f08bc06dac0 msgr2=0x7f08bc06ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3720008782 shutdown_connections 2023-10-17T01:16:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.475+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3720008782 wait complete. 2023-10-17T01:16:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.476+0000 7f08c2f9d700 1 Processor -- start 2023-10-17T01:16:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.476+0000 7f08c2f9d700 1 -- start start 2023-10-17T01:16:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.477+0000 7f08c2f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.478+0000 7f08c2f9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08bc10a710 0x7f08bc19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.478+0000 7f08c1f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.478+0000 7f08c179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08bc10a710 0x7f08bc19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.478+0000 7f08c2f9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc10d230 0x7f08bc1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.478+0000 7f08c1f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42662/0 (socket says 172.21.15.45:42662) 2023-10-17T01:16:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c2f9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1a4ba0 con 0x7f08bc10d230 2023-10-17T01:16:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c2f9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1a4ce0 con 0x7f08bc10a710 2023-10-17T01:16:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c2f9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1a4e70 con 0x7f08bc107d50 2023-10-17T01:16:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c1f9b700 1 -- 172.21.15.45:0/3678615033 learned_addr learned my addr 172.21.15.45:0/3678615033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c279c700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc10d230 0x7f08bc1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c1f9b700 1 -- 172.21.15.45:0/3678615033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08bc10a710 msgr2=0x7f08bc19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08bc10a710 0x7f08bc19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c1f9b700 1 -- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc10d230 msgr2=0x7f08bc1a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc10d230 0x7f08bc1a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.479+0000 7f08c1f9b700 1 -- 172.21.15.45:0/3678615033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8007ce0 con 0x7f08bc107d50 2023-10-17T01:16:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.480+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc19d020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08b8008000 tx=0x7f08b8013750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.480+0000 7f08b2ffd700 1 -- 172.21.15.45:0/3678615033 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800f040 con 0x7f08bc107d50 2023-10-17T01:16:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.480+0000 7f08b2ffd700 1 -- 172.21.15.45:0/3678615033 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b8011d90 con 0x7f08bc107d50 2023-10-17T01:16:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.480+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc1a50f0 con 0x7f08bc107d50 2023-10-17T01:16:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.480+0000 7f08b2ffd700 1 -- 172.21.15.45:0/3678615033 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800f040 con 0x7f08bc107d50 2023-10-17T01:16:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.480+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc1a5570 con 0x7f08bc107d50 2023-10-17T01:16:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.487+0000 7f08b2ffd700 1 -- 172.21.15.45:0/3678615033 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f08b8005880 con 0x7f08bc107d50 2023-10-17T01:16:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.487+0000 7f08d7b30b80 1 Processor -- start 2023-10-17T01:16:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.488+0000 7f08d7b30b80 1 -- start start 2023-10-17T01:16:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.488+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eea76810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.488+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.488+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea77b70 0x5603eea7e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.489+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ee9c59b0 con 0x5603eea771b0 2023-10-17T01:16:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.489+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ee9c3370 con 0x5603eea77b70 2023-10-17T01:16:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.489+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ee9bec60 con 0x5603eea763f0 2023-10-17T01:16:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.489+0000 7f08c279c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eea76810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.489+0000 7f08c279c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eea76810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42672/0 (socket says 172.21.15.45:42672) 2023-10-17T01:16:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.489+0000 7f08c279c700 1 -- 172.21.15.45:0/1432771768 learned_addr learned my addr 172.21.15.45:0/1432771768 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.490+0000 7f08c179a700 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea77b70 0x5603eea7e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.490+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.490+0000 7f08c279c700 1 -- 172.21.15.45:0/1432771768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea77b70 msgr2=0x5603eea7e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.490+0000 7f08c279c700 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea77b70 0x5603eea7e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.490+0000 7f08c279c700 1 -- 172.21.15.45:0/1432771768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 msgr2=0x5603eea77630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08c279c700 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea77630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08c279c700 1 -- 172.21.15.45:0/1432771768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603ee7ecd80 con 0x5603eea763f0 2023-10-17T01:16:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:16:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08c279c700 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eea76810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009a40 tx=0x7f08b400d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a27ef607f452cb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08a7fff700 1 -- 172.21.15.45:0/1432771768 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b400b8f0 con 0x5603eea763f0 2023-10-17T01:16:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08a7fff700 1 -- 172.21.15.45:0/1432771768 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b4015420 con 0x5603eea763f0 2023-10-17T01:16:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08a7fff700 1 -- 172.21.15.45:0/1432771768 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40145e0 con 0x5603eea763f0 2023-10-17T01:16:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.491+0000 7f08b2ffd700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f08a80a0c80 0x7f08a80a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.492+0000 7f08b2ffd700 1 -- 172.21.15.45:0/3678615033 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f08b80cc930 con 0x7f08bc107d50 2023-10-17T01:16:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.492+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f08a80a0c80 0x7f08a80a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.492+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1432771768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 msgr2=0x5603eea76810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.492+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eea76810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009a40 tx=0x7f08b400d040 comp rx=0 tx=0).stop 2023-10-17T01:16:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.493+0000 7f08c1f9b700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f08a80a0c80 0x7f08a80a3140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a1a90 tx=0x7f08b8005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.493+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1432771768 shutdown_connections 2023-10-17T01:16:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.493+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea77b70 0x5603eea7e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.493+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eea76810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.493+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1432771768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.493+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1432771768 >> 172.21.15.45:0/1432771768 conn(0x5603ee9cd070 msgr2=0x5603eea68660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.494+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1432771768 shutdown_connections 2023-10-17T01:16:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.494+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1432771768 wait complete. 2023-10-17T01:16:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.496+0000 7f08d7b30b80 1 -- start start 2023-10-17T01:16:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.496+0000 7f08d7b30b80 1 Processor -- start 2023-10-17T01:16:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea763f0 0x5603eea6ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea6f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea74390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ee95b4b0 con 0x5603eea771b0 2023-10-17T01:16:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ee95b150 con 0x5603eea763f0 2023-10-17T01:16:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ee8428c0 con 0x5603eea77b70 2023-10-17T01:16:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.498+0000 7f08d7b30b80 10 client.0 _ll_register_callbacks cb 0x5603ee8a8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.499+0000 7f08c279c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea763f0 0x5603eea6ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.499+0000 7f08c279c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea763f0 0x5603eea6ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:44980/0 (socket says 172.21.15.45:44980) 2023-10-17T01:16:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.499+0000 7f08c279c700 1 -- 172.21.15.45:0/3893630291 learned_addr learned my addr 172.21.15.45:0/3893630291 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.499+0000 7f08c279c700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea74390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.499+0000 7f08c179a700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea6f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.500+0000 7f08c179a700 1 -- 172.21.15.45:0/3893630291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 msgr2=0x5603eea74390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.500+0000 7f08c179a700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea74390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.500+0000 7f08c179a700 1 -- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea763f0 msgr2=0x5603eea6ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.500+0000 7f08c179a700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea763f0 0x5603eea6ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.500+0000 7f08c179a700 1 -- 172.21.15.45:0/3893630291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b4009670 con 0x5603eea771b0 2023-10-17T01:16:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.500+0000 7f08c179a700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea6f360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b660 tx=0x7f08ac00b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56ca6430aa0760 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.501+0000 7f08a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:16:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.501+0000 7f08a57fa700 1 -- 172.21.15.45:0/3893630291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac01a070 con 0x5603eea771b0 2023-10-17T01:16:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.501+0000 7f08a57fa700 1 -- 172.21.15.45:0/3893630291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f08ac004830 con 0x5603eea771b0 2023-10-17T01:16:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.501+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603ee9e9cb0 con 0x5603eea771b0 2023-10-17T01:16:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.502+0000 7f08a57fa700 1 -- 172.21.15.45:0/3893630291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08ac0049f0 con 0x5603eea771b0 2023-10-17T01:16:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.502+0000 7f08d7b30b80 10 client.15537 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.502+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5603ee7d5c00 con 0x5603eea771b0 2023-10-17T01:16:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.502+0000 7f08a57fa700 1 -- 172.21.15.45:0/3893630291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08ac025020 con 0x5603eea771b0 2023-10-17T01:16:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08a57fa700 1 -- 172.21.15.45:0/3893630291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f08ac01d020 con 0x5603eea771b0 2023-10-17T01:16:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08d7b30b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08d7b30b80 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08d7b30b80 20 client.15537 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eea944b0 0x5603eea968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08d7b30b80 4 client.15537 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:16:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603eea3ccd0 con 0x5603eea944b0 2023-10-17T01:16:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.503+0000 7f08c279c700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eea944b0 0x5603eea968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.504+0000 7f08c279c700 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eea944b0 0x5603eea968d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b894752fd2e43a66 server_cookie=5668a5e18723b6ea in_seq=0 out_seq=0 2023-10-17T01:16:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.504+0000 7f08a57fa700 10 client.15537 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.505+0000 7f08a57fa700 1 -- 172.21.15.45:0/3893630291 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x7f08b402c4f0 con 0x5603eea944b0 2023-10-17T01:16:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.505+0000 7f08a57fa700 10 client.15537 handle_command_reply: tid=0 2023-10-17T01:16:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.505+0000 7f08d7b30b80 1 client.15537 shutdown 2023-10-17T01:16:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 msgr2=0x5603eea6f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea6f360 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b660 tx=0x7f08ac00b970 comp rx=0 tx=0).stop 2023-10-17T01:16:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 shutdown_connections 2023-10-17T01:16:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea74390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eea944b0 0x5603eea968d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea763f0 0x5603eea6ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3893630291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea771b0 0x5603eea6f360 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 >> 172.21.15.45:0/3893630291 conn(0x5603ee9cd070 msgr2=0x5603eea67db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08a57fa700 0 client.15537 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 shutdown_connections 2023-10-17T01:16:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.506+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3893630291 wait complete. 2023-10-17T01:16:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.507+0000 7f08d7b30b80 20 client.15537 trim_cache size 0 max 16384 2023-10-17T01:16:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 Processor -- start 2023-10-17T01:16:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 -- start start 2023-10-17T01:16:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 0x5603eea74350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eea6da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea7ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603eea60480 con 0x5603eea763f0 2023-10-17T01:16:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603eea76ca0 con 0x5603eea771b0 2023-10-17T01:16:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.532+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603eea75f70 con 0x5603eea77b70 2023-10-17T01:16:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.533+0000 7f08c279c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 0x5603eea74350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.533+0000 7f08c179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eea6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.533+0000 7f08c279c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 0x5603eea74350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52620/0 (socket says 172.21.15.45:52620) 2023-10-17T01:16:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.533+0000 7f08c279c700 1 -- 172.21.15.45:0/3113230306 learned_addr learned my addr 172.21.15.45:0/3113230306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.533+0000 7f08c279c700 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea7ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.534+0000 7f08c179a700 1 -- 172.21.15.45:0/3113230306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 msgr2=0x5603eea7ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.534+0000 7f08c179a700 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea7ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.534+0000 7f08c179a700 1 -- 172.21.15.45:0/3113230306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 msgr2=0x5603eea74350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.534+0000 7f08c179a700 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 0x5603eea74350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.534+0000 7f08c179a700 1 -- 172.21.15.45:0/3113230306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603ee8131f0 con 0x5603eea771b0 2023-10-17T01:16:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.534+0000 7f08c279c700 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 0x5603eea74350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.535+0000 7f08c179a700 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eea6da70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b630 tx=0x7f08ac005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7319b65a6dc0c3f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.536+0000 7f08b0ff9700 1 -- 172.21.15.45:0/3113230306 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac01a070 con 0x5603eea771b0 2023-10-17T01:16:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.536+0000 7f08b0ff9700 1 -- 172.21.15.45:0/3113230306 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08ac025030 con 0x5603eea771b0 2023-10-17T01:16:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.536+0000 7f08b0ff9700 1 -- 172.21.15.45:0/3113230306 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac021670 con 0x5603eea771b0 2023-10-17T01:16:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.536+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3113230306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 msgr2=0x5603eea6da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eea6da70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b630 tx=0x7f08ac005f60 comp rx=0 tx=0).stop 2023-10-17T01:16:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3113230306 shutdown_connections 2023-10-17T01:16:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea77b70 0x5603eea7ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea763f0 0x5603eea74350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/3113230306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eea6da70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3113230306 >> 172.21.15.45:0/3113230306 conn(0x5603ee9cd070 msgr2=0x5603eea964c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3113230306 shutdown_connections 2023-10-17T01:16:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 -- 172.21.15.45:0/3113230306 wait complete. 2023-10-17T01:16:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.537+0000 7f08d7b30b80 1 -- start start 2023-10-17T01:16:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.538+0000 7f08d7b30b80 1 Processor -- start 2023-10-17T01:16:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eeadfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eeae2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea77b70 0x5603eeae4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603eeae5090 con 0x5603eea77b70 2023-10-17T01:16:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603eeae5200 con 0x5603eea771b0 2023-10-17T01:16:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603eeae53c0 con 0x5603eea763f0 2023-10-17T01:16:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.539+0000 7f08d7b30b80 10 client.0 _ll_register_callbacks cb 0x7f08ac00ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.540+0000 7f08c179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eeae2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.540+0000 7f08c279c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eeadfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.540+0000 7f08c179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eeae2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:44998/0 (socket says 172.21.15.45:44998) 2023-10-17T01:16:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.540+0000 7f08c179a700 1 -- 172.21.15.45:0/1882094759 learned_addr learned my addr 172.21.15.45:0/1882094759 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.540+0000 7f08c279c700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea77b70 0x5603eeae4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c279c700 1 -- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 msgr2=0x5603eeae2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c279c700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eeae2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c279c700 1 -- 172.21.15.45:0/1882094759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea77b70 msgr2=0x5603eeae4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c279c700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea77b70 0x5603eeae4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c279c700 1 -- 172.21.15.45:0/1882094759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603ee7ecd80 con 0x5603eea763f0 2023-10-17T01:16:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c179a700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eeae2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08c279c700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eeadfe10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009a40 tx=0x7f08b4005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a97aa2f6bef35d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.541+0000 7f08a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:16:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08a57fa700 1 -- 172.21.15.45:0/1882094759 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4004020 con 0x5603eea763f0 2023-10-17T01:16:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08b4015420 con 0x5603eea763f0 2023-10-17T01:16:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08a57fa700 1 -- 172.21.15.45:0/1882094759 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f08b4004180 con 0x5603eea763f0 2023-10-17T01:16:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08a57fa700 1 -- 172.21.15.45:0/1882094759 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08b4004340 con 0x5603eea763f0 2023-10-17T01:16:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08d7b30b80 10 client.25645 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5603ee7d5c00 con 0x5603eea763f0 2023-10-17T01:16:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08a57fa700 1 -- 172.21.15.45:0/1882094759 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b4015420 con 0x5603eea763f0 2023-10-17T01:16:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.542+0000 7f08a57fa700 1 -- 172.21.15.45:0/1882094759 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f08b40039b0 con 0x5603eea763f0 2023-10-17T01:16:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08d7b30b80 10 client.25645 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08d7b30b80 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08d7b30b80 20 client.25645 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eeae6de0 0x5603eeae9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08d7b30b80 4 client.25645 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:16:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603ee9b7ce0 con 0x5603eeae6de0 2023-10-17T01:16:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.543+0000 7f08c179a700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eeae6de0 0x5603eeae9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.544+0000 7f08a57fa700 10 client.25645 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.544+0000 7f08c179a700 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eeae6de0 0x5603eeae9200 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884070c55c57bf3f server_cookie=4c4e8616bd519adf in_seq=0 out_seq=0 2023-10-17T01:16:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.598+0000 7f08a57fa700 1 -- 172.21.15.45:0/1882094759 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603ee9b7ce0 con 0x5603eeae6de0 2023-10-17T01:16:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.598+0000 7f08a57fa700 10 client.25645 handle_command_reply: tid=0 2023-10-17T01:16:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.598+0000 7f08d7b30b80 1 client.25645 shutdown 2023-10-17T01:16:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 msgr2=0x5603eeadfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eeadfe10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009a40 tx=0x7f08b4005ee0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 shutdown_connections 2023-10-17T01:16:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603eea77b70 0x5603eeae4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603eea763f0 0x5603eeadfe10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5603eeae6de0 0x5603eeae9200 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 --2- 172.21.15.45:0/1882094759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5603eea771b0 0x5603eeae2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 >> 172.21.15.45:0/1882094759 conn(0x5603ee9cd070 msgr2=0x5603eea6da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08a57fa700 0 client.25645 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 shutdown_connections 2023-10-17T01:16:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.600+0000 7f08d7b30b80 1 -- 172.21.15.45:0/1882094759 wait complete. 2023-10-17T01:16:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.601+0000 7f08d7b30b80 20 client.25645 trim_cache size 0 max 16384 2023-10-17T01:16:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:16:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:16:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:16:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:16:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f08a80a0c80 msgr2=0x7f08a80a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f08a80a0c80 0x7f08a80a3140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a1a90 tx=0x7f08b8005040 comp rx=0 tx=0).stop 2023-10-17T01:16:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 msgr2=0x7f08bc19d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc19d020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08b8008000 tx=0x7f08b8013750 comp rx=0 tx=0).stop 2023-10-17T01:16:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 shutdown_connections 2023-10-17T01:16:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f08a80a0c80 0x7f08a80a3140 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc10d230 0x7f08bc1a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc107d50 0x7f08bc19d020 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08b8008000 tx=0x7f08b8013750 comp rx=0 tx=0).stop 2023-10-17T01:16:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 --2- 172.21.15.45:0/3678615033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08bc10a710 0x7f08bc19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.602+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 >> 172.21.15.45:0/3678615033 conn(0x7f08bc06dac0 msgr2=0x7f08bc1096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.603+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 shutdown_connections 2023-10-17T01:16:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:24.603+0000 7f08c2f9d700 1 -- 172.21.15.45:0/3678615033 wait complete. 2023-10-17T01:16:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 75.0 s, 3202.8 tps, lat 31.183 ms stddev 34.105 2023-10-17T01:16:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:25 smithi045 ceph-mon[98474]: pgmap v371: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 MiB/s wr, 2.14k op/s 2023-10-17T01:16:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:25 smithi060 ceph-mon[134136]: pgmap v371: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 MiB/s wr, 2.14k op/s 2023-10-17T01:16:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:25 smithi181 ceph-mon[134520]: pgmap v371: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 46 MiB/s wr, 2.14k op/s 2023-10-17T01:16:27.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:16:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:27 smithi045 ceph-mon[98474]: pgmap v372: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 MiB/s wr, 1.94k op/s 2023-10-17T01:16:27.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:27 smithi060 ceph-mon[134136]: pgmap v372: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 MiB/s wr, 1.94k op/s 2023-10-17T01:16:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:27 smithi181 ceph-mon[134520]: pgmap v372: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 MiB/s wr, 1.94k op/s 2023-10-17T01:16:27.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.962+0000 7f73e3dff700 1 -- 172.21.15.45:0/1123441439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc101f60 msgr2=0x7f73dc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:27.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.962+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1123441439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc101f60 0x7f73dc1023c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009990 tx=0x7f73d8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 -- 172.21.15.45:0/1123441439 shutdown_connections 2023-10-17T01:16:27.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1123441439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc102900 0x7f73dc1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1123441439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73dc100d70 0x7f73dc101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1123441439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc101f60 0x7f73dc1023c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 -- 172.21.15.45:0/1123441439 >> 172.21.15.45:0/1123441439 conn(0x7f73dc0fc300 msgr2=0x7f73dc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:27.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 -- 172.21.15.45:0/1123441439 shutdown_connections 2023-10-17T01:16:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.963+0000 7f73e3dff700 1 -- 172.21.15.45:0/1123441439 wait complete. 2023-10-17T01:16:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.965+0000 7f73e3dff700 1 Processor -- start 2023-10-17T01:16:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.966+0000 7f73e3dff700 1 -- start start 2023-10-17T01:16:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.967+0000 7f73e3dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 0x7f73dc06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.967+0000 7f73e3dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73dc101f60 0x7f73dc06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.967+0000 7f73e3dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc102900 0x7f73dc1a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.967+0000 7f73e1b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 0x7f73dc06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.967+0000 7f73e1b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 0x7f73dc06f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46366/0 (socket says 172.21.15.45:46366) 2023-10-17T01:16:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e1b9b700 1 -- 172.21.15.45:0/1465440420 learned_addr learned my addr 172.21.15.45:0/1465440420 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc072be0 con 0x7f73dc101f60 2023-10-17T01:16:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc072d50 con 0x7f73dc100d70 2023-10-17T01:16:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc1a7190 con 0x7f73dc102900 2023-10-17T01:16:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e239c700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc102900 0x7f73dc1a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e139a700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73dc101f60 0x7f73dc06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e1b9b700 1 -- 172.21.15.45:0/1465440420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc102900 msgr2=0x7f73dc1a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc102900 0x7f73dc1a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e1b9b700 1 -- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73dc101f60 msgr2=0x7f73dc06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73dc101f60 0x7f73dc06fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.968+0000 7f73e1b9b700 1 -- 172.21.15.45:0/1465440420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80095f0 con 0x7f73dc100d70 2023-10-17T01:16:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.969+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 0x7f73dc06f5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d980 tx=0x7f73cc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.969+0000 7f73d2ffd700 1 -- 172.21.15.45:0/1465440420 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004020 con 0x7f73dc100d70 2023-10-17T01:16:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.969+0000 7f73d2ffd700 1 -- 172.21.15.45:0/1465440420 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73cc0096d0 con 0x7f73dc100d70 2023-10-17T01:16:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.969+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc072fa0 con 0x7f73dc100d70 2023-10-17T01:16:27.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.970+0000 7f73d2ffd700 1 -- 172.21.15.45:0/1465440420 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0039c0 con 0x7f73dc100d70 2023-10-17T01:16:27.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.970+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc073500 con 0x7f73dc100d70 2023-10-17T01:16:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.977+0000 7f73d2ffd700 1 -- 172.21.15.45:0/1465440420 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f73cc009b40 con 0x7f73dc100d70 2023-10-17T01:16:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.980+0000 7f73d2ffd700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f73c80a0c20 0x7f73c80a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.980+0000 7f73d2ffd700 1 -- 172.21.15.45:0/1465440420 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73cc0cbcf0 con 0x7f73dc100d70 2023-10-17T01:16:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.980+0000 7f73e139a700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f73c80a0c20 0x7f73c80a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.980+0000 7f73f77b7b80 1 Processor -- start 2023-10-17T01:16:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.981+0000 7f73f77b7b80 1 -- start start 2023-10-17T01:16:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.981+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 0x55d38385cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.981+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385d500 0x55d38385dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.981+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38385e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3837b4a10 con 0x55d38385e2b0 2023-10-17T01:16:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3837b23d0 con 0x55d38385cba0 2023-10-17T01:16:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3837adcc0 con 0x55d38385d500 2023-10-17T01:16:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e1b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 0x55d38385cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e1b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 0x55d38385cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46382/0 (socket says 172.21.15.45:46382) 2023-10-17T01:16:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e1b9b700 1 -- 172.21.15.45:0/3600810355 learned_addr learned my addr 172.21.15.45:0/3600810355 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e139a700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f73c80a0c20 0x7f73c80a30e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f73d8000c00 tx=0x7f73d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e239c700 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38385e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e1b9b700 1 -- 172.21.15.45:0/3600810355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385d500 msgr2=0x55d38385dd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:16:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385d500 0x55d38385dd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.982+0000 7f73e1b9b700 1 -- 172.21.15.45:0/3600810355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 msgr2=0x55d38385e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:27.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.983+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38385e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.983+0000 7f73e1b9b700 1 -- 172.21.15.45:0/3600810355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3835dbde0 con 0x55d38385cba0 2023-10-17T01:16:27.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.983+0000 7f73e239c700 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38385e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:16:27.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.983+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 0x55d38385cfc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0992a0 tx=0x7f73cc098c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e3e6b7bef3f0f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:27.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.983+0000 7f73c7fff700 1 -- 172.21.15.45:0/3600810355 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc099310 con 0x55d38385cba0 2023-10-17T01:16:27.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.984+0000 7f73c7fff700 1 -- 172.21.15.45:0/3600810355 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73cc09d140 con 0x55d38385cba0 2023-10-17T01:16:27.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.984+0000 7f73c7fff700 1 -- 172.21.15.45:0/3600810355 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc098800 con 0x55d38385cba0 2023-10-17T01:16:27.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 -- 172.21.15.45:0/3600810355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 msgr2=0x55d38385cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:27.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 0x55d38385cfc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0992a0 tx=0x7f73cc098c70 comp rx=0 tx=0).stop 2023-10-17T01:16:27.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 -- 172.21.15.45:0/3600810355 shutdown_connections 2023-10-17T01:16:27.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38385e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385cba0 0x55d38385cfc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/3600810355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385d500 0x55d38385dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 -- 172.21.15.45:0/3600810355 >> 172.21.15.45:0/3600810355 conn(0x55d3837bc0d0 msgr2=0x55d383857950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:27.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 -- 172.21.15.45:0/3600810355 shutdown_connections 2023-10-17T01:16:27.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.985+0000 7f73f77b7b80 1 -- 172.21.15.45:0/3600810355 wait complete. 2023-10-17T01:16:27.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.986+0000 7f73f77b7b80 1 -- start start 2023-10-17T01:16:27.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.986+0000 7f73f77b7b80 1 Processor -- start 2023-10-17T01:16:27.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 0x55d383866990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 0x55d383866ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38386df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38374a510 con 0x55d38385e2b0 2023-10-17T01:16:27.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38374a1b0 con 0x55d38385d500 2023-10-17T01:16:27.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d383631920 con 0x55d38385cba0 2023-10-17T01:16:27.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.987+0000 7f73f77b7b80 10 client.0 _ll_register_callbacks cb 0x55d383697a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:27.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.988+0000 7f73e1b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 0x55d383866990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.988+0000 7f73e1b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 0x55d383866990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45824/0 (socket says 172.21.15.45:45824) 2023-10-17T01:16:27.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.988+0000 7f73e1b9b700 1 -- 172.21.15.45:0/255351154 learned_addr learned my addr 172.21.15.45:0/255351154 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:27.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.988+0000 7f73e239c700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38386df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e1b9b700 1 -- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 msgr2=0x55d383866ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:27.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 0x55d383866ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e1b9b700 1 -- 172.21.15.45:0/255351154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 msgr2=0x55d38386df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:27.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38386df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:27.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e1b9b700 1 -- 172.21.15.45:0/255351154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc073500 con 0x55d38385cba0 2023-10-17T01:16:27.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e239c700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38386df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:16:27.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 0x55d383866990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0a0e50 tx=0x7f73cc099930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd8203e18f95bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:27.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:16:27.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73c57fa700 1 -- 172.21.15.45:0/255351154 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc099310 con 0x55d38385cba0 2023-10-17T01:16:27.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.989+0000 7f73c57fa700 1 -- 172.21.15.45:0/255351154 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73cc098a10 con 0x55d38385cba0 2023-10-17T01:16:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.990+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3837d8d10 con 0x55d38385cba0 2023-10-17T01:16:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.990+0000 7f73c57fa700 1 -- 172.21.15.45:0/255351154 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73cc0a0ef0 con 0x55d38385cba0 2023-10-17T01:16:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.990+0000 7f73f77b7b80 10 client.25654 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.990+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d3835c4c60 con 0x55d38385cba0 2023-10-17T01:16:27.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.990+0000 7f73c57fa700 1 -- 172.21.15.45:0/255351154 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3837d8d10 con 0x55d38385cba0 2023-10-17T01:16:27.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73c57fa700 1 -- 172.21.15.45:0/255351154 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f73cc003b20 con 0x55d38385cba0 2023-10-17T01:16:27.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73f77b7b80 10 client.25654 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:27.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73f77b7b80 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:27.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73f77b7b80 20 client.25654 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:27.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838723b0 0x55d3838747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:27.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73f77b7b80 4 client.25654 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:16:27.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.991+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d38382bd30 con 0x55d3838723b0 2023-10-17T01:16:27.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.993+0000 7f73e139a700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838723b0 0x55d3838747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:27.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.995+0000 7f73e139a700 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838723b0 0x55d3838747d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8fceda4c6df870 server_cookie=35d404a48daa3f90 in_seq=0 out_seq=0 2023-10-17T01:16:27.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.995+0000 7f73c57fa700 10 client.25654 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:28.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.996+0000 7f73c57fa700 1 -- 172.21.15.45:0/255351154 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d38382bd30 con 0x55d3838723b0 2023-10-17T01:16:28.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.996+0000 7f73c57fa700 10 client.25654 handle_command_reply: tid=0 2023-10-17T01:16:28.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.996+0000 7f73f77b7b80 1 client.25654 shutdown 2023-10-17T01:16:28.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.997+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 msgr2=0x55d383866990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:28.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.997+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 0x55d383866990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0a0e50 tx=0x7f73cc099930 comp rx=0 tx=0).stop 2023-10-17T01:16:28.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 shutdown_connections 2023-10-17T01:16:28.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385e2b0 0x55d38386df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385cba0 0x55d383866990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838723b0 0x55d3838747d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/255351154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 0x55d383866ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 >> 172.21.15.45:0/255351154 conn(0x55d3837bc0d0 msgr2=0x55d383857020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:28.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73c57fa700 0 client.25654 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:28.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 shutdown_connections 2023-10-17T01:16:28.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.998+0000 7f73f77b7b80 1 -- 172.21.15.45:0/255351154 wait complete. 2023-10-17T01:16:28.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:27.999+0000 7f73f77b7b80 20 client.25654 trim_cache size 0 max 16384 2023-10-17T01:16:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.035+0000 7f73f77b7b80 1 Processor -- start 2023-10-17T01:16:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.035+0000 7f73f77b7b80 1 -- start start 2023-10-17T01:16:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.036+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385d500 0x55d383857020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.036+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d383865590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.036+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38386bec0 0x55d3838743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73e139a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d383865590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d383866560 con 0x55d38385d500 2023-10-17T01:16:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73e139a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d383865590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45828/0 (socket says 172.21.15.45:45828) 2023-10-17T01:16:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73e139a700 1 -- 172.21.15.45:0/361151693 learned_addr learned my addr 172.21.15.45:0/361151693 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38386ef10 con 0x55d38386bec0 2023-10-17T01:16:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38382bd30 con 0x55d38385e2b0 2023-10-17T01:16:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73e239c700 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38386bec0 0x55d3838743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.037+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385d500 0x55d383857020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.038+0000 7f73e1b9b700 1 -- 172.21.15.45:0/361151693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 msgr2=0x55d383865590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.038+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d383865590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.038+0000 7f73e1b9b700 1 -- 172.21.15.45:0/361151693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38386bec0 msgr2=0x55d3838743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.038+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38386bec0 0x55d3838743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.038+0000 7f73e1b9b700 1 -- 172.21.15.45:0/361151693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d383602250 con 0x55d38385d500 2023-10-17T01:16:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.039+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385d500 0x55d383857020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f73cc005e10 tx=0x7f73cc084050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cd6ea977047b9a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.040+0000 7f73d0bff700 1 -- 172.21.15.45:0/361151693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0a0ef0 con 0x55d38385d500 2023-10-17T01:16:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.040+0000 7f73d0bff700 1 -- 172.21.15.45:0/361151693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73cc0a1400 con 0x55d38385d500 2023-10-17T01:16:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.040+0000 7f73d0bff700 1 -- 172.21.15.45:0/361151693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc082b60 con 0x55d38385d500 2023-10-17T01:16:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385d500 msgr2=0x55d383857020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385d500 0x55d383857020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f73cc005e10 tx=0x7f73cc084050 comp rx=0 tx=0).stop 2023-10-17T01:16:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 shutdown_connections 2023-10-17T01:16:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38386bec0 0x55d3838743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38385d500 0x55d383857020 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/361151693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d383865590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.042+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 >> 172.21.15.45:0/361151693 conn(0x55d3837bc0d0 msgr2=0x55d38386e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.043+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 shutdown_connections 2023-10-17T01:16:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.043+0000 7f73f77b7b80 1 -- 172.21.15.45:0/361151693 wait complete. 2023-10-17T01:16:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.044+0000 7f73f77b7b80 1 -- start start 2023-10-17T01:16:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.044+0000 7f73f77b7b80 1 Processor -- start 2023-10-17T01:16:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 0x55d3838bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d3838bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 0x55d3838c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3838c37e0 con 0x55d38386bec0 2023-10-17T01:16:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3838c3920 con 0x55d38385d500 2023-10-17T01:16:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3838c3ae0 con 0x55d38385e2b0 2023-10-17T01:16:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.046+0000 7f73f77b7b80 10 client.0 _ll_register_callbacks cb 0x7f73cc098e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.047+0000 7f73e239c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 0x55d3838c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.047+0000 7f73e139a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d3838bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.047+0000 7f73e239c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 0x55d3838c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49992/0 (socket says 172.21.15.45:49992) 2023-10-17T01:16:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.047+0000 7f73e239c700 1 -- 172.21.15.45:0/131661099 learned_addr learned my addr 172.21.15.45:0/131661099 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.047+0000 7f73e239c700 1 -- 172.21.15.45:0/131661099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 msgr2=0x55d3838bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.047+0000 7f73e239c700 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d3838bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.048+0000 7f73e239c700 1 -- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 msgr2=0x55d3838bbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.048+0000 7f73e239c700 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 0x55d3838bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.048+0000 7f73e239c700 1 -- 172.21.15.45:0/131661099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3835c4c60 con 0x55d38386bec0 2023-10-17T01:16:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.048+0000 7f73e239c700 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 0x55d3838c32a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73d4007bf0 tx=0x7f73d4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f40cb66b0d2efac server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.048+0000 7f73c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:16:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.049+0000 7f73c57fa700 1 -- 172.21.15.45:0/131661099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d401e070 con 0x55d38386bec0 2023-10-17T01:16:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.049+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73cc09d140 con 0x55d38386bec0 2023-10-17T01:16:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.049+0000 7f73c57fa700 1 -- 172.21.15.45:0/131661099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73d4011a80 con 0x55d38386bec0 2023-10-17T01:16:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.050+0000 7f73c57fa700 1 -- 172.21.15.45:0/131661099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73d400e030 con 0x55d38386bec0 2023-10-17T01:16:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.050+0000 7f73f77b7b80 10 client.15561 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.050+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d3835da2b0 con 0x55d38386bec0 2023-10-17T01:16:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.051+0000 7f73c57fa700 1 -- 172.21.15.45:0/131661099 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73d4029020 con 0x55d38386bec0 2023-10-17T01:16:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73c57fa700 1 -- 172.21.15.45:0/131661099 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f73d4021020 con 0x55d38386bec0 2023-10-17T01:16:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73f77b7b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73f77b7b80 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73f77b7b80 20 client.15561 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838c56b0 0x55d3838c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73f77b7b80 4 client.15561 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:16:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.053+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3837a6d40 con 0x55d3838c56b0 2023-10-17T01:16:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.057+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838c56b0 0x55d3838c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.059+0000 7f73c57fa700 10 client.15561 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.059+0000 7f73e1b9b700 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838c56b0 0x55d3838c7ad0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca66f17d2f6ba29 server_cookie=9b57b553c926d12b in_seq=0 out_seq=0 2023-10-17T01:16:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.059+0000 7f73c57fa700 1 -- 172.21.15.45:0/131661099 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d3837a6d40 con 0x55d3838c56b0 2023-10-17T01:16:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.059+0000 7f73c57fa700 10 client.15561 handle_command_reply: tid=0 2023-10-17T01:16:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.060+0000 7f73f77b7b80 1 client.15561 shutdown 2023-10-17T01:16:28.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.060+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 msgr2=0x55d3838c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:28.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.060+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 0x55d3838c32a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73d4007bf0 tx=0x7f73d4009920 comp rx=0 tx=0).stop 2023-10-17T01:16:28.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.061+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 shutdown_connections 2023-10-17T01:16:28.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.061+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d38386bec0 0x55d3838c32a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.061+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d3838c56b0 0x55d3838c7ad0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.061+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d38385d500 0x55d3838bbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.061+0000 7f73f77b7b80 1 --2- 172.21.15.45:0/131661099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d38385e2b0 0x55d3838bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.061+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 >> 172.21.15.45:0/131661099 conn(0x55d3837bc0d0 msgr2=0x55d383857020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.062+0000 7f73c57fa700 0 client.15561 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.063+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 shutdown_connections 2023-10-17T01:16:28.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.063+0000 7f73f77b7b80 1 -- 172.21.15.45:0/131661099 wait complete. 2023-10-17T01:16:28.066 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:16:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:16:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:16:28.066 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:16:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:16:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:16:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:16:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:16:28.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:16:28.068 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:16:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.064+0000 7f73f77b7b80 20 client.15561 trim_cache size 0 max 16384 2023-10-17T01:16:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.065+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f73c80a0c20 msgr2=0x7f73c80a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.065+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f73c80a0c20 0x7f73c80a30e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f73d8000c00 tx=0x7f73d8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 msgr2=0x7f73dc06f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 0x7f73dc06f5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d980 tx=0x7f73cc00dc90 comp rx=0 tx=0).stop 2023-10-17T01:16:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 shutdown_connections 2023-10-17T01:16:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f73c80a0c20 0x7f73c80a30e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73dc102900 0x7f73dc1a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73dc100d70 0x7f73dc06f5c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 --2- 172.21.15.45:0/1465440420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73dc101f60 0x7f73dc06fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 >> 172.21.15.45:0/1465440420 conn(0x7f73dc0fc300 msgr2=0x7f73dc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.066+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 shutdown_connections 2023-10-17T01:16:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:28.067+0000 7f73e3dff700 1 -- 172.21.15.45:0/1465440420 wait complete. 2023-10-17T01:16:28.103 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:16:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:29 smithi045 ceph-mon[98474]: pgmap v373: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 1.96k op/s 2023-10-17T01:16:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:29 smithi060 ceph-mon[134136]: pgmap v373: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 1.96k op/s 2023-10-17T01:16:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:29 smithi181 ceph-mon[134520]: pgmap v373: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 MiB/s wr, 1.96k op/s 2023-10-17T01:16:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 80.0 s, 3080.2 tps, lat 32.257 ms stddev 35.486 2023-10-17T01:16:31.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:31 smithi045 ceph-mon[98474]: pgmap v374: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.82k op/s 2023-10-17T01:16:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:31 smithi060 ceph-mon[134136]: pgmap v374: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.82k op/s 2023-10-17T01:16:31.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:31 smithi181 ceph-mon[134520]: pgmap v374: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.82k op/s 2023-10-17T01:16:33.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:33 smithi045 ceph-mon[98474]: pgmap v375: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 1.26k op/s 2023-10-17T01:16:33.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:33 smithi060 ceph-mon[134136]: pgmap v375: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 1.26k op/s 2023-10-17T01:16:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:33 smithi181 ceph-mon[134520]: pgmap v375: 97 pgs: 97 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 1.26k op/s 2023-10-17T01:16:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 85.0 s, 3392.8 tps, lat 29.777 ms stddev 33.173 2023-10-17T01:16:35.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:35 smithi045 ceph-mon[98474]: pgmap v376: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 982 op/s 2023-10-17T01:16:35.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:35 smithi060 ceph-mon[134136]: pgmap v376: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 982 op/s 2023-10-17T01:16:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:35 smithi181 ceph-mon[134520]: pgmap v376: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 982 op/s 2023-10-17T01:16:37.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:37 smithi045 ceph-mon[98474]: pgmap v377: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 672 op/s 2023-10-17T01:16:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:37 smithi060 ceph-mon[134136]: pgmap v377: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 672 op/s 2023-10-17T01:16:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.854+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1352217835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4104900 msgr2=0x7f35b4104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.854+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1352217835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4104900 0x7f35b4104d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b0a0 tx=0x7f35ac00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1352217835 shutdown_connections 2023-10-17T01:16:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1352217835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4104900 0x7f35b4104d80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1352217835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 0x7f35b41019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1352217835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4101f30 0x7f35b41043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1352217835 >> 172.21.15.45:0/1352217835 conn(0x7f35b40fb1a0 msgr2=0x7f35b40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1352217835 shutdown_connections 2023-10-17T01:16:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.855+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1352217835 wait complete. 2023-10-17T01:16:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.857+0000 7f35ba1ff700 1 Processor -- start 2023-10-17T01:16:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.857+0000 7f35ba1ff700 1 -- start start 2023-10-17T01:16:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.858+0000 7f35ba1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 0x7f35b410abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35ba1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 0x7f35b410b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35ba1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4104900 0x7f35b410bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 0x7f35b410b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 0x7f35b410abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35ba1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b410c640 con 0x7f35b4104900 2023-10-17T01:16:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35ba1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b410c7b0 con 0x7f35b40ff5c0 2023-10-17T01:16:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 0x7f35b410abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40988/0 (socket says 172.21.15.45:40988) 2023-10-17T01:16:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 0x7f35b410b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52238/0 (socket says 172.21.15.45:52238) 2023-10-17T01:16:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35ba1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b410c970 con 0x7f35b4101f30 2023-10-17T01:16:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35b37fe700 1 -- 172.21.15.45:0/1248518514 learned_addr learned my addr 172.21.15.45:0/1248518514 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.859+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4104900 0x7f35b410bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.860+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1248518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 msgr2=0x7f35b410abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.860+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 0x7f35b410abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.860+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4104900 msgr2=0x7f35b410bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.860+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4104900 0x7f35b410bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.860+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1248518514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac009d00 con 0x7f35b4101f30 2023-10-17T01:16:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.860+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4104900 0x7f35b410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.861+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 0x7f35b410b110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c9e0 tx=0x7f35a400ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.862+0000 7f35b0ff9700 1 -- 172.21.15.45:0/1248518514 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4004220 con 0x7f35b4101f30 2023-10-17T01:16:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.862+0000 7f35b0ff9700 1 -- 172.21.15.45:0/1248518514 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a400ce80 con 0x7f35b4101f30 2023-10-17T01:16:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.862+0000 7f35b0ff9700 1 -- 172.21.15.45:0/1248518514 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4003cf0 con 0x7f35b4101f30 2023-10-17T01:16:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.862+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b410cc50 con 0x7f35b4101f30 2023-10-17T01:16:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.862+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b410d400 con 0x7f35b4101f30 2023-10-17T01:16:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.868+0000 7f35b0ff9700 1 -- 172.21.15.45:0/1248518514 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f35a4007900 con 0x7f35b4101f30 2023-10-17T01:16:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.871+0000 7f35b0ff9700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35a00a0bf0 0x7f35a00a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.872+0000 7f35b0ff9700 1 -- 172.21.15.45:0/1248518514 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f35a40cbf70 con 0x7f35b4101f30 2023-10-17T01:16:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.872+0000 7f35cdaf3b80 1 Processor -- start 2023-10-17T01:16:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.873+0000 7f35b37fe700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35a00a0bf0 0x7f35a00a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.873+0000 7f35cdaf3b80 1 -- start start 2023-10-17T01:16:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8190 0x561f988e8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 0x561f988e9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9883f9b0 con 0x561f988e8190 2023-10-17T01:16:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9883d370 con 0x561f988e8d90 2023-10-17T01:16:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f98838c60 con 0x561f988e7830 2023-10-17T01:16:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35b3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 0x561f988e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.874+0000 7f35b37fe700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35a00a0bf0 0x7f35a00a30b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f35b410a940 tx=0x7f359c00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 0x561f988e9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41000/0 (socket says 172.21.15.45:41000) 2023-10-17T01:16:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b3fff700 1 -- 172.21.15.45:0/1787521945 learned_addr learned my addr 172.21.15.45:0/1787521945 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8190 0x561f988e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b37fe700 1 -- 172.21.15.45:0/1787521945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 msgr2=0x561f988e9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b37fe700 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 0x561f988e9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b37fe700 1 -- 172.21.15.45:0/1787521945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8190 msgr2=0x561f988e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b37fe700 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8190 0x561f988e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b37fe700 1 -- 172.21.15.45:0/1787521945 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f98666d80 con 0x561f988e7830 2023-10-17T01:16:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8190 0x561f988e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:16:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.875+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 0x561f988e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.876+0000 7f35b37fe700 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f359c010f40 tx=0x7f359c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f16ea58f68c0cbd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.876+0000 7f35a9ffb700 1 -- 172.21.15.45:0/1787521945 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c011db0 con 0x561f988e7830 2023-10-17T01:16:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.877+0000 7f35a9ffb700 1 -- 172.21.15.45:0/1787521945 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c015070 con 0x561f988e7830 2023-10-17T01:16:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.877+0000 7f35a9ffb700 1 -- 172.21.15.45:0/1787521945 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0056a0 con 0x561f988e7830 2023-10-17T01:16:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.877+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/1787521945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 msgr2=0x561f988e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.877+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f359c010f40 tx=0x7f359c013000 comp rx=0 tx=0).stop 2023-10-17T01:16:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/1787521945 shutdown_connections 2023-10-17T01:16:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8d90 0x561f988e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/1787521945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8190 0x561f988e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/1787521945 >> 172.21.15.45:0/1787521945 conn(0x561f98847070 msgr2=0x561f988e22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/1787521945 shutdown_connections 2023-10-17T01:16:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.878+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/1787521945 wait complete. 2023-10-17T01:16:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.879+0000 7f35cdaf3b80 1 -- start start 2023-10-17T01:16:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 Processor -- start 2023-10-17T01:16:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e7830 0x561f988f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f987d54b0 con 0x561f988e7830 2023-10-17T01:16:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f987d5150 con 0x561f988e8190 2023-10-17T01:16:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.880+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f986bc8c0 con 0x561f988e8d90 2023-10-17T01:16:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.881+0000 7f35cdaf3b80 10 client.0 _ll_register_callbacks cb 0x561f98722a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.881+0000 7f35b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.881+0000 7f35b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52248/0 (socket says 172.21.15.45:52248) 2023-10-17T01:16:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.881+0000 7f35b3fff700 1 -- 172.21.15.45:0/4278938849 learned_addr learned my addr 172.21.15.45:0/4278938849 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.881+0000 7f35b37fe700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e7830 0x561f988f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.881+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.882+0000 7f35b37fe700 1 -- 172.21.15.45:0/4278938849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 msgr2=0x561f988f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.882+0000 7f35b37fe700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.882+0000 7f35b37fe700 1 -- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 msgr2=0x561f988f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.882+0000 7f35b37fe700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.882+0000 7f35b37fe700 1 -- 172.21.15.45:0/4278938849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b41058b0 con 0x561f988e7830 2023-10-17T01:16:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.883+0000 7f35b37fe700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e7830 0x561f988f1440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f359c00bc90 tx=0x7f359c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71641e79028a7688 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.883+0000 7f3592ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:16:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.883+0000 7f3592ffd700 1 -- 172.21.15.45:0/4278938849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0048f0 con 0x561f988e7830 2023-10-17T01:16:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.883+0000 7f3592ffd700 1 -- 172.21.15.45:0/4278938849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f359c017070 con 0x561f988e7830 2023-10-17T01:16:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.883+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f98863cb0 con 0x561f988e7830 2023-10-17T01:16:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.885+0000 7f3592ffd700 1 -- 172.21.15.45:0/4278938849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f359c004a50 con 0x561f988e7830 2023-10-17T01:16:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.885+0000 7f35cdaf3b80 10 client.15567 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.885+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f359c010c20 con 0x561f988e7830 2023-10-17T01:16:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f3592ffd700 1 -- 172.21.15.45:0/4278938849 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f98863cb0 con 0x561f988e7830 2023-10-17T01:16:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f3592ffd700 1 -- 172.21.15.45:0/4278938849 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f359c024030 con 0x561f988e7830 2023-10-17T01:16:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35cdaf3b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35cdaf3b80 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35cdaf3b80 20 client.15567 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f988fd1e0 0x561f988ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35cdaf3b80 4 client.15567 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:16:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f988b6cd0 con 0x561f988fd1e0 2023-10-17T01:16:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.886+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f988fd1e0 0x561f988ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.887+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f988fd1e0 0x561f988ff600 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2a487f3d560720 server_cookie=21cb2584b8ed941 in_seq=0 out_seq=0 2023-10-17T01:16:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.887+0000 7f3592ffd700 10 client.15567 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.889+0000 7f3592ffd700 1 -- 172.21.15.45:0/4278938849 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x561f988b6cd0 con 0x561f988fd1e0 2023-10-17T01:16:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.889+0000 7f3592ffd700 10 client.15567 handle_command_reply: tid=0 2023-10-17T01:16:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.889+0000 7f35cdaf3b80 1 client.15567 shutdown 2023-10-17T01:16:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e7830 msgr2=0x561f988f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e7830 0x561f988f1440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f359c00bc90 tx=0x7f359c011de0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 shutdown_connections 2023-10-17T01:16:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e7830 0x561f988f1440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f988fd1e0 0x561f988ff600 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/4278938849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 >> 172.21.15.45:0/4278938849 conn(0x561f98847070 msgr2=0x561f988e1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.890+0000 7f3592ffd700 0 client.15567 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.891+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 shutdown_connections 2023-10-17T01:16:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.891+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/4278938849 wait complete. 2023-10-17T01:16:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.891+0000 7f35cdaf3b80 20 client.15567 trim_cache size 0 max 16384 2023-10-17T01:16:37.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:37 smithi181 ceph-mon[134520]: pgmap v377: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 672 op/s 2023-10-17T01:16:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.931+0000 7f35cdaf3b80 1 Processor -- start 2023-10-17T01:16:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.931+0000 7f35cdaf3b80 1 -- start start 2023-10-17T01:16:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.931+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.931+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.931+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8d90 0x561f988ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f988fcd40 con 0x561f988e8d90 2023-10-17T01:16:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f988f08f0 con 0x561f988e8190 2023-10-17T01:16:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f988e8bf0 con 0x561f988e7830 2023-10-17T01:16:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52258/0 (socket says 172.21.15.45:52258) 2023-10-17T01:16:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 -- 172.21.15.45:0/3402098929 learned_addr learned my addr 172.21.15.45:0/3402098929 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 -- 172.21.15.45:0/3402098929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 msgr2=0x561f988f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 -- 172.21.15.45:0/3402098929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8d90 msgr2=0x561f988ebcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8d90 0x561f988ebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.932+0000 7f35b37fe700 1 -- 172.21.15.45:0/3402098929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f9864fc00 con 0x561f988e7830 2023-10-17T01:16:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.933+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:16:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.933+0000 7f35b37fe700 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f359c00bb30 tx=0x7f359c00bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e5f4a748aadb8c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.933+0000 7f35aa7fc700 1 -- 172.21.15.45:0/3402098929 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c003880 con 0x561f988e7830 2023-10-17T01:16:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.933+0000 7f35aa7fc700 1 -- 172.21.15.45:0/3402098929 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c005910 con 0x561f988e7830 2023-10-17T01:16:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.933+0000 7f35aa7fc700 1 -- 172.21.15.45:0/3402098929 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c03ca10 con 0x561f988e7830 2023-10-17T01:16:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/3402098929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 msgr2=0x561f988e7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f359c00bb30 tx=0x7f359c00bb60 comp rx=0 tx=0).stop 2023-10-17T01:16:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/3402098929 shutdown_connections 2023-10-17T01:16:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f988e8d90 0x561f988ebcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e7830 0x561f988e7c10 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f359c00bb30 tx=0x7f359c00bb60 comp rx=0 tx=0).stop 2023-10-17T01:16:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/3402098929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/3402098929 >> 172.21.15.45:0/3402098929 conn(0x561f98847070 msgr2=0x561f988f0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/3402098929 shutdown_connections 2023-10-17T01:16:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.935+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/3402098929 wait complete. 2023-10-17T01:16:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.936+0000 7f35cdaf3b80 1 -- start start 2023-10-17T01:16:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.936+0000 7f35cdaf3b80 1 Processor -- start 2023-10-17T01:16:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9894b190 0x561f9894d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9894dac0 con 0x561f9894b190 2023-10-17T01:16:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9894dc00 con 0x561f988e8190 2023-10-17T01:16:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9895edd0 con 0x561f988e8d90 2023-10-17T01:16:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.937+0000 7f35cdaf3b80 10 client.0 _ll_register_callbacks cb 0x7f35ac009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.938+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41036/0 (socket says 172.21.15.45:41036) 2023-10-17T01:16:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 -- 172.21.15.45:0/208703676 learned_addr learned my addr 172.21.15.45:0/208703676 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 -- 172.21.15.45:0/208703676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 msgr2=0x561f988f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 -- 172.21.15.45:0/208703676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9894b190 msgr2=0x561f9894d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.939+0000 7f35b37fe700 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9894b190 0x561f9894d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.940+0000 7f35b37fe700 1 -- 172.21.15.45:0/208703676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c010c20 con 0x561f988e8190 2023-10-17T01:16:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.940+0000 7f35b37fe700 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f6050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f359c03c620 tx=0x7f359c01ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f31abf21f58d7154 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.941+0000 7f3592ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:16:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.941+0000 7f3592ffd700 1 -- 172.21.15.45:0/208703676 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c03c950 con 0x561f988e8190 2023-10-17T01:16:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.941+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f359c015070 con 0x561f988e8190 2023-10-17T01:16:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.941+0000 7f3592ffd700 1 -- 172.21.15.45:0/208703676 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f359c041070 con 0x561f988e8190 2023-10-17T01:16:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.942+0000 7f3592ffd700 1 -- 172.21.15.45:0/208703676 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f359c03c950 con 0x561f988e8190 2023-10-17T01:16:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.942+0000 7f35cdaf3b80 10 client.25697 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:37.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.942+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561f9868d1f0 con 0x561f988e8190 2023-10-17T01:16:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.942+0000 7f3592ffd700 1 -- 172.21.15.45:0/208703676 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c015070 con 0x561f988e8190 2023-10-17T01:16:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f3592ffd700 1 -- 172.21.15.45:0/208703676 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f359c024070 con 0x561f988e8190 2023-10-17T01:16:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f35cdaf3b80 10 client.25697 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f35cdaf3b80 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f35cdaf3b80 20 client.25697 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f989602f0 0x561f98962710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f35cdaf3b80 4 client.25697 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:16:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.943+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f98831ce0 con 0x561f989602f0 2023-10-17T01:16:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.944+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f989602f0 0x561f98962710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.945+0000 7f35b2ffd700 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f989602f0 0x561f98962710 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e69d0c25073eed server_cookie=8102b0a3ab2d09a3 in_seq=0 out_seq=0 2023-10-17T01:16:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.945+0000 7f3592ffd700 10 client.25697 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.962+0000 7f3592ffd700 1 -- 172.21.15.45:0/208703676 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f98831ce0 con 0x561f989602f0 2023-10-17T01:16:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.962+0000 7f3592ffd700 10 client.25697 handle_command_reply: tid=0 2023-10-17T01:16:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.963+0000 7f35cdaf3b80 1 client.25697 shutdown 2023-10-17T01:16:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.963+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 msgr2=0x561f988f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.963+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f6050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f359c03c620 tx=0x7f359c01ee40 comp rx=0 tx=0).stop 2023-10-17T01:16:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 shutdown_connections 2023-10-17T01:16:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f9894b190 0x561f9894d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561f989602f0 0x561f98962710 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561f988e8190 0x561f988f6050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 --2- 172.21.15.45:0/208703676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f988e8d90 0x561f988f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f3592ffd700 0 client.25697 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 >> 172.21.15.45:0/208703676 conn(0x561f98847070 msgr2=0x561f988ebcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 shutdown_connections 2023-10-17T01:16:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.964+0000 7f35cdaf3b80 1 -- 172.21.15.45:0/208703676 wait complete. 2023-10-17T01:16:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.965+0000 7f35cdaf3b80 20 client.25697 trim_cache size 0 max 16384 2023-10-17T01:16:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:16:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:16:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:16:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:16:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35a00a0bf0 msgr2=0x7f35a00a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35a00a0bf0 0x7f35a00a30b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f35b410a940 tx=0x7f359c00b500 comp rx=0 tx=0).stop 2023-10-17T01:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 msgr2=0x7f35b410b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 0x7f35b410b110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c9e0 tx=0x7f35a400ccf0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 shutdown_connections 2023-10-17T01:16:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35a00a0bf0 0x7f35a00a30b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4104900 0x7f35b410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35b40ff5c0 0x7f35b410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 --2- 172.21.15.45:0/1248518514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35b4101f30 0x7f35b410b110 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.966+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 >> 172.21.15.45:0/1248518514 conn(0x7f35b40fb1a0 msgr2=0x7f35b4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.967+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 shutdown_connections 2023-10-17T01:16:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:37.967+0000 7f35ba1ff700 1 -- 172.21.15.45:0/1248518514 wait complete. 2023-10-17T01:16:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:39 smithi045 ceph-mon[98474]: pgmap v378: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 776 op/s 2023-10-17T01:16:39.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:39 smithi060 ceph-mon[134136]: pgmap v378: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 776 op/s 2023-10-17T01:16:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:39 smithi181 ceph-mon[134520]: pgmap v378: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 776 op/s 2023-10-17T01:16:40.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 90.0 s, 2823.3 tps, lat 35.296 ms stddev 40.562 2023-10-17T01:16:41.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:41 smithi045 ceph-mon[98474]: pgmap v379: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 886 op/s 2023-10-17T01:16:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:41 smithi060 ceph-mon[134136]: pgmap v379: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 886 op/s 2023-10-17T01:16:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:41 smithi181 ceph-mon[134520]: pgmap v379: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 886 op/s 2023-10-17T01:16:43.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:43 smithi045 ceph-mon[98474]: pgmap v380: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 890 op/s 2023-10-17T01:16:43.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:43 smithi060 ceph-mon[134136]: pgmap v380: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 890 op/s 2023-10-17T01:16:43.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:43 smithi181 ceph-mon[134520]: pgmap v380: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 890 op/s 2023-10-17T01:16:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 95.0 s, 2702.1 tps, lat 37.020 ms stddev 41.614 2023-10-17T01:16:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:45 smithi060 ceph-mon[134136]: pgmap v381: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-17T01:16:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:45 smithi181 ceph-mon[134520]: pgmap v381: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-17T01:16:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:45 smithi045 ceph-mon[98474]: pgmap v381: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-17T01:16:47.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:47 smithi060 ceph-mon[134136]: pgmap v382: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 918 op/s 2023-10-17T01:16:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:47 smithi181 ceph-mon[134520]: pgmap v382: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 918 op/s 2023-10-17T01:16:48.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:47 smithi045 ceph-mon[98474]: pgmap v382: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 918 op/s 2023-10-17T01:16:49.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:49 smithi060 ceph-mon[134136]: pgmap v383: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:16:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:49 smithi181 ceph-mon[134520]: pgmap v383: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:16:50.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:49 smithi045 ceph-mon[98474]: pgmap v383: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:16:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 100.0 s, 2271.2 tps, lat 44.063 ms stddev 86.425 2023-10-17T01:16:51.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.241+0000 7fd39b3ff700 1 -- 172.21.15.45:0/1167031947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 msgr2=0x7fd394076dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.241+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/1167031947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 0x7fd394076dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd390009990 tx=0x7fd390009ca0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.242+0000 7fd39b3ff700 1 -- 172.21.15.45:0/1167031947 shutdown_connections 2023-10-17T01:16:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.242+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/1167031947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 0x7fd394077b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.242+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/1167031947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 0x7fd394075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.242+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/1167031947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 0x7fd394076dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.243+0000 7fd39b3ff700 1 -- 172.21.15.45:0/1167031947 >> 172.21.15.45:0/1167031947 conn(0x7fd3940fe500 msgr2=0x7fd394100940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.243+0000 7fd39b3ff700 1 -- 172.21.15.45:0/1167031947 shutdown_connections 2023-10-17T01:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.243+0000 7fd39b3ff700 1 -- 172.21.15.45:0/1167031947 wait complete. 2023-10-17T01:16:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.245+0000 7fd39b3ff700 1 Processor -- start 2023-10-17T01:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.245+0000 7fd39b3ff700 1 -- start start 2023-10-17T01:16:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.246+0000 7fd39b3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 0x7fd39406f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.246+0000 7fd39b3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 0x7fd39406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.246+0000 7fd39b3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 0x7fd394070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39b3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394070fc0 con 0x7fd394075710 2023-10-17T01:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 0x7fd394070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 0x7fd394070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45306/0 (socket says 172.21.15.45:45306) 2023-10-17T01:16:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39999c700 1 -- 172.21.15.45:0/3846572728 learned_addr learned my addr 172.21.15.45:0/3846572728 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39899a700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 0x7fd39406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394071130 con 0x7fd394077310 2023-10-17T01:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.247+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940712f0 con 0x7fd394076950 2023-10-17T01:16:51.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.248+0000 7fd39919b700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 0x7fd39406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.248+0000 7fd39919b700 1 -- 172.21.15.45:0/3846572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 msgr2=0x7fd39406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.248+0000 7fd39919b700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 0x7fd39406fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.249+0000 7fd39919b700 1 -- 172.21.15.45:0/3846572728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 msgr2=0x7fd394070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.249+0000 7fd39919b700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 0x7fd394070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.249+0000 7fd39919b700 1 -- 172.21.15.45:0/3846572728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3900095f0 con 0x7fd394075710 2023-10-17T01:16:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.249+0000 7fd39919b700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 0x7fd39406f550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd38c007d60 tx=0x7fd38c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:51.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.251+0000 7fd38a7fc700 1 -- 172.21.15.45:0/3846572728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00ad40 con 0x7fd394075710 2023-10-17T01:16:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.251+0000 7fd38a7fc700 1 -- 172.21.15.45:0/3846572728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd38c0128e0 con 0x7fd394075710 2023-10-17T01:16:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.251+0000 7fd38a7fc700 1 -- 172.21.15.45:0/3846572728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c011730 con 0x7fd394075710 2023-10-17T01:16:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.251+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3940715d0 con 0x7fd394075710 2023-10-17T01:16:51.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.251+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd394071d80 con 0x7fd394075710 2023-10-17T01:16:51.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.258+0000 7fd3aedccb80 1 Processor -- start 2023-10-17T01:16:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.259+0000 7fd38a7fc700 1 -- 172.21.15.45:0/3846572728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd38c012a50 con 0x7fd394075710 2023-10-17T01:16:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.261+0000 7fd38a7fc700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3800a0d20 0x7fd3800a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.261+0000 7fd38a7fc700 1 -- 172.21.15.45:0/3846572728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd38c0cc150 con 0x7fd394075710 2023-10-17T01:16:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.262+0000 7fd39999c700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3800a0d20 0x7fd3800a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.262+0000 7fd3aedccb80 1 -- start start 2023-10-17T01:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.262+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.262+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 0x5567b07bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.262+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd39919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd39999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd39919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07bec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37308/0 (socket says 172.21.15.45:37308) 2023-10-17T01:16:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd39919b700 1 -- 172.21.15.45:0/1345059133 learned_addr learned my addr 172.21.15.45:0/1345059133 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b07169b0 con 0x5567b07bf190 2023-10-17T01:16:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b0714370 con 0x5567b07bfd90 2023-10-17T01:16:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.263+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b070fc60 con 0x5567b07be830 2023-10-17T01:16:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 -- 172.21.15.45:0/1345059133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 msgr2=0x5567b07bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 -- 172.21.15.45:0/1345059133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 msgr2=0x5567b07bf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 0x5567b07bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 -- 172.21.15.45:0/1345059133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567b053dd80 con 0x5567b07bfd90 2023-10-17T01:16:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3800a0d20 0x7fd3800a31e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135f0 tx=0x7fd3840139b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.264+0000 7fd39999c700 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07c0210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd38400f5a0 tx=0x7fd384019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e069a5c8ee8a386 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.265+0000 7fd37f7fe700 1 -- 172.21.15.45:0/1345059133 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400fe70 con 0x5567b07bfd90 2023-10-17T01:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.265+0000 7fd37f7fe700 1 -- 172.21.15.45:0/1345059133 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd384019070 con 0x5567b07bfd90 2023-10-17T01:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.265+0000 7fd37f7fe700 1 -- 172.21.15.45:0/1345059133 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384011470 con 0x5567b07bfd90 2023-10-17T01:16:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.266+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 msgr2=0x5567b07c0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.266+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07c0210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd38400f5a0 tx=0x7fd384019000 comp rx=0 tx=0).stop 2023-10-17T01:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.267+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 shutdown_connections 2023-10-17T01:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.267+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07c0210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.267+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.267+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/1345059133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 0x5567b07bf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.267+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 >> 172.21.15.45:0/1345059133 conn(0x5567b071e070 msgr2=0x5567b07b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.272+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 shutdown_connections 2023-10-17T01:16:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.272+0000 7fd3aedccb80 1 -- 172.21.15.45:0/1345059133 wait complete. 2023-10-17T01:16:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.273+0000 7fd3aedccb80 1 -- start start 2023-10-17T01:16:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.273+0000 7fd3aedccb80 1 Processor -- start 2023-10-17T01:16:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.274+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.274+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 0x5567b07c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b06ac4b0 con 0x5567b07bf190 2023-10-17T01:16:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b06ac150 con 0x5567b07bfd90 2023-10-17T01:16:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b05938c0 con 0x5567b07be830 2023-10-17T01:16:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd3aedccb80 10 client.0 _ll_register_callbacks cb 0x5567b05f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd39919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd39919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37322/0 (socket says 172.21.15.45:37322) 2023-10-17T01:16:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd39919b700 1 -- 172.21.15.45:0/60570033 learned_addr learned my addr 172.21.15.45:0/60570033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.275+0000 7fd39999c700 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.276+0000 7fd39919b700 1 -- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 msgr2=0x5567b07cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.276+0000 7fd39919b700 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.276+0000 7fd39919b700 1 -- 172.21.15.45:0/60570033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 msgr2=0x5567b07c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.276+0000 7fd39919b700 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 0x5567b07c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.276+0000 7fd39919b700 1 -- 172.21.15.45:0/60570033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38400f360 con 0x5567b07be830 2023-10-17T01:16:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.277+0000 7fd39919b700 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07c83e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0954f0 tx=0x7fd38c019390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ac479b00747185 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.277+0000 7fd37cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:16:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.278+0000 7fd37cff9700 1 -- 172.21.15.45:0/60570033 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c09d360 con 0x5567b07be830 2023-10-17T01:16:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.278+0000 7fd37cff9700 1 -- 172.21.15.45:0/60570033 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd38c099770 con 0x5567b07be830 2023-10-17T01:16:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.278+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567b073acb0 con 0x5567b07be830 2023-10-17T01:16:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.279+0000 7fd37cff9700 1 -- 172.21.15.45:0/60570033 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd38c09d5a0 con 0x5567b07be830 2023-10-17T01:16:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.279+0000 7fd3aedccb80 10 client.25687 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.279+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5567b0526c00 con 0x5567b07be830 2023-10-17T01:16:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.279+0000 7fd37cff9700 1 -- 172.21.15.45:0/60570033 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567b073acb0 con 0x5567b07be830 2023-10-17T01:16:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd37cff9700 1 -- 172.21.15.45:0/60570033 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd38c0a1350 con 0x5567b07be830 2023-10-17T01:16:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd3aedccb80 10 client.25687 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd3aedccb80 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd3aedccb80 20 client.25687 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b07d42a0 0x5567b07d66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd3aedccb80 4 client.25687 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:16:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.280+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567b078dcd0 con 0x5567b07d42a0 2023-10-17T01:16:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.281+0000 7fd39899a700 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b07d42a0 0x5567b07d66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.282+0000 7fd39899a700 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b07d42a0 0x5567b07d66c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6208e4f4e61bb6 server_cookie=20579fa2b004f0f2 in_seq=0 out_seq=0 2023-10-17T01:16:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.282+0000 7fd37cff9700 10 client.25687 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.283+0000 7fd37cff9700 1 -- 172.21.15.45:0/60570033 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5567b078dcd0 con 0x5567b07d42a0 2023-10-17T01:16:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.283+0000 7fd37cff9700 10 client.25687 handle_command_reply: tid=0 2023-10-17T01:16:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.283+0000 7fd3aedccb80 1 client.25687 shutdown 2023-10-17T01:16:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 msgr2=0x5567b07c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07c83e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0954f0 tx=0x7fd38c019390 comp rx=0 tx=0).stop 2023-10-17T01:16:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 shutdown_connections 2023-10-17T01:16:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b07d42a0 0x5567b07d66c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bfd90 0x5567b07cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07be830 0x5567b07c83e0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0954f0 tx=0x7fd38c019390 comp rx=0 tx=0).stop 2023-10-17T01:16:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/60570033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bf190 0x5567b07c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 >> 172.21.15.45:0/60570033 conn(0x5567b071e070 msgr2=0x5567b07b8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.284+0000 7fd37cff9700 0 client.25687 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.285+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 shutdown_connections 2023-10-17T01:16:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.285+0000 7fd3aedccb80 1 -- 172.21.15.45:0/60570033 wait complete. 2023-10-17T01:16:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.286+0000 7fd3aedccb80 20 client.25687 trim_cache size 0 max 16384 2023-10-17T01:16:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.314+0000 7fd3aedccb80 1 Processor -- start 2023-10-17T01:16:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.315+0000 7fd3aedccb80 1 -- start start 2023-10-17T01:16:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.315+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bf190 0x5567b07c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bfd90 0x5567b07b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07cd910 0x5567b07c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b07ba5b0 con 0x5567b07cd910 2023-10-17T01:16:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b07d3e30 con 0x5567b07bf190 2023-10-17T01:16:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b07c92e0 con 0x5567b07bfd90 2023-10-17T01:16:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd39919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bf190 0x5567b07c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd39919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bf190 0x5567b07c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45338/0 (socket says 172.21.15.45:45338) 2023-10-17T01:16:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd39899a700 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bfd90 0x5567b07b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.316+0000 7fd39919b700 1 -- 172.21.15.45:0/23818901 learned_addr learned my addr 172.21.15.45:0/23818901 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.317+0000 7fd39899a700 1 -- 172.21.15.45:0/23818901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bf190 msgr2=0x5567b07c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.317+0000 7fd39899a700 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bf190 0x5567b07c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.317+0000 7fd39899a700 1 -- 172.21.15.45:0/23818901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07cd910 msgr2=0x5567b07c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.317+0000 7fd39899a700 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07cd910 0x5567b07c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.317+0000 7fd39899a700 1 -- 172.21.15.45:0/23818901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567b05641f0 con 0x5567b07bfd90 2023-10-17T01:16:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.317+0000 7fd39899a700 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bfd90 0x5567b07b8a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5567b07be5d0 tx=0x7fd390009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2ec96c149b3e1a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.319+0000 7fd37ffff700 1 -- 172.21.15.45:0/23818901 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39001d040 con 0x5567b07bfd90 2023-10-17T01:16:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.319+0000 7fd37ffff700 1 -- 172.21.15.45:0/23818901 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd390025070 con 0x5567b07bfd90 2023-10-17T01:16:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.319+0000 7fd37ffff700 1 -- 172.21.15.45:0/23818901 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900214f0 con 0x5567b07bfd90 2023-10-17T01:16:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.320+0000 7fd3aedccb80 1 -- 172.21.15.45:0/23818901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bfd90 msgr2=0x5567b07b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.320+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bfd90 0x5567b07b8a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5567b07be5d0 tx=0x7fd390009530 comp rx=0 tx=0).stop 2023-10-17T01:16:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.321+0000 7fd3aedccb80 1 -- 172.21.15.45:0/23818901 shutdown_connections 2023-10-17T01:16:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.322+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07cd910 0x5567b07c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.322+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07bf190 0x5567b07c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.322+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/23818901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bfd90 0x5567b07b8a00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.322+0000 7fd3aedccb80 1 -- 172.21.15.45:0/23818901 >> 172.21.15.45:0/23818901 conn(0x5567b071e070 msgr2=0x5567b07b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.323+0000 7fd3aedccb80 1 -- 172.21.15.45:0/23818901 shutdown_connections 2023-10-17T01:16:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.323+0000 7fd3aedccb80 1 -- 172.21.15.45:0/23818901 wait complete. 2023-10-17T01:16:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.324+0000 7fd3aedccb80 1 -- start start 2023-10-17T01:16:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.324+0000 7fd3aedccb80 1 Processor -- start 2023-10-17T01:16:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bf190 0x5567b081de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bfd90 0x5567b081e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 0x5567b08253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b0825920 con 0x5567b07bfd90 2023-10-17T01:16:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b0825a60 con 0x5567b07cd910 2023-10-17T01:16:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b0825bf0 con 0x5567b07bf190 2023-10-17T01:16:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.325+0000 7fd3aedccb80 10 client.0 _ll_register_callbacks cb 0x7fd38c099380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 0x5567b08253e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 0x5567b08253e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45348/0 (socket says 172.21.15.45:45348) 2023-10-17T01:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39999c700 1 -- 172.21.15.45:0/3454938861 learned_addr learned my addr 172.21.15.45:0/3454938861 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39919b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bf190 0x5567b081de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39899a700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bfd90 0x5567b081e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39999c700 1 -- 172.21.15.45:0/3454938861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bf190 msgr2=0x5567b081de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39999c700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bf190 0x5567b081de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.326+0000 7fd39999c700 1 -- 172.21.15.45:0/3454938861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bfd90 msgr2=0x5567b081e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.327+0000 7fd39999c700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bfd90 0x5567b081e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.327+0000 7fd39999c700 1 -- 172.21.15.45:0/3454938861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3940785c0 con 0x5567b07cd910 2023-10-17T01:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.327+0000 7fd37cff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.327+0000 7fd39999c700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 0x5567b08253e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd38401ffd0 tx=0x7fd384005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f420624325363a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.327+0000 7fd39919b700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bf190 0x5567b081de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.327+0000 7fd37cff9700 1 -- 172.21.15.45:0/3454938861 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840132a0 con 0x5567b07cd910 2023-10-17T01:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd39899a700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bfd90 0x5567b081e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:16:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd37cff9700 1 -- 172.21.15.45:0/3454938861 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd38400fc40 con 0x5567b07cd910 2023-10-17T01:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd384019070 con 0x5567b07cd910 2023-10-17T01:16:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd37cff9700 1 -- 172.21.15.45:0/3454938861 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3840132a0 con 0x5567b07cd910 2023-10-17T01:16:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd3aedccb80 10 client.25715 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5567b053c250 con 0x5567b07cd910 2023-10-17T01:16:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.328+0000 7fd37cff9700 1 -- 172.21.15.45:0/3454938861 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd384019070 con 0x5567b07cd910 2023-10-17T01:16:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd37cff9700 1 -- 172.21.15.45:0/3454938861 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd384030020 con 0x5567b07cd910 2023-10-17T01:16:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd3aedccb80 10 client.25715 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd3aedccb80 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd3aedccb80 20 client.25715 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b08275d0 0x5567b08299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd3aedccb80 4 client.25715 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:16:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.329+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567b0708ce0 con 0x5567b08275d0 2023-10-17T01:16:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.330+0000 7fd39919b700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b08275d0 0x5567b08299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.331+0000 7fd37cff9700 10 client.25715 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.331+0000 7fd39919b700 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b08275d0 0x5567b08299f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508f715eb07b7a0c server_cookie=9d238e3a1e117924 in_seq=0 out_seq=0 2023-10-17T01:16:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.345+0000 7fd37cff9700 1 -- 172.21.15.45:0/3454938861 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567b0708ce0 con 0x5567b08275d0 2023-10-17T01:16:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.345+0000 7fd37cff9700 10 client.25715 handle_command_reply: tid=0 2023-10-17T01:16:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.345+0000 7fd3aedccb80 1 client.25715 shutdown 2023-10-17T01:16:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 msgr2=0x5567b08253e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 0x5567b08253e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd38401ffd0 tx=0x7fd384005bb0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 shutdown_connections 2023-10-17T01:16:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5567b08275d0 0x5567b08299f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5567b07cd910 0x5567b08253e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5567b07bf190 0x5567b081de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 --2- 172.21.15.45:0/3454938861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567b07bfd90 0x5567b081e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd37cff9700 0 client.25715 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 >> 172.21.15.45:0/3454938861 conn(0x5567b071e070 msgr2=0x5567b07c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.347+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 shutdown_connections 2023-10-17T01:16:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.348+0000 7fd3aedccb80 1 -- 172.21.15.45:0/3454938861 wait complete. 2023-10-17T01:16:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.348+0000 7fd3aedccb80 20 client.25715 trim_cache size 0 max 16384 2023-10-17T01:16:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:16:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:16:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:16:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:16:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.349+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3800a0d20 msgr2=0x7fd3800a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.349+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3800a0d20 0x7fd3800a31e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135f0 tx=0x7fd3840139b0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.349+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 msgr2=0x7fd39406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.349+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 0x7fd39406f550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd38c007d60 tx=0x7fd38c00a340 comp rx=0 tx=0).stop 2023-10-17T01:16:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 shutdown_connections 2023-10-17T01:16:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd3800a0d20 0x7fd3800a31e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd394077310 0x7fd394070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd394075710 0x7fd39406f550 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 --2- 172.21.15.45:0/3846572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd394076950 0x7fd39406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 >> 172.21.15.45:0/3846572728 conn(0x7fd3940fe500 msgr2=0x7fd3941000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 shutdown_connections 2023-10-17T01:16:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:16:51.350+0000 7fd39b3ff700 1 -- 172.21.15.45:0/3846572728 wait complete. 2023-10-17T01:16:51.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:51 smithi060 ceph-mon[134136]: pgmap v384: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.14k op/s 2023-10-17T01:16:51.915 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:51 smithi181 ceph-mon[134520]: pgmap v384: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.14k op/s 2023-10-17T01:16:52.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:51 smithi045 ceph-mon[98474]: pgmap v384: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.14k op/s 2023-10-17T01:16:52.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:52 smithi181 ceph-mon[134520]: pgmap v385: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.37k op/s 2023-10-17T01:16:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:52 smithi045 ceph-mon[98474]: pgmap v385: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.37k op/s 2023-10-17T01:16:53.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:52 smithi060 ceph-mon[134136]: pgmap v385: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 1.37k op/s 2023-10-17T01:16:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 105.0 s, 3089.8 tps, lat 32.286 ms stddev 34.785 2023-10-17T01:16:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:55 smithi060 ceph-mon[134136]: pgmap v386: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 2.02k op/s 2023-10-17T01:16:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:55 smithi181 ceph-mon[134520]: pgmap v386: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 2.02k op/s 2023-10-17T01:16:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:55 smithi045 ceph-mon[98474]: pgmap v386: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 2.02k op/s 2023-10-17T01:16:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:57 smithi181 ceph-mon[134520]: pgmap v387: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 1.87k op/s 2023-10-17T01:16:58.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:57 smithi045 ceph-mon[98474]: pgmap v387: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 1.87k op/s 2023-10-17T01:16:58.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:57 smithi060 ceph-mon[134136]: pgmap v387: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 1.87k op/s 2023-10-17T01:16:58.104 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:16:58.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 -- 172.21.15.45:0/3063142279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c100e30 msgr2=0x7fd50c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 --2- 172.21.15.45:0/3063142279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c100e30 0x7fd50c101290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80099c0 tx=0x7fd4f8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 -- 172.21.15.45:0/3063142279 shutdown_connections 2023-10-17T01:16:58.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 --2- 172.21.15.45:0/3063142279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c1017d0 0x7fd50c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 --2- 172.21.15.45:0/3063142279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd50c0ffc40 0x7fd50c100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 --2- 172.21.15.45:0/3063142279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c100e30 0x7fd50c101290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.466+0000 7fd513dff700 1 -- 172.21.15.45:0/3063142279 >> 172.21.15.45:0/3063142279 conn(0x7fd50c0fb1b0 msgr2=0x7fd50c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:58.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.467+0000 7fd513dff700 1 -- 172.21.15.45:0/3063142279 shutdown_connections 2023-10-17T01:16:58.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.467+0000 7fd513dff700 1 -- 172.21.15.45:0/3063142279 wait complete. 2023-10-17T01:16:58.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.468+0000 7fd513dff700 1 Processor -- start 2023-10-17T01:16:58.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 -- start start 2023-10-17T01:16:58.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 0x7fd50c196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd50c100e30 0x7fd50c196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c1017d0 0x7fd50c19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c19e610 con 0x7fd50c100e30 2023-10-17T01:16:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c19e750 con 0x7fd50c1017d0 2023-10-17T01:16:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.469+0000 7fd513dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c10a5e0 con 0x7fd50c0ffc40 2023-10-17T01:16:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 0x7fd50c196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 0x7fd50c196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42492/0 (socket says 172.21.15.45:42492) 2023-10-17T01:16:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 -- 172.21.15.45:0/684926022 learned_addr learned my addr 172.21.15.45:0/684926022 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd51239c700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c1017d0 0x7fd50c19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd51139a700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd50c100e30 0x7fd50c196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 -- 172.21.15.45:0/684926022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c1017d0 msgr2=0x7fd50c19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c1017d0 0x7fd50c19e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 -- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd50c100e30 msgr2=0x7fd50c196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd50c100e30 0x7fd50c196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.470+0000 7fd511b9b700 1 -- 172.21.15.45:0/684926022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80095f0 con 0x7fd50c0ffc40 2023-10-17T01:16:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.471+0000 7fd511b9b700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 0x7fd50c196a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd50000efc0 tx=0x7fd50000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.471+0000 7fd50affd700 1 -- 172.21.15.45:0/684926022 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500018070 con 0x7fd50c0ffc40 2023-10-17T01:16:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.471+0000 7fd50affd700 1 -- 172.21.15.45:0/684926022 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd50000f040 con 0x7fd50c0ffc40 2023-10-17T01:16:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.471+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c10a860 con 0x7fd50c0ffc40 2023-10-17T01:16:58.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.471+0000 7fd50affd700 1 -- 172.21.15.45:0/684926022 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500004310 con 0x7fd50c0ffc40 2023-10-17T01:16:58.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.471+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c10adc0 con 0x7fd50c0ffc40 2023-10-17T01:16:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.476+0000 7fd50affd700 1 -- 172.21.15.45:0/684926022 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd50000ca70 con 0x7fd50c0ffc40 2023-10-17T01:16:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.478+0000 7fd50affd700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd4fc0a0c20 0x7fd4fc0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.478+0000 7fd50affd700 1 -- 172.21.15.45:0/684926022 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd5000cc040 con 0x7fd50c0ffc40 2023-10-17T01:16:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.479+0000 7fd51139a700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd4fc0a0c20 0x7fd4fc0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.481+0000 7fd52768db80 1 Processor -- start 2023-10-17T01:16:58.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.481+0000 7fd51139a700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd4fc0a0c20 0x7fd4fc0a30e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b580 tx=0x7fd4f801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:58.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 -- start start 2023-10-17T01:16:58.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a057d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a057e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a057ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a04cca10 con 0x55a8a057dd60 2023-10-17T01:16:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a04ca3d0 con 0x55a8a057eae0 2023-10-17T01:16:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.482+0000 7fd52768db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a04c5cc0 con 0x55a8a057d400 2023-10-17T01:16:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a057d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a057d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42504/0 (socket says 172.21.15.45:42504) 2023-10-17T01:16:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd511b9b700 1 -- 172.21.15.45:0/1558737406 learned_addr learned my addr 172.21.15.45:0/1558737406 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd51239c700 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a057ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd51139a700 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a057e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd51239c700 1 -- 172.21.15.45:0/1558737406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 msgr2=0x55a8a057d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd51239c700 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a057d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.483+0000 7fd51239c700 1 -- 172.21.15.45:0/1558737406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 msgr2=0x55a8a057e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.484+0000 7fd51239c700 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a057e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.484+0000 7fd51239c700 1 -- 172.21.15.45:0/1558737406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8a02f3de0 con 0x55a8a057eae0 2023-10-17T01:16:58.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.484+0000 7fd511b9b700 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a057d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:58.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.484+0000 7fd51239c700 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a057ef60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd504004630 tx=0x7fd50400b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c87dd1ac56b19b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.485+0000 7fd4f7fff700 1 -- 172.21.15.45:0/1558737406 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504004a30 con 0x55a8a057eae0 2023-10-17T01:16:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.485+0000 7fd4f7fff700 1 -- 172.21.15.45:0/1558737406 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5040054e0 con 0x55a8a057eae0 2023-10-17T01:16:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.485+0000 7fd4f7fff700 1 -- 172.21.15.45:0/1558737406 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504004a30 con 0x55a8a057eae0 2023-10-17T01:16:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.485+0000 7fd52768db80 1 -- 172.21.15.45:0/1558737406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 msgr2=0x55a8a057ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.485+0000 7fd52768db80 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a057ef60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd504004630 tx=0x7fd50400b970 comp rx=0 tx=0).stop 2023-10-17T01:16:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 -- 172.21.15.45:0/1558737406 shutdown_connections 2023-10-17T01:16:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a057ef60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a057d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 --2- 172.21.15.45:0/1558737406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a057e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 -- 172.21.15.45:0/1558737406 >> 172.21.15.45:0/1558737406 conn(0x55a8a04d40d0 msgr2=0x55a8a056f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 -- 172.21.15.45:0/1558737406 shutdown_connections 2023-10-17T01:16:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.486+0000 7fd52768db80 1 -- 172.21.15.45:0/1558737406 wait complete. 2023-10-17T01:16:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.487+0000 7fd52768db80 1 -- start start 2023-10-17T01:16:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.487+0000 7fd52768db80 1 Processor -- start 2023-10-17T01:16:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.489+0000 7fd52768db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a0587090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.490+0000 7fd52768db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a05875d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.490+0000 7fd52768db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a058e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.490+0000 7fd52768db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0462510 con 0x55a8a057dd60 2023-10-17T01:16:58.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.490+0000 7fd52768db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a04621b0 con 0x55a8a057eae0 2023-10-17T01:16:58.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.490+0000 7fd52768db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0349920 con 0x55a8a057d400 2023-10-17T01:16:58.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.490+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a0587090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51139a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a05875d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a058e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a058e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48648/0 (socket says 172.21.15.45:48648) 2023-10-17T01:16:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 -- 172.21.15.45:0/2790882234 learned_addr learned my addr 172.21.15.45:0/2790882234 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd52768db80 10 client.0 _ll_register_callbacks cb 0x55a8a03afa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51139a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a05875d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50392/0 (socket says 172.21.15.45:50392) 2023-10-17T01:16:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a0587090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42516/0 (socket says 172.21.15.45:42516) 2023-10-17T01:16:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 -- 172.21.15.45:0/2790882234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 msgr2=0x55a8a0587090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a0587090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 -- 172.21.15.45:0/2790882234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 msgr2=0x55a8a05875d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a05875d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51239c700 1 -- 172.21.15.45:0/2790882234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c10adc0 con 0x55a8a057eae0 2023-10-17T01:16:58.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.491+0000 7fd51139a700 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a05875d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:58.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.492+0000 7fd51239c700 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a058e610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b5f0 tx=0x7fd504005350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8b2e9bf71914f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:58.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.492+0000 7fd4f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:16:58.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.492+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2790882234 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50401d070 con 0x55a8a057eae0 2023-10-17T01:16:58.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.492+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2790882234 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd504004be0 con 0x55a8a057eae0 2023-10-17T01:16:58.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.492+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8a04f0d10 con 0x55a8a057eae0 2023-10-17T01:16:58.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.493+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2790882234 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd504003760 con 0x55a8a057eae0 2023-10-17T01:16:58.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.493+0000 7fd52768db80 10 client.25727 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:58.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.493+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd504004290 con 0x55a8a057eae0 2023-10-17T01:16:58.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.493+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2790882234 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8a04f0d10 con 0x55a8a057eae0 2023-10-17T01:16:58.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2790882234 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd504011930 con 0x55a8a057eae0 2023-10-17T01:16:58.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd52768db80 10 client.25727 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:58.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd52768db80 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:58.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd52768db80 20 client.25727 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:58.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd52768db80 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a0592d90 0x55a8a05951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd52768db80 4 client.25727 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:16:58.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.494+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8a0543d30 con 0x55a8a0592d90 2023-10-17T01:16:58.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.495+0000 7fd511b9b700 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a0592d90 0x55a8a05951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.496+0000 7fd511b9b700 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a0592d90 0x55a8a05951b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d8b6762a0576ff server_cookie=7bc3624141c3b2a1 in_seq=0 out_seq=0 2023-10-17T01:16:58.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.496+0000 7fd4f57fa700 10 client.25727 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:58.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2790882234 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55a8a0543d30 con 0x55a8a0592d90 2023-10-17T01:16:58.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd4f57fa700 10 client.25727 handle_command_reply: tid=0 2023-10-17T01:16:58.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 client.25727 shutdown 2023-10-17T01:16:58.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 msgr2=0x55a8a058e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a058e610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b5f0 tx=0x7fd504005350 comp rx=0 tx=0).stop 2023-10-17T01:16:58.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 shutdown_connections 2023-10-17T01:16:58.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a0592d90 0x55a8a05951b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057eae0 0x55a8a058e610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a0587090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 --2- 172.21.15.45:0/2790882234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a05875d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 >> 172.21.15.45:0/2790882234 conn(0x55a8a04d40d0 msgr2=0x55a8a056f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:58.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.497+0000 7fd4f57fa700 0 client.25727 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:58.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.498+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 shutdown_connections 2023-10-17T01:16:58.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.498+0000 7fd52768db80 1 -- 172.21.15.45:0/2790882234 wait complete. 2023-10-17T01:16:58.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.498+0000 7fd52768db80 20 client.25727 trim_cache size 0 max 16384 2023-10-17T01:16:58.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.537+0000 7fd52768db80 1 Processor -- start 2023-10-17T01:16:58.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.537+0000 7fd52768db80 1 -- start start 2023-10-17T01:16:58.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd52768db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a05819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd52768db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057dd60 0x55a8a0581ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd52768db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057eae0 0x55a8a0587090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a05819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a05819a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42522/0 (socket says 172.21.15.45:42522) 2023-10-17T01:16:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd51139a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057dd60 0x55a8a0581ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.538+0000 7fd51139a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057dd60 0x55a8a0581ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48656/0 (socket says 172.21.15.45:48656) 2023-10-17T01:16:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd511b9b700 1 -- 172.21.15.45:0/110573096 learned_addr learned my addr 172.21.15.45:0/110573096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a057e960 con 0x55a8a057eae0 2023-10-17T01:16:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a05928d0 con 0x55a8a057dd60 2023-10-17T01:16:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0586540 con 0x55a8a057d400 2023-10-17T01:16:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd511b9b700 1 -- 172.21.15.45:0/110573096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057dd60 msgr2=0x55a8a0581ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd511b9b700 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057dd60 0x55a8a0581ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd511b9b700 1 -- 172.21.15.45:0/110573096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057eae0 msgr2=0x55a8a0587090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd511b9b700 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057eae0 0x55a8a0587090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.539+0000 7fd511b9b700 1 -- 172.21.15.45:0/110573096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8a02dcc60 con 0x55a8a057d400 2023-10-17T01:16:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.540+0000 7fd511b9b700 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a05819a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd500004b80 tx=0x7fd500084040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b49b42b9a83038d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.540+0000 7fd508bff700 1 -- 172.21.15.45:0/110573096 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500099df0 con 0x55a8a057d400 2023-10-17T01:16:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.540+0000 7fd508bff700 1 -- 172.21.15.45:0/110573096 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5000996f0 con 0x55a8a057d400 2023-10-17T01:16:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.540+0000 7fd508bff700 1 -- 172.21.15.45:0/110573096 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500097a90 con 0x55a8a057d400 2023-10-17T01:16:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 msgr2=0x55a8a05819a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a05819a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd500004b80 tx=0x7fd500084040 comp rx=0 tx=0).stop 2023-10-17T01:16:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 shutdown_connections 2023-10-17T01:16:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057eae0 0x55a8a0587090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057d400 0x55a8a05819a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 --2- 172.21.15.45:0/110573096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057dd60 0x55a8a0581ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.541+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 >> 172.21.15.45:0/110573096 conn(0x55a8a04d40d0 msgr2=0x55a8a0584e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:58.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.542+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 shutdown_connections 2023-10-17T01:16:58.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.542+0000 7fd52768db80 1 -- 172.21.15.45:0/110573096 wait complete. 2023-10-17T01:16:58.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.542+0000 7fd52768db80 1 -- start start 2023-10-17T01:16:58.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.543+0000 7fd52768db80 1 Processor -- start 2023-10-17T01:16:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 0x55a8a058c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a058a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057eae0 0x55a8a058aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a058b4e0 con 0x55a8a057dd60 2023-10-17T01:16:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a058b650 con 0x55a8a057d400 2023-10-17T01:16:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a058b810 con 0x55a8a057eae0 2023-10-17T01:16:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.544+0000 7fd52768db80 10 client.0 _ll_register_callbacks cb 0x7fd5000092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:16:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.545+0000 7fd51239c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057eae0 0x55a8a058aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.545+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 0x55a8a058c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.545+0000 7fd511b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 0x55a8a058c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48664/0 (socket says 172.21.15.45:48664) 2023-10-17T01:16:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.545+0000 7fd511b9b700 1 -- 172.21.15.45:0/2209190402 learned_addr learned my addr 172.21.15.45:0/2209190402 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:16:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.545+0000 7fd51139a700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a058a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.546+0000 7fd511b9b700 1 -- 172.21.15.45:0/2209190402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057eae0 msgr2=0x55a8a058aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.546+0000 7fd511b9b700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057eae0 0x55a8a058aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.546+0000 7fd511b9b700 1 -- 172.21.15.45:0/2209190402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 msgr2=0x55a8a058a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:16:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.546+0000 7fd511b9b700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a058a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.546+0000 7fd511b9b700 1 -- 172.21.15.45:0/2209190402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c10a860 con 0x55a8a057d400 2023-10-17T01:16:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.547+0000 7fd51139a700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a058a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:16:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.548+0000 7fd511b9b700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 0x55a8a058c370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd5000a11a0 tx=0x7fd500003d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775f55a9ca6683d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:16:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.548+0000 7fd4f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:16:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.548+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2209190402 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50009d520 con 0x55a8a057d400 2023-10-17T01:16:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.548+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2209190402 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd5000af040 con 0x55a8a057d400 2023-10-17T01:16:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.548+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5040054e0 con 0x55a8a057d400 2023-10-17T01:16:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.549+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2209190402 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5000c9c20 con 0x55a8a057d400 2023-10-17T01:16:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.549+0000 7fd52768db80 10 client.25733 fetch_fsmap learned FSMap version 33 2023-10-17T01:16:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.549+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a8a031a250 con 0x55a8a057d400 2023-10-17T01:16:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.549+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2209190402 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5040054e0 con 0x55a8a057d400 2023-10-17T01:16:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2209190402 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd5000a1240 con 0x55a8a057d400 2023-10-17T01:16:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd52768db80 10 client.25733 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:16:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd52768db80 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:16:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd52768db80 20 client.25733 populate_metadata read hostname 'smithi045' 2023-10-17T01:16:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd52768db80 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a05f6b10 0x55a8a05f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:16:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd52768db80 4 client.25733 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:16:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a8a04bed40 con 0x55a8a05f6b10 2023-10-17T01:16:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.550+0000 7fd51139a700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a05f6b10 0x55a8a05f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:16:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.551+0000 7fd51139a700 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a05f6b10 0x55a8a05f8f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e70f431a19c372a9 server_cookie=656a60f4405e92cd in_seq=0 out_seq=0 2023-10-17T01:16:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.551+0000 7fd4f57fa700 10 client.25733 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.551+0000 7fd4f57fa700 1 -- 172.21.15.45:0/2209190402 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a8a04bed40 con 0x55a8a05f6b10 2023-10-17T01:16:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.551+0000 7fd4f57fa700 10 client.25733 handle_command_reply: tid=0 2023-10-17T01:16:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.551+0000 7fd52768db80 1 client.25733 shutdown 2023-10-17T01:16:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 msgr2=0x55a8a058c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 0x55a8a058c370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd5000a11a0 tx=0x7fd500003d90 comp rx=0 tx=0).stop 2023-10-17T01:16:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 shutdown_connections 2023-10-17T01:16:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a8a057eae0 0x55a8a058aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a8a05f6b10 0x55a8a05f8f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a8a057d400 0x55a8a058c370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 --2- 172.21.15.45:0/2209190402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a8a057dd60 0x55a8a058a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd4f57fa700 0 client.25733 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:16:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.552+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 >> 172.21.15.45:0/2209190402 conn(0x55a8a04d40d0 msgr2=0x55a8a0584e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.553+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 shutdown_connections 2023-10-17T01:16:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.553+0000 7fd52768db80 1 -- 172.21.15.45:0/2209190402 wait complete. 2023-10-17T01:16:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.553+0000 7fd52768db80 20 client.25733 trim_cache size 0 max 16384 2023-10-17T01:16:58.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:16:58.561 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:16:58.561 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:16:58.561 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:16:58.561 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:16:58.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:16:58.562 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:16:58.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:16:58.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:16:58.562 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:16:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd4fc0a0c20 msgr2=0x7fd4fc0a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd4fc0a0c20 0x7fd4fc0a30e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b580 tx=0x7fd4f801a040 comp rx=0 tx=0).stop 2023-10-17T01:16:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 msgr2=0x7fd50c196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:16:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 0x7fd50c196a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd50000efc0 tx=0x7fd50000c3a0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 shutdown_connections 2023-10-17T01:16:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd4fc0a0c20 0x7fd4fc0a30e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c1017d0 0x7fd50c19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50c0ffc40 0x7fd50c196a90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 --2- 172.21.15.45:0/684926022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd50c100e30 0x7fd50c196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:16:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 >> 172.21.15.45:0/684926022 conn(0x7fd50c0fb1b0 msgr2=0x7fd50c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:16:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.554+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 shutdown_connections 2023-10-17T01:16:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:16:58.555+0000 7fd513dff700 1 -- 172.21.15.45:0/684926022 wait complete. 2023-10-17T01:16:58.591 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:16:58.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:16:58 smithi181 ceph-mon[134520]: pgmap v388: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 MiB/s wr, 1.89k op/s 2023-10-17T01:16:59.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:16:58 smithi045 ceph-mon[98474]: pgmap v388: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 MiB/s wr, 1.89k op/s 2023-10-17T01:16:59.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:16:58 smithi060 ceph-mon[134136]: pgmap v388: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 MiB/s wr, 1.89k op/s 2023-10-17T01:17:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 110.0 s, 3096.7 tps, lat 32.378 ms stddev 36.917 2023-10-17T01:17:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:01 smithi045 ceph-mon[98474]: pgmap v389: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 MiB/s wr, 1.88k op/s 2023-10-17T01:17:01.838 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:01 smithi060 ceph-mon[134136]: pgmap v389: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 MiB/s wr, 1.88k op/s 2023-10-17T01:17:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:01 smithi181 ceph-mon[134520]: pgmap v389: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 MiB/s wr, 1.88k op/s 2023-10-17T01:17:03.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:03 smithi045 ceph-mon[98474]: pgmap v390: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 MiB/s wr, 1.71k op/s 2023-10-17T01:17:03.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:03 smithi060 ceph-mon[134136]: pgmap v390: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 MiB/s wr, 1.71k op/s 2023-10-17T01:17:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:03 smithi181 ceph-mon[134520]: pgmap v390: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 MiB/s wr, 1.71k op/s 2023-10-17T01:17:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.604+0000 7f9069bff700 1 -- 172.21.15.45:0/30246819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9064107fc0 msgr2=0x7f906410a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.604+0000 7f9069bff700 1 --2- 172.21.15.45:0/30246819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9064107fc0 0x7f906410a3b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90580098f0 tx=0x7f9058009c00 comp rx=0 tx=0).stop 2023-10-17T01:17:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.605+0000 7f9069bff700 1 -- 172.21.15.45:0/30246819 shutdown_connections 2023-10-17T01:17:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.605+0000 7f9069bff700 1 --2- 172.21.15.45:0/30246819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906410d420 0x7f906410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.605+0000 7f9069bff700 1 --2- 172.21.15.45:0/30246819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9064107fc0 0x7f906410a3b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.605+0000 7f9069bff700 1 --2- 172.21.15.45:0/30246819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f906410a980 0x7f906410cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.605+0000 7f9069bff700 1 -- 172.21.15.45:0/30246819 >> 172.21.15.45:0/30246819 conn(0x7f906406db80 msgr2=0x7f906406ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.606+0000 7f9069bff700 1 -- 172.21.15.45:0/30246819 shutdown_connections 2023-10-17T01:17:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.606+0000 7f9069bff700 1 -- 172.21.15.45:0/30246819 wait complete. 2023-10-17T01:17:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.610+0000 7f9069bff700 1 Processor -- start 2023-10-17T01:17:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.611+0000 7f9069bff700 1 -- start start 2023-10-17T01:17:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.612+0000 7f9069bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9064107fc0 0x7f90641a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.612+0000 7f9069bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 0x7f90641a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.612+0000 7f9069bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f906410d420 0x7f90641a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.612+0000 7f9069bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90641a7350 con 0x7f9064107fc0 2023-10-17T01:17:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.612+0000 7f9069bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90641a74c0 con 0x7f906410d420 2023-10-17T01:17:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.612+0000 7f9069bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90641a7680 con 0x7f906410a980 2023-10-17T01:17:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.613+0000 7f9062ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 0x7f90641a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.613+0000 7f9062ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 0x7f90641a5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42552/0 (socket says 172.21.15.45:42552) 2023-10-17T01:17:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.613+0000 7f9062ffd700 1 -- 172.21.15.45:0/3420942891 learned_addr learned my addr 172.21.15.45:0/3420942891 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.613+0000 7f90637fe700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9064107fc0 0x7f90641a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.613+0000 7f9063fff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f906410d420 0x7f90641a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.614+0000 7f90637fe700 1 -- 172.21.15.45:0/3420942891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 msgr2=0x7f90641a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.614+0000 7f90637fe700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 0x7f90641a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.614+0000 7f90637fe700 1 -- 172.21.15.45:0/3420942891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f906410d420 msgr2=0x7f90641a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.614+0000 7f90637fe700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f906410d420 0x7f90641a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.614+0000 7f90637fe700 1 -- 172.21.15.45:0/3420942891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009520 con 0x7f9064107fc0 2023-10-17T01:17:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.614+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 0x7f90641a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.615+0000 7f90637fe700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9064107fc0 0x7f90641a5930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90580098c0 tx=0x7f905800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.616+0000 7f9060ff9700 1 -- 172.21.15.45:0/3420942891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905801d070 con 0x7f9064107fc0 2023-10-17T01:17:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.616+0000 7f9060ff9700 1 -- 172.21.15.45:0/3420942891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905800fe90 con 0x7f9064107fc0 2023-10-17T01:17:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.616+0000 7f9060ff9700 1 -- 172.21.15.45:0/3420942891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058003e40 con 0x7f9064107fc0 2023-10-17T01:17:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.616+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90641a7900 con 0x7f9064107fc0 2023-10-17T01:17:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.616+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906419c810 con 0x7f9064107fc0 2023-10-17T01:17:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.622+0000 7f9060ff9700 1 -- 172.21.15.45:0/3420942891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9058005250 con 0x7f9064107fc0 2023-10-17T01:17:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.623+0000 7f9060ff9700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f904c0a0ca0 0x7f904c0a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.624+0000 7f9060ff9700 1 -- 172.21.15.45:0/3420942891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f90580cc530 con 0x7f9064107fc0 2023-10-17T01:17:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.624+0000 7f907d55bb80 1 Processor -- start 2023-10-17T01:17:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.624+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f904c0a0ca0 0x7f904c0a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.624+0000 7f907d55bb80 1 -- start start 2023-10-17T01:17:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f2328c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2329790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f232a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f907d55bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f22809b0 con 0x5617f2329190 2023-10-17T01:17:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f907d55bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f227e370 con 0x5617f2329d90 2023-10-17T01:17:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f907d55bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2279c60 con 0x5617f2328830 2023-10-17T01:17:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f90637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f2328c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f90637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f2328c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42562/0 (socket says 172.21.15.45:42562) 2023-10-17T01:17:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.625+0000 7f90637fe700 1 -- 172.21.15.45:0/213222835 learned_addr learned my addr 172.21.15.45:0/213222835 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.626+0000 7f9063fff700 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f232a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.626+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f904c0a0ca0 0x7f904c0a3160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f905400afd0 tx=0x7f905400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.626+0000 7f9062ffd700 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2329790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.627+0000 7f9063fff700 1 -- 172.21.15.45:0/213222835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 msgr2=0x5617f2328c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.627+0000 7f9063fff700 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f2328c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.627+0000 7f9063fff700 1 -- 172.21.15.45:0/213222835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 msgr2=0x5617f2329790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.627+0000 7f9063fff700 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2329790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.627+0000 7f9063fff700 1 -- 172.21.15.45:0/213222835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f20a7d80 con 0x5617f2329d90 2023-10-17T01:17:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.627+0000 7f90637fe700 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f2328c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.628+0000 7f9063fff700 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f232a210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f905c009510 tx=0x7f905c00bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966e618077cf567e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.628+0000 7f9049ffb700 1 -- 172.21.15.45:0/213222835 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c019070 con 0x5617f2329d90 2023-10-17T01:17:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.629+0000 7f9049ffb700 1 -- 172.21.15.45:0/213222835 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c011040 con 0x5617f2329d90 2023-10-17T01:17:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.629+0000 7f9049ffb700 1 -- 172.21.15.45:0/213222835 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c019070 con 0x5617f2329d90 2023-10-17T01:17:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.630+0000 7f907d55bb80 1 -- 172.21.15.45:0/213222835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 msgr2=0x5617f232a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.630+0000 7f907d55bb80 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f232a210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f905c009510 tx=0x7f905c00bcf0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.631+0000 7f907d55bb80 1 -- 172.21.15.45:0/213222835 shutdown_connections 2023-10-17T01:17:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.631+0000 7f907d55bb80 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f232a210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.631+0000 7f907d55bb80 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f2328c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.631+0000 7f907d55bb80 1 --2- 172.21.15.45:0/213222835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2329790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.631+0000 7f907d55bb80 1 -- 172.21.15.45:0/213222835 >> 172.21.15.45:0/213222835 conn(0x5617f2288070 msgr2=0x5617f23232c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.632+0000 7f907d55bb80 1 -- 172.21.15.45:0/213222835 shutdown_connections 2023-10-17T01:17:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.632+0000 7f907d55bb80 1 -- 172.21.15.45:0/213222835 wait complete. 2023-10-17T01:17:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.633+0000 7f907d55bb80 1 -- start start 2023-10-17T01:17:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.633+0000 7f907d55bb80 1 Processor -- start 2023-10-17T01:17:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.635+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f23323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.635+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2332900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.635+0000 7f90637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f23323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.635+0000 7f90637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f23323c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42570/0 (socket says 172.21.15.45:42570) 2023-10-17T01:17:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.635+0000 7f90637fe700 1 -- 172.21.15.45:0/4070987280 learned_addr learned my addr 172.21.15.45:0/4070987280 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2339940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f22164b0 con 0x5617f2329190 2023-10-17T01:17:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f9062ffd700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2332900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2216150 con 0x5617f2329d90 2023-10-17T01:17:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f20fd8c0 con 0x5617f2328830 2023-10-17T01:17:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f9063fff700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2339940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f907d55bb80 10 client.0 _ll_register_callbacks cb 0x5617f2163a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f90637fe700 1 -- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 msgr2=0x5617f2339940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f90637fe700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2339940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f90637fe700 1 -- 172.21.15.45:0/4070987280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 msgr2=0x5617f2332900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f90637fe700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2332900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f90637fe700 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906419c810 con 0x5617f2328830 2023-10-17T01:17:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.636+0000 7f9062ffd700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2332900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.637+0000 7f90637fe700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f23323c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90580ca3d0 tx=0x7f9058095f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1257e0fc45c180f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.637+0000 7f9042ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:17:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.637+0000 7f9042ffd700 1 -- 172.21.15.45:0/4070987280 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058099c40 con 0x5617f2328830 2023-10-17T01:17:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.637+0000 7f9042ffd700 1 -- 172.21.15.45:0/4070987280 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f905809da70 con 0x5617f2328830 2023-10-17T01:17:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.637+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617f22a4cb0 con 0x5617f2328830 2023-10-17T01:17:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.637+0000 7f9042ffd700 1 -- 172.21.15.45:0/4070987280 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90580a1c20 con 0x5617f2328830 2023-10-17T01:17:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.638+0000 7f907d55bb80 10 client.25723 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.638+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f905c0091f0 con 0x5617f2328830 2023-10-17T01:17:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.638+0000 7f9042ffd700 1 -- 172.21.15.45:0/4070987280 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617f22a4cb0 con 0x5617f2328830 2023-10-17T01:17:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.638+0000 7f9042ffd700 1 -- 172.21.15.45:0/4070987280 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9058099130 con 0x5617f2328830 2023-10-17T01:17:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.638+0000 7f907d55bb80 10 client.25723 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.640+0000 7f907d55bb80 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.640+0000 7f907d55bb80 20 client.25723 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.640+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f233de10 0x5617f2340230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.640+0000 7f907d55bb80 4 client.25723 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.640+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f22f7cd0 con 0x5617f233de10 2023-10-17T01:17:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.640+0000 7f9062ffd700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f233de10 0x5617f2340230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.641+0000 7f9062ffd700 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f233de10 0x5617f2340230 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7950e82f783844c server_cookie=7d289eb1f6f4be14 in_seq=0 out_seq=0 2023-10-17T01:17:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.641+0000 7f9042ffd700 10 client.25723 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.642+0000 7f9042ffd700 1 -- 172.21.15.45:0/4070987280 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5617f22f7cd0 con 0x5617f233de10 2023-10-17T01:17:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.642+0000 7f9042ffd700 10 client.25723 handle_command_reply: tid=0 2023-10-17T01:17:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.642+0000 7f907d55bb80 1 client.25723 shutdown 2023-10-17T01:17:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 msgr2=0x5617f23323c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f23323c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90580ca3d0 tx=0x7f9058095f80 comp rx=0 tx=0).stop 2023-10-17T01:17:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 shutdown_connections 2023-10-17T01:17:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f233de10 0x5617f2340230 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2339940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2328830 0x5617f23323c0 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90580ca3d0 tx=0x7f9058095f80 comp rx=0 tx=0).stop 2023-10-17T01:17:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 --2- 172.21.15.45:0/4070987280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2332900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 >> 172.21.15.45:0/4070987280 conn(0x5617f2288070 msgr2=0x5617f2322a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f9042ffd700 0 client.25723 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 shutdown_connections 2023-10-17T01:17:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.643+0000 7f907d55bb80 1 -- 172.21.15.45:0/4070987280 wait complete. 2023-10-17T01:17:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.644+0000 7f907d55bb80 20 client.25723 trim_cache size 0 max 16384 2023-10-17T01:17:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.679+0000 7f907d55bb80 1 Processor -- start 2023-10-17T01:17:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.679+0000 7f907d55bb80 1 -- start start 2023-10-17T01:17:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.680+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329190 0x5617f232c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.680+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329d90 0x5617f233a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.680+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f232cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.680+0000 7f907d55bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f22f7cd0 con 0x5617f2329d90 2023-10-17T01:17:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.680+0000 7f907d55bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f22a4cb0 con 0x5617f2329190 2023-10-17T01:17:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.680+0000 7f907d55bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2333540 con 0x5617f233a650 2023-10-17T01:17:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f90637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329190 0x5617f232c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f9063fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f232cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f9063fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f232cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42572/0 (socket says 172.21.15.45:42572) 2023-10-17T01:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f9063fff700 1 -- 172.21.15.45:0/3962625752 learned_addr learned my addr 172.21.15.45:0/3962625752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329d90 0x5617f233a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f90637fe700 1 -- 172.21.15.45:0/3962625752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 msgr2=0x5617f232cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f90637fe700 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f232cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f90637fe700 1 -- 172.21.15.45:0/3962625752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329d90 msgr2=0x5617f233a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f90637fe700 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329d90 0x5617f233a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.681+0000 7f90637fe700 1 -- 172.21.15.45:0/3962625752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f2090c00 con 0x5617f2329190 2023-10-17T01:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.682+0000 7f90637fe700 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329190 0x5617f232c210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f905800faa0 tx=0x7f9058009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4472a48ff35a8e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.682+0000 7f904a7fc700 1 -- 172.21.15.45:0/3962625752 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90580a1a20 con 0x5617f2329190 2023-10-17T01:17:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.682+0000 7f904a7fc700 1 -- 172.21.15.45:0/3962625752 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905809da70 con 0x5617f2329190 2023-10-17T01:17:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.682+0000 7f904a7fc700 1 -- 172.21.15.45:0/3962625752 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90580cc0c0 con 0x5617f2329190 2023-10-17T01:17:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.683+0000 7f907d55bb80 1 -- 172.21.15.45:0/3962625752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329190 msgr2=0x5617f232c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.683+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329190 0x5617f232c210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f905800faa0 tx=0x7f9058009810 comp rx=0 tx=0).stop 2023-10-17T01:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.683+0000 7f907d55bb80 1 -- 172.21.15.45:0/3962625752 shutdown_connections 2023-10-17T01:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.683+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f232cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.683+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329190 0x5617f232c210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.683+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3962625752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329d90 0x5617f233a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.684+0000 7f907d55bb80 1 -- 172.21.15.45:0/3962625752 >> 172.21.15.45:0/3962625752 conn(0x5617f2288070 msgr2=0x5617f23310c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.684+0000 7f907d55bb80 1 -- 172.21.15.45:0/3962625752 shutdown_connections 2023-10-17T01:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.684+0000 7f907d55bb80 1 -- 172.21.15.45:0/3962625752 wait complete. 2023-10-17T01:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.685+0000 7f907d55bb80 1 -- start start 2023-10-17T01:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.685+0000 7f907d55bb80 1 Processor -- start 2023-10-17T01:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.688+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2337010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.688+0000 7f907d55bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2337590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.688+0000 7f90637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2337010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.689+0000 7f9062ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2337590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.689+0000 7f9062ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2337590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48720/0 (socket says 172.21.15.45:48720) 2023-10-17T01:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.689+0000 7f9062ffd700 1 -- 172.21.15.45:0/3745675348 learned_addr learned my addr 172.21.15.45:0/3745675348 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.689+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f23358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.689+0000 7f9063fff700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f23358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2335e90 con 0x5617f2329190 2023-10-17T01:17:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2336000 con 0x5617f2329d90 2023-10-17T01:17:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f23361c0 con 0x5617f233a650 2023-10-17T01:17:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f907d55bb80 10 client.0 _ll_register_callbacks cb 0x5617f2163a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f9062ffd700 1 -- 172.21.15.45:0/3745675348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 msgr2=0x5617f23358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f23358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f9062ffd700 1 -- 172.21.15.45:0/3745675348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 msgr2=0x5617f2337010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2337010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f9062ffd700 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c0091f0 con 0x5617f2329d90 2023-10-17T01:17:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.690+0000 7f90637fe700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2337010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:17:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.691+0000 7f9042ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:17:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.691+0000 7f9062ffd700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2337590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5617f23285d0 tx=0x7f9054004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958d54d68d46ae73 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.691+0000 7f9042ffd700 1 -- 172.21.15.45:0/3745675348 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905401f050 con 0x5617f2329d90 2023-10-17T01:17:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.691+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f905c011040 con 0x5617f2329d90 2023-10-17T01:17:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.691+0000 7f9042ffd700 1 -- 172.21.15.45:0/3745675348 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f905403b560 con 0x5617f2329d90 2023-10-17T01:17:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.692+0000 7f9042ffd700 1 -- 172.21.15.45:0/3745675348 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9054028e60 con 0x5617f2329d90 2023-10-17T01:17:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.692+0000 7f907d55bb80 10 client.25751 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.692+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5617f20ce1f0 con 0x5617f2329d90 2023-10-17T01:17:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.692+0000 7f9042ffd700 1 -- 172.21.15.45:0/3745675348 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c011040 con 0x5617f2329d90 2023-10-17T01:17:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.692+0000 7f9042ffd700 1 -- 172.21.15.45:0/3745675348 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f905404e840 con 0x5617f2329d90 2023-10-17T01:17:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.693+0000 7f907d55bb80 10 client.25751 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.693+0000 7f907d55bb80 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.693+0000 7f907d55bb80 20 client.25751 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.694+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f2390ae0 0x5617f2392f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.694+0000 7f907d55bb80 4 client.25751 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:17:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.694+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617f2272ce0 con 0x5617f2390ae0 2023-10-17T01:17:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.694+0000 7f90637fe700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f2390ae0 0x5617f2392f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.696+0000 7f9042ffd700 10 client.25751 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.696+0000 7f90637fe700 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f2390ae0 0x5617f2392f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d5cb8042f2083e server_cookie=23a95d1cf99b6943 in_seq=0 out_seq=0 2023-10-17T01:17:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.709+0000 7f9042ffd700 1 -- 172.21.15.45:0/3745675348 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617f2272ce0 con 0x5617f2390ae0 2023-10-17T01:17:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.709+0000 7f9042ffd700 10 client.25751 handle_command_reply: tid=0 2023-10-17T01:17:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.709+0000 7f907d55bb80 1 client.25751 shutdown 2023-10-17T01:17:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 msgr2=0x5617f2337590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2337590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5617f23285d0 tx=0x7f9054004750 comp rx=0 tx=0).stop 2023-10-17T01:17:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 shutdown_connections 2023-10-17T01:17:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f233a650 0x5617f23358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617f2329190 0x5617f2337010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5617f2390ae0 0x5617f2392f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 --2- 172.21.15.45:0/3745675348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617f2329d90 0x5617f2337590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.710+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 >> 172.21.15.45:0/3745675348 conn(0x5617f2288070 msgr2=0x5617f232c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.711+0000 7f9042ffd700 0 client.25751 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.711+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 shutdown_connections 2023-10-17T01:17:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.711+0000 7f907d55bb80 1 -- 172.21.15.45:0/3745675348 wait complete. 2023-10-17T01:17:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.712+0000 7f907d55bb80 20 client.25751 trim_cache size 0 max 16384 2023-10-17T01:17:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:17:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:17:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:17:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:17:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f904c0a0ca0 msgr2=0x7f904c0a3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f904c0a0ca0 0x7f904c0a3160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f905400afd0 tx=0x7f905400c040 comp rx=0 tx=0).stop 2023-10-17T01:17:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9064107fc0 msgr2=0x7f90641a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9064107fc0 0x7f90641a5930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90580098c0 tx=0x7f905800bb00 comp rx=0 tx=0).stop 2023-10-17T01:17:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 shutdown_connections 2023-10-17T01:17:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f904c0a0ca0 0x7f904c0a3160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f906410d420 0x7f90641a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9064107fc0 0x7f90641a5930 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 --2- 172.21.15.45:0/3420942891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f906410a980 0x7f90641a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.713+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 >> 172.21.15.45:0/3420942891 conn(0x7f906406db80 msgr2=0x7f90641096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.714+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 shutdown_connections 2023-10-17T01:17:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:04.715+0000 7f9069bff700 1 -- 172.21.15.45:0/3420942891 wait complete. 2023-10-17T01:17:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 115.0 s, 3023.0 tps, lat 33.059 ms stddev 35.345 2023-10-17T01:17:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:05 smithi045 ceph-mon[98474]: pgmap v391: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 1.43k op/s 2023-10-17T01:17:05.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:05 smithi060 ceph-mon[134136]: pgmap v391: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 1.43k op/s 2023-10-17T01:17:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:05 smithi181 ceph-mon[134520]: pgmap v391: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 1.43k op/s 2023-10-17T01:17:07.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:07 smithi045 ceph-mon[98474]: pgmap v392: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 713 op/s 2023-10-17T01:17:07.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:07 smithi060 ceph-mon[134136]: pgmap v392: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 713 op/s 2023-10-17T01:17:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:07 smithi181 ceph-mon[134520]: pgmap v392: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 713 op/s 2023-10-17T01:17:09.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:09 smithi060 ceph-mon[134136]: pgmap v393: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 729 op/s 2023-10-17T01:17:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:09 smithi181 ceph-mon[134520]: pgmap v393: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 729 op/s 2023-10-17T01:17:10.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:09 smithi045 ceph-mon[98474]: pgmap v393: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 729 op/s 2023-10-17T01:17:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 120.0 s, 3036.4 tps, lat 32.801 ms stddev 42.289 2023-10-17T01:17:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:11 smithi060 ceph-mon[134136]: pgmap v394: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 830 op/s 2023-10-17T01:17:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:11 smithi181 ceph-mon[134520]: pgmap v394: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 830 op/s 2023-10-17T01:17:12.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:11 smithi045 ceph-mon[98474]: pgmap v394: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 830 op/s 2023-10-17T01:17:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:13 smithi060 ceph-mon[134136]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 806 op/s 2023-10-17T01:17:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:13 smithi181 ceph-mon[134520]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 806 op/s 2023-10-17T01:17:14.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:13 smithi045 ceph-mon[98474]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 806 op/s 2023-10-17T01:17:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 125.0 s, 2369.3 tps, lat 42.275 ms stddev 49.003 2023-10-17T01:17:15.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:15 smithi060 ceph-mon[134136]: pgmap v396: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 948 op/s 2023-10-17T01:17:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:15 smithi181 ceph-mon[134520]: pgmap v396: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 948 op/s 2023-10-17T01:17:16.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:15 smithi045 ceph-mon[98474]: pgmap v396: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 948 op/s 2023-10-17T01:17:16.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:16 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:17:16.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:16 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:17:17.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:16 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:17:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: pgmap v397: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 941 op/s 2023-10-17T01:17:17.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:17.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:17.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:17.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:17.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:17.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:17 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.000+0000 7fd299bff700 1 -- 172.21.15.45:0/3301428038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 msgr2=0x7fd294108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.000+0000 7fd299bff700 1 --2- 172.21.15.45:0/3301428038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 0x7fd294108090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00f530 tx=0x7fd28c00f840 comp rx=0 tx=0).stop 2023-10-17T01:17:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.001+0000 7fd299bff700 1 -- 172.21.15.45:0/3301428038 shutdown_connections 2023-10-17T01:17:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.001+0000 7fd299bff700 1 --2- 172.21.15.45:0/3301428038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 0x7fd294108090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.001+0000 7fd299bff700 1 --2- 172.21.15.45:0/3301428038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940ffc50 0x7fd294100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.001+0000 7fd299bff700 1 --2- 172.21.15.45:0/3301428038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd2941012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.001+0000 7fd299bff700 1 -- 172.21.15.45:0/3301428038 >> 172.21.15.45:0/3301428038 conn(0x7fd2940fb1a0 msgr2=0x7fd2940fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.001+0000 7fd299bff700 1 -- 172.21.15.45:0/3301428038 shutdown_connections 2023-10-17T01:17:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.002+0000 7fd299bff700 1 -- 172.21.15.45:0/3301428038 wait complete. 2023-10-17T01:17:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.003+0000 7fd299bff700 1 Processor -- start 2023-10-17T01:17:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.004+0000 7fd299bff700 1 -- start start 2023-10-17T01:17:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.005+0000 7fd299bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940ffc50 0x7fd29419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd299bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd29419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd299bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 0x7fd29419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd299bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29419ebd0 con 0x7fd2940ffc50 2023-10-17T01:17:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd299bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29419ed40 con 0x7fd294100e40 2023-10-17T01:17:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd299bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29419ef00 con 0x7fd2941017e0 2023-10-17T01:17:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd29419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd29419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43794/0 (socket says 172.21.15.45:43794) 2023-10-17T01:17:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd292ffd700 1 -- 172.21.15.45:0/3818723919 learned_addr learned my addr 172.21.15.45:0/3818723919 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.006+0000 7fd293fff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 0x7fd29419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.007+0000 7fd292ffd700 1 -- 172.21.15.45:0/3818723919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 msgr2=0x7fd29419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.007+0000 7fd292ffd700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 0x7fd29419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.007+0000 7fd292ffd700 1 -- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940ffc50 msgr2=0x7fd29419d1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-17T01:17:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.007+0000 7fd292ffd700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940ffc50 0x7fd29419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.007+0000 7fd292ffd700 1 -- 172.21.15.45:0/3818723919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009d00 con 0x7fd294100e40 2023-10-17T01:17:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.007+0000 7fd292ffd700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd29419d6f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b550 tx=0x7fd28800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.008+0000 7fd290ff9700 1 -- 172.21.15.45:0/3818723919 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288011830 con 0x7fd294100e40 2023-10-17T01:17:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.008+0000 7fd290ff9700 1 -- 172.21.15.45:0/3818723919 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd288011e50 con 0x7fd294100e40 2023-10-17T01:17:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.008+0000 7fd290ff9700 1 -- 172.21.15.45:0/3818723919 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800f2e0 con 0x7fd294100e40 2023-10-17T01:17:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.008+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29419f1e0 con 0x7fd294100e40 2023-10-17T01:17:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.008+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941a59a0 con 0x7fd294100e40 2023-10-17T01:17:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.014+0000 7fd290ff9700 1 -- 172.21.15.45:0/3818723919 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd288011990 con 0x7fd294100e40 2023-10-17T01:17:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.018+0000 7fd290ff9700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd27c0a0c20 0x7fd27c0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.018+0000 7fd290ff9700 1 -- 172.21.15.45:0/3818723919 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2880cbb10 con 0x7fd294100e40 2023-10-17T01:17:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.019+0000 7fd2ad4deb80 1 Processor -- start 2023-10-17T01:17:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.020+0000 7fd2ad4deb80 1 -- start start 2023-10-17T01:17:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.020+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.020+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6190 0x563c458f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.020+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.020+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c4584d9b0 con 0x563c458f6190 2023-10-17T01:17:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.020+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c4584b370 con 0x563c458f5830 2023-10-17T01:17:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.021+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c45846c60 con 0x563c458f6d90 2023-10-17T01:17:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.021+0000 7fd2937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.021+0000 7fd2937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43800/0 (socket says 172.21.15.45:43800) 2023-10-17T01:17:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.021+0000 7fd2937fe700 1 -- 172.21.15.45:0/1717976669 learned_addr learned my addr 172.21.15.45:0/1717976669 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.021+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd27c0a0c20 0x7fd27c0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.022+0000 7fd293fff700 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.022+0000 7fd293fff700 1 -- 172.21.15.45:0/1717976669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 msgr2=0x563c458f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.022+0000 7fd293fff700 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.022+0000 7fd293fff700 1 -- 172.21.15.45:0/1717976669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6190 msgr2=0x563c458f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.022+0000 7fd293fff700 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6190 0x563c458f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.023+0000 7fd293fff700 1 -- 172.21.15.45:0/1717976669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c45674d80 con 0x563c458f6d90 2023-10-17T01:17:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.024+0000 7fd293fff700 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458f7210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009fd0 tx=0x7fd28c0093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832d384b0ee2f8c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.024+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd27c0a0c20 0x7fd27c0a30e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd28400bfd0 tx=0x7fd28400b850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.024+0000 7fd279ffb700 1 -- 172.21.15.45:0/1717976669 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00e070 con 0x563c458f6d90 2023-10-17T01:17:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.024+0000 7fd279ffb700 1 -- 172.21.15.45:0/1717976669 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd28c0096a0 con 0x563c458f6d90 2023-10-17T01:17:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.025+0000 7fd279ffb700 1 -- 172.21.15.45:0/1717976669 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0176d0 con 0x563c458f6d90 2023-10-17T01:17:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.026+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/1717976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 msgr2=0x563c458f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.026+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458f7210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009fd0 tx=0x7fd28c0093a0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.027+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/1717976669 shutdown_connections 2023-10-17T01:17:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.027+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458f7210 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.027+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.027+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/1717976669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6190 0x563c458f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.027+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/1717976669 >> 172.21.15.45:0/1717976669 conn(0x563c45855070 msgr2=0x563c458f02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.028+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/1717976669 shutdown_connections 2023-10-17T01:17:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.028+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/1717976669 wait complete. 2023-10-17T01:17:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.029+0000 7fd2ad4deb80 1 -- start start 2023-10-17T01:17:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.029+0000 7fd2ad4deb80 1 Processor -- start 2023-10-17T01:17:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6190 0x563c458ff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6d90 0x563c459069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c457e34b0 con 0x563c458f6d90 2023-10-17T01:17:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c457e3150 con 0x563c458f5830 2023-10-17T01:17:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c456ca8c0 con 0x563c458f6190 2023-10-17T01:17:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2ad4deb80 10 client.0 _ll_register_callbacks cb 0x563c45730a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6190 0x563c458ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458ff430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43816/0 (socket says 172.21.15.45:43816) 2023-10-17T01:17:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6190 0x563c458ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59092/0 (socket says 172.21.15.45:59092) 2023-10-17T01:17:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd2937fe700 1 -- 172.21.15.45:0/3309391465 learned_addr learned my addr 172.21.15.45:0/3309391465 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.030+0000 7fd293fff700 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6d90 0x563c459069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd2937fe700 1 -- 172.21.15.45:0/3309391465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6190 msgr2=0x563c458ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6190 0x563c458ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd2937fe700 1 -- 172.21.15.45:0/3309391465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6d90 msgr2=0x563c459069b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6d90 0x563c459069b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd2937fe700 1 -- 172.21.15.45:0/3309391465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941089e0 con 0x563c458f5830 2023-10-17T01:17:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458ff430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd284009340 tx=0x7fd28400b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df948cea2510311d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.031+0000 7fd272ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:17:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.032+0000 7fd272ffd700 1 -- 172.21.15.45:0/3309391465 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28400ba00 con 0x563c458f5830 2023-10-17T01:17:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.032+0000 7fd272ffd700 1 -- 172.21.15.45:0/3309391465 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd28401b460 con 0x563c458f5830 2023-10-17T01:17:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.033+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c45871cb0 con 0x563c458f5830 2023-10-17T01:17:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.033+0000 7fd272ffd700 1 -- 172.21.15.45:0/3309391465 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28401b620 con 0x563c458f5830 2023-10-17T01:17:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.033+0000 7fd2ad4deb80 10 client.25760 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.033+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563c4565dc00 con 0x563c458f5830 2023-10-17T01:17:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd272ffd700 1 -- 172.21.15.45:0/3309391465 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c45871cb0 con 0x563c458f5830 2023-10-17T01:17:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd272ffd700 1 -- 172.21.15.45:0/3309391465 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd284023890 con 0x563c458f5830 2023-10-17T01:17:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd2ad4deb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd2ad4deb80 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd2ad4deb80 20 client.25760 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4590b3c0 0x563c4590d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd2ad4deb80 4 client.25760 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.034+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c458c4cd0 con 0x563c4590b3c0 2023-10-17T01:17:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.035+0000 7fd292ffd700 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4590b3c0 0x563c4590d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.035+0000 7fd272ffd700 10 client.25760 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.035+0000 7fd292ffd700 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4590b3c0 0x563c4590d7e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e46d5a97fd1ed64 server_cookie=f0844888b75e99cd in_seq=0 out_seq=0 2023-10-17T01:17:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.036+0000 7fd272ffd700 1 -- 172.21.15.45:0/3309391465 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x563c458c4cd0 con 0x563c4590b3c0 2023-10-17T01:17:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.036+0000 7fd272ffd700 10 client.25760 handle_command_reply: tid=0 2023-10-17T01:17:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.036+0000 7fd2ad4deb80 1 client.25760 shutdown 2023-10-17T01:17:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 msgr2=0x563c458ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458ff430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd284009340 tx=0x7fd28400b720 comp rx=0 tx=0).stop 2023-10-17T01:17:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 shutdown_connections 2023-10-17T01:17:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f6d90 0x563c459069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4590b3c0 0x563c4590d7e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f5830 0x563c458ff430 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3309391465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6190 0x563c458ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 >> 172.21.15.45:0/3309391465 conn(0x563c45855070 msgr2=0x563c458efa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd272ffd700 0 client.25760 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.037+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 shutdown_connections 2023-10-17T01:17:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.038+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3309391465 wait complete. 2023-10-17T01:17:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.038+0000 7fd2ad4deb80 20 client.25760 trim_cache size 0 max 16384 2023-10-17T01:17:18.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: pgmap v397: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 941 op/s 2023-10-17T01:17:18.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:17 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.076+0000 7fd2ad4deb80 1 Processor -- start 2023-10-17T01:17:18.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: pgmap v397: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 941 op/s 2023-10-17T01:17:18.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:17 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.077+0000 7fd2ad4deb80 1 -- start start 2023-10-17T01:17:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.077+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c458fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.078+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c458f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.078+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.078+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c458ea9e0 con 0x563c458f5830 2023-10-17T01:17:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.078+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c458ed010 con 0x563c458f6190 2023-10-17T01:17:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.078+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c458e8280 con 0x563c458f6d90 2023-10-17T01:17:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.078+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c458f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.079+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c458f9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43826/0 (socket says 172.21.15.45:43826) 2023-10-17T01:17:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.079+0000 7fd292ffd700 1 -- 172.21.15.45:0/3489155860 learned_addr learned my addr 172.21.15.45:0/3489155860 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.079+0000 7fd2937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c458fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.079+0000 7fd293fff700 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.080+0000 7fd2937fe700 1 -- 172.21.15.45:0/3489155860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 msgr2=0x563c458fa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.080+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458fa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.080+0000 7fd2937fe700 1 -- 172.21.15.45:0/3489155860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 msgr2=0x563c458f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.080+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c458f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.080+0000 7fd2937fe700 1 -- 172.21.15.45:0/3489155860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c4569b1f0 con 0x563c458f5830 2023-10-17T01:17:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.080+0000 7fd292ffd700 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c458f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.081+0000 7fd293fff700 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:17:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.082+0000 7fd2937fe700 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c458fe030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd284023bf0 tx=0x7fd28400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca3860cf4ddc5c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.083+0000 7fd27a7fc700 1 -- 172.21.15.45:0/3489155860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284018e50 con 0x563c458f5830 2023-10-17T01:17:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.083+0000 7fd27a7fc700 1 -- 172.21.15.45:0/3489155860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd284011070 con 0x563c458f5830 2023-10-17T01:17:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.083+0000 7fd27a7fc700 1 -- 172.21.15.45:0/3489155860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28401aa10 con 0x563c458f5830 2023-10-17T01:17:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.084+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3489155860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 msgr2=0x563c458fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.084+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c458fe030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd284023bf0 tx=0x7fd28400b920 comp rx=0 tx=0).stop 2023-10-17T01:17:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.085+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3489155860 shutdown_connections 2023-10-17T01:17:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.085+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c458fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.085+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c458fe030 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.085+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/3489155860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c458f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.085+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3489155860 >> 172.21.15.45:0/3489155860 conn(0x563c45855070 msgr2=0x563c4590d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.086+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3489155860 shutdown_connections 2023-10-17T01:17:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.086+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/3489155860 wait complete. 2023-10-17T01:17:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.086+0000 7fd2ad4deb80 1 -- start start 2023-10-17T01:17:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.087+0000 7fd2ad4deb80 1 Processor -- start 2023-10-17T01:17:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c45956d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c45959270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c4595b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c4595bd00 con 0x563c458f5830 2023-10-17T01:17:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c4595be70 con 0x563c458f6190 2023-10-17T01:17:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c4595c030 con 0x563c458f6d90 2023-10-17T01:17:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.088+0000 7fd2ad4deb80 10 client.0 _ll_register_callbacks cb 0x7fd284005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.089+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c45959270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.089+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c45959270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43836/0 (socket says 172.21.15.45:43836) 2023-10-17T01:17:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.089+0000 7fd292ffd700 1 -- 172.21.15.45:0/489486741 learned_addr learned my addr 172.21.15.45:0/489486741 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd2937fe700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c45956d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd293fff700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c4595b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd292ffd700 1 -- 172.21.15.45:0/489486741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 msgr2=0x563c4595b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd292ffd700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c4595b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd292ffd700 1 -- 172.21.15.45:0/489486741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 msgr2=0x563c45956d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd292ffd700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c45956d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd292ffd700 1 -- 172.21.15.45:0/489486741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941a59a0 con 0x563c458f6190 2023-10-17T01:17:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.090+0000 7fd2937fe700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c45956d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:17:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd292ffd700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c45959270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd28800bfd0 tx=0x7fd28800be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7ea55da93cb232 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd272ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:17:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd272ffd700 1 -- 172.21.15.45:0/489486741 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880974d0 con 0x563c458f6190 2023-10-17T01:17:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd272ffd700 1 -- 172.21.15.45:0/489486741 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2880c96f0 con 0x563c458f6190 2023-10-17T01:17:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd28c0096a0 con 0x563c458f6190 2023-10-17T01:17:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd272ffd700 1 -- 172.21.15.45:0/489486741 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd288098620 con 0x563c458f6190 2023-10-17T01:17:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd2ad4deb80 10 client.25766 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563c4565dc00 con 0x563c458f6190 2023-10-17T01:17:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.091+0000 7fd272ffd700 1 -- 172.21.15.45:0/489486741 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd288080020 con 0x563c458f6190 2023-10-17T01:17:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd272ffd700 1 -- 172.21.15.45:0/489486741 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd288098090 con 0x563c458f6190 2023-10-17T01:17:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2ad4deb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2ad4deb80 10 client.25766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2ad4deb80 20 client.25766 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4595da50 0x563c4595fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2ad4deb80 4 client.25766 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:17:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c4583fce0 con 0x563c4595da50 2023-10-17T01:17:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2937fe700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4595da50 0x563c4595fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd272ffd700 10 client.25766 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.092+0000 7fd2937fe700 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4595da50 0x563c4595fe70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf7f06a89b5b41b server_cookie=141f40b24f246164 in_seq=0 out_seq=0 2023-10-17T01:17:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.106+0000 7fd272ffd700 1 -- 172.21.15.45:0/489486741 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c4583fce0 con 0x563c4595da50 2023-10-17T01:17:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.106+0000 7fd272ffd700 10 client.25766 handle_command_reply: tid=0 2023-10-17T01:17:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.106+0000 7fd2ad4deb80 1 client.25766 shutdown 2023-10-17T01:17:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 msgr2=0x563c45959270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c45959270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd28800bfd0 tx=0x7fd28800be50 comp rx=0 tx=0).stop 2023-10-17T01:17:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 shutdown_connections 2023-10-17T01:17:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c458f6d90 0x563c4595b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563c458f5830 0x563c45956d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563c4595da50 0x563c4595fe70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 --2- 172.21.15.45:0/489486741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563c458f6190 0x563c45959270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.107+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 >> 172.21.15.45:0/489486741 conn(0x563c45855070 msgr2=0x563c458fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.108+0000 7fd272ffd700 0 client.25766 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.108+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 shutdown_connections 2023-10-17T01:17:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.108+0000 7fd2ad4deb80 1 -- 172.21.15.45:0/489486741 wait complete. 2023-10-17T01:17:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.108+0000 7fd2ad4deb80 20 client.25766 trim_cache size 0 max 16384 2023-10-17T01:17:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:17:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:17:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:17:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:17:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.110+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd27c0a0c20 msgr2=0x7fd27c0a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.110+0000 7fd299bff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd27c0a0c20 0x7fd27c0a30e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd28400bfd0 tx=0x7fd28400b850 comp rx=0 tx=0).stop 2023-10-17T01:17:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.110+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 msgr2=0x7fd29419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd29419d6f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b550 tx=0x7fd28800b860 comp rx=0 tx=0).stop 2023-10-17T01:17:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 shutdown_connections 2023-10-17T01:17:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd27c0a0c20 0x7fd27c0a30e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2941017e0 0x7fd29419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940ffc50 0x7fd29419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 --2- 172.21.15.45:0/3818723919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd294100e40 0x7fd29419d6f0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b550 tx=0x7fd28800b860 comp rx=0 tx=0).stop 2023-10-17T01:17:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 >> 172.21.15.45:0/3818723919 conn(0x7fd2940fb1a0 msgr2=0x7fd294104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 shutdown_connections 2023-10-17T01:17:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:18.111+0000 7fd299bff700 1 -- 172.21.15.45:0/3818723919 wait complete. 2023-10-17T01:17:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:18 smithi181 ceph-mon[134520]: pgmap v398: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-10-17T01:17:19.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:18 smithi045 ceph-mon[98474]: pgmap v398: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-10-17T01:17:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:18 smithi060 ceph-mon[134136]: pgmap v398: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-10-17T01:17:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:19 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:17:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:19 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:17:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:19 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:20.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:19 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:17:20.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:19 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:17:20.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:19 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:20.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:19 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:17:20.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:19 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:17:20.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:19 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:17:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 130.0 s, 2700.3 tps, lat 37.052 ms stddev 42.955 2023-10-17T01:17:20.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:20 smithi181 ceph-mon[134520]: pgmap v399: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-17T01:17:21.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:20 smithi045 ceph-mon[98474]: pgmap v399: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-17T01:17:21.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:20 smithi060 ceph-mon[134136]: pgmap v399: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-17T01:17:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:23 smithi181 ceph-mon[134520]: pgmap v400: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 971 op/s 2023-10-17T01:17:24.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:23 smithi045 ceph-mon[98474]: pgmap v400: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 971 op/s 2023-10-17T01:17:24.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:23 smithi060 ceph-mon[134136]: pgmap v400: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 971 op/s 2023-10-17T01:17:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 135.0 s, 2184.5 tps, lat 45.911 ms stddev 92.882 2023-10-17T01:17:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:25 smithi045 ceph-mon[98474]: pgmap v401: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 MiB/s wr, 1.50k op/s 2023-10-17T01:17:25.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:25 smithi060 ceph-mon[134136]: pgmap v401: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 MiB/s wr, 1.50k op/s 2023-10-17T01:17:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:25 smithi181 ceph-mon[134520]: pgmap v401: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 MiB/s wr, 1.50k op/s 2023-10-17T01:17:27.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:27 smithi045 ceph-mon[98474]: pgmap v402: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 1.64k op/s 2023-10-17T01:17:27.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:27 smithi060 ceph-mon[134136]: pgmap v402: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 1.64k op/s 2023-10-17T01:17:27.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:27 smithi181 ceph-mon[134520]: pgmap v402: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 37 MiB/s wr, 1.64k op/s 2023-10-17T01:17:28.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:17:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 -- 172.21.15.45:0/782230363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 msgr2=0x7f02ac102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 --2- 172.21.15.45:0/782230363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac102850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f029c00f4c0 tx=0x7f029c00f7d0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 -- 172.21.15.45:0/782230363 shutdown_connections 2023-10-17T01:17:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 --2- 172.21.15.45:0/782230363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac1058d0 0x7f02ac107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 --2- 172.21.15.45:0/782230363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac102850 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 --2- 172.21.15.45:0/782230363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 -- 172.21.15.45:0/782230363 >> 172.21.15.45:0/782230363 conn(0x7f02ac0fa4e0 msgr2=0x7f02ac0fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 -- 172.21.15.45:0/782230363 shutdown_connections 2023-10-17T01:17:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.965+0000 7f02b15ff700 1 -- 172.21.15.45:0/782230363 wait complete. 2023-10-17T01:17:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 Processor -- start 2023-10-17T01:17:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 -- start start 2023-10-17T01:17:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac1058d0 0x7f02ac1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1a4c70 con 0x7f02ac1058d0 2023-10-17T01:17:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1a4db0 con 0x7f02ac102e20 2023-10-17T01:17:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.967+0000 7f02b15ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1a4f40 con 0x7f02ac100460 2023-10-17T01:17:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56680/0 (socket says 172.21.15.45:56680) 2023-10-17T01:17:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02aaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02a3fff700 1 -- 172.21.15.45:0/4052040270 learned_addr learned my addr 172.21.15.45:0/4052040270 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac1058d0 0x7f02ac1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02aaffd700 1 -- 172.21.15.45:0/4052040270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 msgr2=0x7f02ac19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02aaffd700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02aaffd700 1 -- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac1058d0 msgr2=0x7f02ac1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02aaffd700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac1058d0 0x7f02ac1a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.968+0000 7f02aaffd700 1 -- 172.21.15.45:0/4052040270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009520 con 0x7f02ac100460 2023-10-17T01:17:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.969+0000 7f02a3fff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.969+0000 7f02aaffd700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac19d0f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f029c000f80 tx=0x7f029c016d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.969+0000 7f02a8ff9700 1 -- 172.21.15.45:0/4052040270 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c012070 con 0x7f02ac100460 2023-10-17T01:17:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.969+0000 7f02a8ff9700 1 -- 172.21.15.45:0/4052040270 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029c013040 con 0x7f02ac100460 2023-10-17T01:17:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.969+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c00f170 con 0x7f02ac100460 2023-10-17T01:17:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.970+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac1a53a0 con 0x7f02ac100460 2023-10-17T01:17:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.970+0000 7f02a8ff9700 1 -- 172.21.15.45:0/4052040270 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00d7b0 con 0x7f02ac100460 2023-10-17T01:17:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.973+0000 7f02a8ff9700 1 -- 172.21.15.45:0/4052040270 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f029c00b220 con 0x7f02ac100460 2023-10-17T01:17:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.975+0000 7f02a8ff9700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02980a0bf0 0x7f02980a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.975+0000 7f02a8ff9700 1 -- 172.21.15.45:0/4052040270 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f029c0cc9b0 con 0x7f02ac100460 2023-10-17T01:17:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.976+0000 7f02a3fff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02980a0bf0 0x7f02980a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.977+0000 7f02a3fff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02980a0bf0 0x7f02980a30b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0294009fd0 tx=0x7f0294009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 Processor -- start 2023-10-17T01:17:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 -- start start 2023-10-17T01:17:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5ce500 0x561bff5ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff525a10 con 0x561bff5ce500 2023-10-17T01:17:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5233d0 con 0x561bff5cf2b0 2023-10-17T01:17:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.980+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff51ecc0 con 0x561bff5cdba0 2023-10-17T01:17:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.981+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.981+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56690/0 (socket says 172.21.15.45:56690) 2023-10-17T01:17:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.981+0000 7f02ab7fe700 1 -- 172.21.15.45:0/534648805 learned_addr learned my addr 172.21.15.45:0/534648805 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.981+0000 7f02a3fff700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5ce500 0x561bff5ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.981+0000 7f02aaffd700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02a3fff700 1 -- 172.21.15.45:0/534648805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 msgr2=0x561bff5cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:28.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02a3fff700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02a3fff700 1 -- 172.21.15.45:0/534648805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 msgr2=0x561bff5cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:28.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02a3fff700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02a3fff700 1 -- 172.21.15.45:0/534648805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bff34cde0 con 0x561bff5ce500 2023-10-17T01:17:28.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:28.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.983+0000 7f02aaffd700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:28.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.985+0000 7f02a3fff700 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5ce500 0x561bff5ced20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0294011e50 tx=0x7f0294014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f438a43b146d76b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:28.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.985+0000 7f02a17fa700 1 -- 172.21.15.45:0/534648805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029401b630 con 0x561bff5ce500 2023-10-17T01:17:28.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.985+0000 7f02a17fa700 1 -- 172.21.15.45:0/534648805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029401bc50 con 0x561bff5ce500 2023-10-17T01:17:28.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.985+0000 7f02a17fa700 1 -- 172.21.15.45:0/534648805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029401ccd0 con 0x561bff5ce500 2023-10-17T01:17:28.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 -- 172.21.15.45:0/534648805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5ce500 msgr2=0x561bff5ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:28.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5ce500 0x561bff5ced20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0294011e50 tx=0x7f0294014000 comp rx=0 tx=0).stop 2023-10-17T01:17:28.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 -- 172.21.15.45:0/534648805 shutdown_connections 2023-10-17T01:17:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/534648805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5ce500 0x561bff5ced20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.988+0000 7f02c4e42b80 1 -- 172.21.15.45:0/534648805 >> 172.21.15.45:0/534648805 conn(0x561bff52d0d0 msgr2=0x561bff5c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:28.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.989+0000 7f02c4e42b80 1 -- 172.21.15.45:0/534648805 shutdown_connections 2023-10-17T01:17:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.989+0000 7f02c4e42b80 1 -- 172.21.15.45:0/534648805 wait complete. 2023-10-17T01:17:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.989+0000 7f02c4e42b80 1 -- start start 2023-10-17T01:17:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 Processor -- start 2023-10-17T01:17:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cdba0 0x561bff5d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 0x561bff5d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5dd0e0 0x561bff5df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff4bb510 con 0x561bff5dd0e0 2023-10-17T01:17:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff4bb1b0 con 0x561bff5cdba0 2023-10-17T01:17:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff3a2920 con 0x561bff5cf2b0 2023-10-17T01:17:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.990+0000 7f02c4e42b80 10 client.0 _ll_register_callbacks cb 0x561bff408a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:28.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 0x561bff5d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 0x561bff5d80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38970/0 (socket says 172.21.15.45:38970) 2023-10-17T01:17:28.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 -- 172.21.15.45:0/636510997 learned_addr learned my addr 172.21.15.45:0/636510997 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5dd0e0 0x561bff5df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02aaffd700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cdba0 0x561bff5d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 -- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cdba0 msgr2=0x561bff5d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cdba0 0x561bff5d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 -- 172.21.15.45:0/636510997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5dd0e0 msgr2=0x561bff5df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5dd0e0 0x561bff5df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.991+0000 7f02a3fff700 1 -- 172.21.15.45:0/636510997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac1a53a0 con 0x561bff5cf2b0 2023-10-17T01:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.992+0000 7f02a3fff700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 0x561bff5d80b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0294009de0 tx=0x7f029401ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b202e22aef3093 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.992+0000 7f02927fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.992+0000 7f02927fc700 1 -- 172.21.15.45:0/636510997 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294005450 con 0x561bff5cf2b0 2023-10-17T01:17:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.992+0000 7f02927fc700 1 -- 172.21.15.45:0/636510997 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f029401bdc0 con 0x561bff5cf2b0 2023-10-17T01:17:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.992+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bff549d10 con 0x561bff5cf2b0 2023-10-17T01:17:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.993+0000 7f02927fc700 1 -- 172.21.15.45:0/636510997 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02940055b0 con 0x561bff5cf2b0 2023-10-17T01:17:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.993+0000 7f02c4e42b80 10 client.25756 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.993+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f02a4009820 con 0x561bff5cf2b0 2023-10-17T01:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.994+0000 7f02927fc700 1 -- 172.21.15.45:0/636510997 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bff549d10 con 0x561bff5cf2b0 2023-10-17T01:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.994+0000 7f02927fc700 1 -- 172.21.15.45:0/636510997 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0294004a30 con 0x561bff5cf2b0 2023-10-17T01:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02c4e42b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02c4e42b80 10 client.25756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02c4e42b80 20 client.25756 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff5e3c50 0x561bff5e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02c4e42b80 4 client.25756 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bff59cd30 con 0x561bff5e3c50 2023-10-17T01:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.995+0000 7f02aaffd700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff5e3c50 0x561bff5e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.996+0000 7f02aaffd700 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff5e3c50 0x561bff5e6070 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ec3fe7b101fe6e server_cookie=7d1f6c183f67912c in_seq=0 out_seq=0 2023-10-17T01:17:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.996+0000 7f02927fc700 10 client.25756 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.997+0000 7f02927fc700 1 -- 172.21.15.45:0/636510997 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x561bff59cd30 con 0x561bff5e3c50 2023-10-17T01:17:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.997+0000 7f02927fc700 10 client.25756 handle_command_reply: tid=0 2023-10-17T01:17:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.997+0000 7f02c4e42b80 1 client.25756 shutdown 2023-10-17T01:17:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.998+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 msgr2=0x561bff5d80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.998+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 0x561bff5d80b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0294009de0 tx=0x7f029401ce40 comp rx=0 tx=0).stop 2023-10-17T01:17:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 shutdown_connections 2023-10-17T01:17:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5dd0e0 0x561bff5df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff5e3c50 0x561bff5e6070 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cdba0 0x561bff5d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/636510997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cf2b0 0x561bff5d80b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02927fc700 0 client.25756 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 >> 172.21.15.45:0/636510997 conn(0x561bff52d0d0 msgr2=0x561bff5c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 shutdown_connections 2023-10-17T01:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:28.999+0000 7f02c4e42b80 1 -- 172.21.15.45:0/636510997 wait complete. 2023-10-17T01:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.000+0000 7f02c4e42b80 20 client.25756 trim_cache size 0 max 16384 2023-10-17T01:17:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.035+0000 7f02c4e42b80 1 Processor -- start 2023-10-17T01:17:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.036+0000 7f02c4e42b80 1 -- start start 2023-10-17T01:17:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.036+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cdba0 0x561bff5c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5dd0e0 0x561bff5d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5d8d60 con 0x561bff5cdba0 2023-10-17T01:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5d5ee0 con 0x561bff5cf2b0 2023-10-17T01:17:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5e37e0 con 0x561bff5dd0e0 2023-10-17T01:17:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5d7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56706/0 (socket says 172.21.15.45:56706) 2023-10-17T01:17:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02a3fff700 1 -- 172.21.15.45:0/3558827692 learned_addr learned my addr 172.21.15.45:0/3558827692 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.037+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5dd0e0 0x561bff5d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a3fff700 1 -- 172.21.15.45:0/3558827692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5dd0e0 msgr2=0x561bff5d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a3fff700 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5dd0e0 0x561bff5d59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a3fff700 1 -- 172.21.15.45:0/3558827692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cdba0 msgr2=0x561bff5c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a3fff700 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cdba0 0x561bff5c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a3fff700 1 -- 172.21.15.45:0/3558827692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294017050 con 0x561bff5cf2b0 2023-10-17T01:17:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a3fff700 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5d7220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0294016040 tx=0x7f0294005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1831b86a6ff2fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a1ffb700 1 -- 172.21.15.45:0/3558827692 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029401c610 con 0x561bff5cf2b0 2023-10-17T01:17:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a1ffb700 1 -- 172.21.15.45:0/3558827692 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0294025440 con 0x561bff5cf2b0 2023-10-17T01:17:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.038+0000 7f02a1ffb700 1 -- 172.21.15.45:0/3558827692 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294004b70 con 0x561bff5cf2b0 2023-10-17T01:17:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3558827692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 msgr2=0x561bff5d7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5d7220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0294016040 tx=0x7f0294005d00 comp rx=0 tx=0).stop 2023-10-17T01:17:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3558827692 shutdown_connections 2023-10-17T01:17:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5dd0e0 0x561bff5d59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cdba0 0x561bff5c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3558827692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5cf2b0 0x561bff5d7220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3558827692 >> 172.21.15.45:0/3558827692 conn(0x561bff52d0d0 msgr2=0x561bff5c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3558827692 shutdown_connections 2023-10-17T01:17:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.039+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3558827692 wait complete. 2023-10-17T01:17:29.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.040+0000 7f02c4e42b80 1 -- start start 2023-10-17T01:17:29.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.040+0000 7f02c4e42b80 1 Processor -- start 2023-10-17T01:17:29.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.040+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cf2b0 0x561bff5f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02c4e42b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 0x561bff5f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5f2540 con 0x561bff5cf2b0 2023-10-17T01:17:29.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5f26b0 con 0x561bff5dd0e0 2023-10-17T01:17:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02c4e42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff5f2820 con 0x561bff5cdba0 2023-10-17T01:17:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02c4e42b80 10 client.0 _ll_register_callbacks cb 0x7f02a4009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.041+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 0x561bff5f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 0x561bff5f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56712/0 (socket says 172.21.15.45:56712) 2023-10-17T01:17:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 -- 172.21.15.45:0/3400434073 learned_addr learned my addr 172.21.15.45:0/3400434073 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02a3fff700 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cf2b0 0x561bff5f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 -- 172.21.15.45:0/3400434073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 msgr2=0x561bff5f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 -- 172.21.15.45:0/3400434073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cf2b0 msgr2=0x561bff5f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cf2b0 0x561bff5f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 -- 172.21.15.45:0/3400434073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a4009820 con 0x561bff5dd0e0 2023-10-17T01:17:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02ab7fe700 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 0x561bff5f2000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x561bff5c3320 tx=0x7f02a4013a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe97ab8534dab0f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.042+0000 7f02927fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:17:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.043+0000 7f02927fc700 1 -- 172.21.15.45:0/3400434073 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4018070 con 0x561bff5dd0e0 2023-10-17T01:17:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.043+0000 7f02927fc700 1 -- 172.21.15.45:0/3400434073 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02a4011650 con 0x561bff5dd0e0 2023-10-17T01:17:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.043+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f029401bc50 con 0x561bff5dd0e0 2023-10-17T01:17:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.043+0000 7f02927fc700 1 -- 172.21.15.45:0/3400434073 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02a4004830 con 0x561bff5dd0e0 2023-10-17T01:17:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.043+0000 7f02c4e42b80 10 client.25784 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.043+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561bff335c60 con 0x561bff5dd0e0 2023-10-17T01:17:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.044+0000 7f02927fc700 1 -- 172.21.15.45:0/3400434073 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029401bc50 con 0x561bff5dd0e0 2023-10-17T01:17:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.044+0000 7f02927fc700 1 -- 172.21.15.45:0/3400434073 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f02a4005020 con 0x561bff5dd0e0 2023-10-17T01:17:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02c4e42b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02c4e42b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02c4e42b80 20 client.25784 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff647530 0x561bff649950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02c4e42b80 4 client.25784 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:17:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561bff517d40 con 0x561bff647530 2023-10-17T01:17:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.045+0000 7f02aaffd700 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff647530 0x561bff649950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.046+0000 7f02aaffd700 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff647530 0x561bff649950 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130ce6c49bd1977e server_cookie=321ce3d944f2e013 in_seq=0 out_seq=0 2023-10-17T01:17:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.046+0000 7f02927fc700 10 client.25784 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.046+0000 7f02927fc700 1 -- 172.21.15.45:0/3400434073 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561bff517d40 con 0x561bff647530 2023-10-17T01:17:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.046+0000 7f02927fc700 10 client.25784 handle_command_reply: tid=0 2023-10-17T01:17:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.046+0000 7f02c4e42b80 1 client.25784 shutdown 2023-10-17T01:17:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 msgr2=0x561bff5f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 0x561bff5f2000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x561bff5c3320 tx=0x7f02a4013a70 comp rx=0 tx=0).stop 2023-10-17T01:17:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 shutdown_connections 2023-10-17T01:17:29.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bff647530 0x561bff649950 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bff5dd0e0 0x561bff5f2000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bff5cdba0 0x561bff5f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 --2- 172.21.15.45:0/3400434073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bff5cf2b0 0x561bff5f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 >> 172.21.15.45:0/3400434073 conn(0x561bff52d0d0 msgr2=0x561bff5ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.047+0000 7f02927fc700 0 client.25784 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.048+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 shutdown_connections 2023-10-17T01:17:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.048+0000 7f02c4e42b80 1 -- 172.21.15.45:0/3400434073 wait complete. 2023-10-17T01:17:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.048+0000 7f02c4e42b80 20 client.25784 trim_cache size 0 max 16384 2023-10-17T01:17:29.061 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:17:29.062 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:17:29.062 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:17:29.062 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:17:29.063 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:17:29.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:17:29.064 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:17:29.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:17:29.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:17:29.064 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:17:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02980a0bf0 msgr2=0x7f02980a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02980a0bf0 0x7f02980a30b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0294009fd0 tx=0x7f0294009260 comp rx=0 tx=0).stop 2023-10-17T01:17:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 msgr2=0x7f02ac19d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac19d0f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f029c000f80 tx=0x7f029c016d20 comp rx=0 tx=0).stop 2023-10-17T01:17:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 shutdown_connections 2023-10-17T01:17:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f02980a0bf0 0x7f02980a30b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02ac1058d0 0x7f02ac1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac100460 0x7f02ac19d0f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 --2- 172.21.15.45:0/4052040270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02ac102e20 0x7f02ac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.049+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 >> 172.21.15.45:0/4052040270 conn(0x7f02ac0fa4e0 msgr2=0x7f02ac1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.050+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 shutdown_connections 2023-10-17T01:17:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:29.050+0000 7f02b15ff700 1 -- 172.21.15.45:0/4052040270 wait complete. 2023-10-17T01:17:29.084 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:17:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:29 smithi045 ceph-mon[98474]: pgmap v403: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.75k op/s 2023-10-17T01:17:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:29 smithi060 ceph-mon[134136]: pgmap v403: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.75k op/s 2023-10-17T01:17:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:29 smithi181 ceph-mon[134520]: pgmap v403: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 1.75k op/s 2023-10-17T01:17:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 140.0 s, 2900.0 tps, lat 34.440 ms stddev 38.932 2023-10-17T01:17:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 -- 172.21.15.45:0/660113029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40fef90 msgr2=0x7ff3a40ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/660113029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40fef90 0x7ff3a40ff3b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff394009990 tx=0x7ff394009ca0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 -- 172.21.15.45:0/660113029 shutdown_connections 2023-10-17T01:17:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/660113029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a4100b20 0x7ff3a41073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/660113029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40fef90 0x7ff3a40ff3b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/660113029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a4100180 0x7ff3a41005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 -- 172.21.15.45:0/660113029 >> 172.21.15.45:0/660113029 conn(0x7ff3a40fa500 msgr2=0x7ff3a40fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 -- 172.21.15.45:0/660113029 shutdown_connections 2023-10-17T01:17:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.391+0000 7ff3a99ff700 1 -- 172.21.15.45:0/660113029 wait complete. 2023-10-17T01:17:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 Processor -- start 2023-10-17T01:17:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 -- start start 2023-10-17T01:17:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 0x7ff3a419d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a4100180 0x7ff3a419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a4100b20 0x7ff3a419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a419eba0 con 0x7ff3a4100b20 2023-10-17T01:17:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a419ed10 con 0x7ff3a40fef90 2023-10-17T01:17:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.393+0000 7ff3a99ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a419eed0 con 0x7ff3a4100180 2023-10-17T01:17:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 0x7ff3a419d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 0x7ff3a419d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56728/0 (socket says 172.21.15.45:56728) 2023-10-17T01:17:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a4100b20 0x7ff3a419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/825084267 learned_addr learned my addr 172.21.15.45:0/825084267 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a4100180 0x7ff3a419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/825084267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a4100180 msgr2=0x7ff3a419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a4100180 0x7ff3a419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a4100b20 msgr2=0x7ff3a419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.394+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a4100b20 0x7ff3a419e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.395+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/825084267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940095f0 con 0x7ff3a40fef90 2023-10-17T01:17:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.395+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 0x7ff3a419d180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff394004710 tx=0x7ff394009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.395+0000 7ff3a89fd700 1 -- 172.21.15.45:0/825084267 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39401d070 con 0x7ff3a40fef90 2023-10-17T01:17:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.395+0000 7ff3a89fd700 1 -- 172.21.15.45:0/825084267 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff394021410 con 0x7ff3a40fef90 2023-10-17T01:17:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.395+0000 7ff3a89fd700 1 -- 172.21.15.45:0/825084267 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394017650 con 0x7ff3a40fef90 2023-10-17T01:17:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.395+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a419f150 con 0x7ff3a40fef90 2023-10-17T01:17:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.396+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a41a5790 con 0x7ff3a40fef90 2023-10-17T01:17:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.403+0000 7ff3a89fd700 1 -- 172.21.15.45:0/825084267 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff39400fb00 con 0x7ff3a40fef90 2023-10-17T01:17:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.405+0000 7ff3a89fd700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff38c0a0b20 0x7ff38c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.405+0000 7ff3a89fd700 1 -- 172.21.15.45:0/825084267 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff3940cd550 con 0x7ff3a40fef90 2023-10-17T01:17:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.406+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff38c0a0b20 0x7ff38c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.406+0000 7ff3bd24db80 1 Processor -- start 2023-10-17T01:17:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.407+0000 7ff3bd24db80 1 -- start start 2023-10-17T01:17:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff38c0a0b20 0x7ff38c0a2fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff39800a850 tx=0x7ff39800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab3976bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3976d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab396b29b0 con 0x55ab3976cda0 2023-10-17T01:17:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab396b0370 con 0x55ab3976b840 2023-10-17T01:17:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab396abc60 con 0x55ab3976c1a0 2023-10-17T01:17:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab3976bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3a27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab3976bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56744/0 (socket says 172.21.15.45:56744) 2023-10-17T01:17:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/2884959704 learned_addr learned my addr 172.21.15.45:0/2884959704 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.408+0000 7ff3a27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:39004/0 (socket says 172.21.15.45:39004) 2023-10-17T01:17:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3976d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/2884959704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 msgr2=0x55ab3976c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/2884959704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 msgr2=0x55ab3976d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3976d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/2884959704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab394d9d80 con 0x55ab3976b840 2023-10-17T01:17:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.409+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab3976bc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3940a2710 tx=0x7ff39409a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ed15c2d4abcbee server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.410+0000 7ff389ffb700 1 -- 172.21.15.45:0/2884959704 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39409b360 con 0x55ab3976b840 2023-10-17T01:17:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.410+0000 7ff389ffb700 1 -- 172.21.15.45:0/2884959704 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff39409ac60 con 0x55ab3976b840 2023-10-17T01:17:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.410+0000 7ff389ffb700 1 -- 172.21.15.45:0/2884959704 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39409aec0 con 0x55ab3976b840 2023-10-17T01:17:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.410+0000 7ff3bd24db80 1 -- 172.21.15.45:0/2884959704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 msgr2=0x55ab3976bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.410+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab3976bc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3940a2710 tx=0x7ff39409a7a0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 -- 172.21.15.45:0/2884959704 shutdown_connections 2023-10-17T01:17:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3976d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab3976bc60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/2884959704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 -- 172.21.15.45:0/2884959704 >> 172.21.15.45:0/2884959704 conn(0x55ab396ba070 msgr2=0x55ab397552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 -- 172.21.15.45:0/2884959704 shutdown_connections 2023-10-17T01:17:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.411+0000 7ff3bd24db80 1 -- 172.21.15.45:0/2884959704 wait complete. 2023-10-17T01:17:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.412+0000 7ff3bd24db80 1 -- start start 2023-10-17T01:17:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.412+0000 7ff3bd24db80 1 Processor -- start 2023-10-17T01:17:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab39775080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976c1a0 0x55ab397755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3977c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab396484b0 con 0x55ab3976cda0 2023-10-17T01:17:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab39648150 con 0x55ab3976c1a0 2023-10-17T01:17:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab3952f8c0 con 0x55ab3976b840 2023-10-17T01:17:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.413+0000 7ff3bd24db80 10 client.0 _ll_register_callbacks cb 0x55ab39595a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.414+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab39775080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.414+0000 7ff3a27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976c1a0 0x55ab397755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.414+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab39775080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:39008/0 (socket says 172.21.15.45:39008) 2023-10-17T01:17:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.414+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/1226046811 learned_addr learned my addr 172.21.15.45:0/1226046811 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.414+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3977c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976c1a0 msgr2=0x55ab397755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976c1a0 0x55ab397755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/1226046811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 msgr2=0x55ab3977c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3977c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/1226046811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3980099e0 con 0x55ab3976b840 2023-10-17T01:17:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3977c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.415+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab39775080 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff39409abf0 tx=0x7ff394005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b88db963e2b041 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:17:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff382ffd700 1 -- 172.21.15.45:0/1226046811 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394099000 con 0x55ab3976b840 2023-10-17T01:17:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff382ffd700 1 -- 172.21.15.45:0/1226046811 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff39409ea90 con 0x55ab3976b840 2023-10-17T01:17:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab396d6cb0 con 0x55ab3976b840 2023-10-17T01:17:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff382ffd700 1 -- 172.21.15.45:0/1226046811 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3940b3160 con 0x55ab3976b840 2023-10-17T01:17:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff3bd24db80 10 client.25771 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7ff3a41a5790 con 0x55ab3976b840 2023-10-17T01:17:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.416+0000 7ff382ffd700 1 -- 172.21.15.45:0/1226046811 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab396d6cb0 con 0x55ab3976b840 2023-10-17T01:17:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff382ffd700 1 -- 172.21.15.45:0/1226046811 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff3940cd150 con 0x55ab3976b840 2023-10-17T01:17:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3bd24db80 10 client.25771 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3bd24db80 10 client.25771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3bd24db80 20 client.25771 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397919a0 0x55ab39793dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3bd24db80 4 client.25771 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab39729cd0 con 0x55ab397919a0 2023-10-17T01:17:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.417+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397919a0 0x55ab39793dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.418+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397919a0 0x55ab39793dc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1df1d6d76910254 server_cookie=9aa3060f1f3c1925 in_seq=0 out_seq=0 2023-10-17T01:17:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.418+0000 7ff382ffd700 10 client.25771 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.419+0000 7ff382ffd700 1 -- 172.21.15.45:0/1226046811 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ab39729cd0 con 0x55ab397919a0 2023-10-17T01:17:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.419+0000 7ff382ffd700 10 client.25771 handle_command_reply: tid=0 2023-10-17T01:17:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.419+0000 7ff3bd24db80 1 client.25771 shutdown 2023-10-17T01:17:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 msgr2=0x55ab39775080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab39775080 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff39409abf0 tx=0x7ff394005c00 comp rx=0 tx=0).stop 2023-10-17T01:17:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 shutdown_connections 2023-10-17T01:17:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab3977c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab39775080 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397919a0 0x55ab39793dc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1226046811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976c1a0 0x55ab397755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 >> 172.21.15.45:0/1226046811 conn(0x55ab396ba070 msgr2=0x55ab39754a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.420+0000 7ff382ffd700 0 client.25771 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.421+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 shutdown_connections 2023-10-17T01:17:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.421+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1226046811 wait complete. 2023-10-17T01:17:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.422+0000 7ff3bd24db80 20 client.25771 trim_cache size 0 max 16384 2023-10-17T01:17:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.447+0000 7ff3bd24db80 1 Processor -- start 2023-10-17T01:17:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 -- start start 2023-10-17T01:17:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab39773210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab397702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab396d6cb0 con 0x55ab3976cda0 2023-10-17T01:17:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab39776270 con 0x55ab3976b840 2023-10-17T01:17:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.448+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab39756d40 con 0x55ab3976c1a0 2023-10-17T01:17:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.449+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab39773210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.449+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab39773210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56770/0 (socket says 172.21.15.45:56770) 2023-10-17T01:17:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.450+0000 7ff3a2ffd700 1 -- 172.21.15.45:0/1876517175 learned_addr learned my addr 172.21.15.45:0/1876517175 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.450+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.451+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab397702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a37fe700 1 -- 172.21.15.45:0/1876517175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 msgr2=0x55ab3976fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a37fe700 1 -- 172.21.15.45:0/1876517175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 msgr2=0x55ab39773210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab39773210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a37fe700 1 -- 172.21.15.45:0/1876517175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab394c2c00 con 0x55ab3976cda0 2023-10-17T01:17:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab39773210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:17:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.453+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab397702a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ab39773be0 tx=0x7ff39c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a400222179b951 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.455+0000 7ff38a7fc700 1 -- 172.21.15.45:0/1876517175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00e040 con 0x55ab3976cda0 2023-10-17T01:17:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.455+0000 7ff38a7fc700 1 -- 172.21.15.45:0/1876517175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff39c00f040 con 0x55ab3976cda0 2023-10-17T01:17:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.455+0000 7ff38a7fc700 1 -- 172.21.15.45:0/1876517175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c017670 con 0x55ab3976cda0 2023-10-17T01:17:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.456+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1876517175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 msgr2=0x55ab397702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.456+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab397702a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ab39773be0 tx=0x7ff39c009920 comp rx=0 tx=0).stop 2023-10-17T01:17:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.456+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1876517175 shutdown_connections 2023-10-17T01:17:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.457+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976cda0 0x55ab397702a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.457+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976b840 0x55ab39773210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.457+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1876517175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976c1a0 0x55ab3976fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.457+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1876517175 >> 172.21.15.45:0/1876517175 conn(0x55ab396ba070 msgr2=0x55ab39773d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.458+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1876517175 shutdown_connections 2023-10-17T01:17:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.458+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1876517175 wait complete. 2023-10-17T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.459+0000 7ff3bd24db80 1 -- start start 2023-10-17T01:17:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.460+0000 7ff3bd24db80 1 Processor -- start 2023-10-17T01:17:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.461+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab397dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.461+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 0x55ab397df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3bd24db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976cda0 0x55ab397e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3a2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab397dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab397e25e0 con 0x55ab3976c1a0 2023-10-17T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab397e2750 con 0x55ab3976cda0 2023-10-17T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3bd24db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab397e2910 con 0x55ab3976b840 2023-10-17T01:17:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3bd24db80 10 client.0 _ll_register_callbacks cb 0x7ff39800a300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3a27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 0x55ab397df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3a27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 0x55ab397df810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47726/0 (socket says 172.21.15.45:47726) 2023-10-17T01:17:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.462+0000 7ff3a27fc700 1 -- 172.21.15.45:0/1766298375 learned_addr learned my addr 172.21.15.45:0/1766298375 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.463+0000 7ff3a27fc700 1 -- 172.21.15.45:0/1766298375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 msgr2=0x55ab397dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab397dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a27fc700 1 -- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976cda0 msgr2=0x55ab397e1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976cda0 0x55ab397e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976cda0 0x55ab397e1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a27fc700 1 -- 172.21.15.45:0/1766298375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a41a5790 con 0x55ab3976c1a0 2023-10-17T01:17:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a37fe700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976cda0 0x55ab397e1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:17:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.464+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab397dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.466+0000 7ff3a27fc700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 0x55ab397df810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff39800a3b0 tx=0x7ff398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2ad0acc32b43b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.466+0000 7ff382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:17:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.466+0000 7ff382ffd700 1 -- 172.21.15.45:0/1766298375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398003a60 con 0x55ab3976c1a0 2023-10-17T01:17:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.466+0000 7ff382ffd700 1 -- 172.21.15.45:0/1766298375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff398003bc0 con 0x55ab3976c1a0 2023-10-17T01:17:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.466+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff39409ac60 con 0x55ab3976c1a0 2023-10-17T01:17:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.467+0000 7ff382ffd700 1 -- 172.21.15.45:0/1766298375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff398003d80 con 0x55ab3976c1a0 2023-10-17T01:17:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.467+0000 7ff3bd24db80 10 client.15636 fetch_fsmap learned FSMap version 33 2023-10-17T01:17:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.467+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7ff39c003a00 con 0x55ab3976c1a0 2023-10-17T01:17:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.467+0000 7ff382ffd700 1 -- 172.21.15.45:0/1766298375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff39409ac60 con 0x55ab3976c1a0 2023-10-17T01:17:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff382ffd700 1 -- 172.21.15.45:0/1766298375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff398018500 con 0x55ab3976c1a0 2023-10-17T01:17:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff3bd24db80 10 client.15636 fetch_fsmap finished waiting for FSMap version 33 2023-10-17T01:17:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff3bd24db80 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff3bd24db80 20 client.15636 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397e43a0 0x55ab397e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff3bd24db80 4 client.15636 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:17:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.468+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab396a4ce0 con 0x55ab397e43a0 2023-10-17T01:17:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.469+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397e43a0 0x55ab397e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.470+0000 7ff3a2ffd700 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397e43a0 0x55ab397e67c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72f338ebb0a2f7e server_cookie=88b21489a3fea401 in_seq=0 out_seq=0 2023-10-17T01:17:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.470+0000 7ff382ffd700 10 client.15636 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.487+0000 7ff382ffd700 1 -- 172.21.15.45:0/1766298375 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab396a4ce0 con 0x55ab397e43a0 2023-10-17T01:17:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.487+0000 7ff382ffd700 10 client.15636 handle_command_reply: tid=0 2023-10-17T01:17:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.487+0000 7ff3bd24db80 1 client.15636 shutdown 2023-10-17T01:17:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.487+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 msgr2=0x55ab397df810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.487+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 0x55ab397df810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff39800a3b0 tx=0x7ff398004750 comp rx=0 tx=0).stop 2023-10-17T01:17:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 shutdown_connections 2023-10-17T01:17:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ab397e43a0 0x55ab397e67c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab3976cda0 0x55ab397e1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab3976b840 0x55ab397dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff382ffd700 0 client.15636 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 --2- 172.21.15.45:0/1766298375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab3976c1a0 0x55ab397df810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 >> 172.21.15.45:0/1766298375 conn(0x55ab396ba070 msgr2=0x55ab39773210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 shutdown_connections 2023-10-17T01:17:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 1 -- 172.21.15.45:0/1766298375 wait complete. 2023-10-17T01:17:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.488+0000 7ff3bd24db80 20 client.15636 trim_cache size 0 max 16384 2023-10-17T01:17:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:17:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:17:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:17:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:17:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff38c0a0b20 msgr2=0x7ff38c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff38c0a0b20 0x7ff38c0a2fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff39800a850 tx=0x7ff39800c040 comp rx=0 tx=0).stop 2023-10-17T01:17:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 msgr2=0x7ff3a419d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 0x7ff3a419d180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff394004710 tx=0x7ff394009580 comp rx=0 tx=0).stop 2023-10-17T01:17:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 shutdown_connections 2023-10-17T01:17:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff38c0a0b20 0x7ff38c0a2fe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a4100b20 0x7ff3a419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3a40fef90 0x7ff3a419d180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 --2- 172.21.15.45:0/825084267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a4100180 0x7ff3a419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.489+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 >> 172.21.15.45:0/825084267 conn(0x7ff3a40fa500 msgr2=0x7ff3a40fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.490+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 shutdown_connections 2023-10-17T01:17:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:31.490+0000 7ff3a99ff700 1 -- 172.21.15.45:0/825084267 wait complete. 2023-10-17T01:17:31.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:31 smithi045 ceph-mon[98474]: pgmap v404: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 1.89k op/s 2023-10-17T01:17:31.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:31 smithi060 ceph-mon[134136]: pgmap v404: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 1.89k op/s 2023-10-17T01:17:31.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:31 smithi181 ceph-mon[134520]: pgmap v404: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 1.89k op/s 2023-10-17T01:17:33.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:33 smithi060 ceph-mon[134136]: pgmap v405: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 42 MiB/s wr, 1.84k op/s 2023-10-17T01:17:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:33 smithi181 ceph-mon[134520]: pgmap v405: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 42 MiB/s wr, 1.84k op/s 2023-10-17T01:17:34.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:33 smithi045 ceph-mon[98474]: pgmap v405: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 42 MiB/s wr, 1.84k op/s 2023-10-17T01:17:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 145.0 s, 3012.4 tps, lat 33.013 ms stddev 35.469 2023-10-17T01:17:35.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:35 smithi060 ceph-mon[134136]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 MiB/s wr, 1.88k op/s 2023-10-17T01:17:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:35 smithi181 ceph-mon[134520]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 MiB/s wr, 1.88k op/s 2023-10-17T01:17:36.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:35 smithi045 ceph-mon[98474]: pgmap v406: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 MiB/s wr, 1.88k op/s 2023-10-17T01:17:37.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:37 smithi060 ceph-mon[134136]: pgmap v407: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 1.31k op/s 2023-10-17T01:17:37.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:37 smithi181 ceph-mon[134520]: pgmap v407: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 1.31k op/s 2023-10-17T01:17:38.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:37 smithi045 ceph-mon[98474]: pgmap v407: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 1.31k op/s 2023-10-17T01:17:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:39 smithi181 ceph-mon[134520]: pgmap v408: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2023-10-17T01:17:40.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:39 smithi045 ceph-mon[98474]: pgmap v408: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2023-10-17T01:17:40.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:39 smithi060 ceph-mon[134136]: pgmap v408: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2023-10-17T01:17:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 150.0 s, 3198.4 tps, lat 31.202 ms stddev 34.667 2023-10-17T01:17:40.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:40 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:41.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:40 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:40 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:41.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:41 smithi181 ceph-mon[134520]: pgmap v409: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-10-17T01:17:42.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:41 smithi045 ceph-mon[98474]: pgmap v409: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-10-17T01:17:42.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:41 smithi060 ceph-mon[134136]: pgmap v409: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-10-17T01:17:42.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:42 smithi181 ceph-mon[134520]: pgmap v410: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 830 op/s 2023-10-17T01:17:43.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:42 smithi045 ceph-mon[98474]: pgmap v410: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 830 op/s 2023-10-17T01:17:43.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:42 smithi060 ceph-mon[134136]: pgmap v410: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 830 op/s 2023-10-17T01:17:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 -- 172.21.15.45:0/323262488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e81017e0 msgr2=0x7f68e8108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/323262488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e81017e0 0x7f68e8108090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009fc0 tx=0x7f68dc00b330 comp rx=0 tx=0).stop 2023-10-17T01:17:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 -- 172.21.15.45:0/323262488 shutdown_connections 2023-10-17T01:17:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/323262488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e81017e0 0x7f68e8108090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/323262488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e80ffc50 0x7f68e8100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/323262488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 0x7f68e81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.751+0000 7f68ed1ff700 1 -- 172.21.15.45:0/323262488 >> 172.21.15.45:0/323262488 conn(0x7f68e80fb1a0 msgr2=0x7f68e80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.752+0000 7f68ed1ff700 1 -- 172.21.15.45:0/323262488 shutdown_connections 2023-10-17T01:17:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.753+0000 7f68ed1ff700 1 -- 172.21.15.45:0/323262488 wait complete. 2023-10-17T01:17:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.754+0000 7f68ed1ff700 1 Processor -- start 2023-10-17T01:17:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.755+0000 7f68ed1ff700 1 -- start start 2023-10-17T01:17:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68ed1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 0x7f68e8198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68ed1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 0x7f68e8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68ed1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e81017e0 0x7f68e819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68ed1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819a850 con 0x7f68e8100e40 2023-10-17T01:17:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68ed1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819a9c0 con 0x7f68e80ffc50 2023-10-17T01:17:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68ed1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819ab80 con 0x7f68e81017e0 2023-10-17T01:17:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.756+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 0x7f68e8198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 0x7f68e8198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39828/0 (socket says 172.21.15.45:39828) 2023-10-17T01:17:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 -- 172.21.15.45:0/4005435248 learned_addr learned my addr 172.21.15.45:0/4005435248 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e759e700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e81017e0 0x7f68e819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e659c700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 0x7f68e8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 -- 172.21.15.45:0/4005435248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e81017e0 msgr2=0x7f68e819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e81017e0 0x7f68e819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 -- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 msgr2=0x7f68e8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 0x7f68e8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.757+0000 7f68e6d9d700 1 -- 172.21.15.45:0/4005435248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc009c40 con 0x7f68e80ffc50 2023-10-17T01:17:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.758+0000 7f68e659c700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 0x7f68e8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:17:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.758+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 0x7f68e8198ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68e000e970 tx=0x7f68e000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.758+0000 7f68cffff700 1 -- 172.21.15.45:0/4005435248 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0009790 con 0x7f68e80ffc50 2023-10-17T01:17:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.758+0000 7f68cffff700 1 -- 172.21.15.45:0/4005435248 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68e000eec0 con 0x7f68e80ffc50 2023-10-17T01:17:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.758+0000 7f68cffff700 1 -- 172.21.15.45:0/4005435248 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0003cf0 con 0x7f68e80ffc50 2023-10-17T01:17:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.759+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e819ae60 con 0x7f68e80ffc50 2023-10-17T01:17:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.759+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e81a1580 con 0x7f68e80ffc50 2023-10-17T01:17:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.766+0000 7f68cffff700 1 -- 172.21.15.45:0/4005435248 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f68e000ccd0 con 0x7f68e80ffc50 2023-10-17T01:17:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.768+0000 7f68cffff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68d00a0b80 0x7f68d00a3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.768+0000 7f68cffff700 1 -- 172.21.15.45:0/4005435248 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68e0014070 con 0x7f68e80ffc50 2023-10-17T01:17:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.769+0000 7f68e659c700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68d00a0b80 0x7f68d00a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.770+0000 7f68e659c700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68d00a0b80 0x7f68d00a3040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f68d8005fd0 tx=0x7f68d8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.770+0000 7f6900acbb80 1 Processor -- start 2023-10-17T01:17:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 -- start start 2023-10-17T01:17:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92bc9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bca190 0x55ca92bca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92b219b0 con 0x55ca92bcad90 2023-10-17T01:17:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92b1f370 con 0x55ca92bca190 2023-10-17T01:17:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.771+0000 7f6900acbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92b1ac60 con 0x55ca92bc9830 2023-10-17T01:17:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.772+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92bc9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.772+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92bc9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60132/0 (socket says 172.21.15.45:60132) 2023-10-17T01:17:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.772+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2808679246 learned_addr learned my addr 172.21.15.45:0/2808679246 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.772+0000 7f68e659c700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bca190 0x55ca92bca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.773+0000 7f68e759e700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.773+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2808679246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bca190 msgr2=0x55ca92bca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.773+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bca190 0x55ca92bca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.773+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2808679246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 msgr2=0x55ca92bcb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.773+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.773+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2808679246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca92948d80 con 0x55ca92bc9830 2023-10-17T01:17:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.774+0000 7f68e659c700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bca190 0x55ca92bca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.774+0000 7f68e759e700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:17:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.774+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92bc9c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68e00a1500 tx=0x7f68e0099320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f656f8a7b688c94 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.775+0000 7f68cd7fa700 1 -- 172.21.15.45:0/2808679246 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00b1070 con 0x55ca92bc9830 2023-10-17T01:17:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.775+0000 7f68cd7fa700 1 -- 172.21.15.45:0/2808679246 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68e009d7f0 con 0x55ca92bc9830 2023-10-17T01:17:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.775+0000 7f68cd7fa700 1 -- 172.21.15.45:0/2808679246 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00b92b0 con 0x55ca92bc9830 2023-10-17T01:17:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.776+0000 7f6900acbb80 1 -- 172.21.15.45:0/2808679246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 msgr2=0x55ca92bc9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.776+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92bc9c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68e00a1500 tx=0x7f68e0099320 comp rx=0 tx=0).stop 2023-10-17T01:17:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 -- 172.21.15.45:0/2808679246 shutdown_connections 2023-10-17T01:17:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92bc9c50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2808679246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bca190 0x55ca92bca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 -- 172.21.15.45:0/2808679246 >> 172.21.15.45:0/2808679246 conn(0x55ca92b29070 msgr2=0x55ca92bc42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 -- 172.21.15.45:0/2808679246 shutdown_connections 2023-10-17T01:17:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.777+0000 7f6900acbb80 1 -- 172.21.15.45:0/2808679246 wait complete. 2023-10-17T01:17:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.778+0000 7f6900acbb80 1 -- start start 2023-10-17T01:17:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.778+0000 7f6900acbb80 1 Processor -- start 2023-10-17T01:17:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92becc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bca190 0x55ca92bed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bf41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92ab74b0 con 0x55ca92bcad90 2023-10-17T01:17:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92ab7150 con 0x55ca92bc9830 2023-10-17T01:17:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9299e8c0 con 0x55ca92bca190 2023-10-17T01:17:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f6900acbb80 10 client.0 _ll_register_callbacks cb 0x55ca92a04a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92becc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f68e659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bca190 0x55ca92bed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92becc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39850/0 (socket says 172.21.15.45:39850) 2023-10-17T01:17:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.780+0000 7f68e6d9d700 1 -- 172.21.15.45:0/3422899319 learned_addr learned my addr 172.21.15.45:0/3422899319 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.781+0000 7f68e659c700 1 -- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 msgr2=0x55ca92becc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.781+0000 7f68e659c700 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92becc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.781+0000 7f68e659c700 1 -- 172.21.15.45:0/3422899319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 msgr2=0x55ca92bf41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.781+0000 7f68e659c700 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bf41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.781+0000 7f68e659c700 1 -- 172.21.15.45:0/3422899319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d8010910 con 0x55ca92bca190 2023-10-17T01:17:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.782+0000 7f68e659c700 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bca190 0x55ca92bed170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ca92bc95d0 tx=0x7f68d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9316850c0a99fbe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.783+0000 7f68caff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:17:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.783+0000 7f68caff5700 1 -- 172.21.15.45:0/3422899319 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8029670 con 0x55ca92bca190 2023-10-17T01:17:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.783+0000 7f68caff5700 1 -- 172.21.15.45:0/3422899319 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68d8020430 con 0x55ca92bca190 2023-10-17T01:17:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.784+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca92b45cb0 con 0x55ca92bca190 2023-10-17T01:17:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.784+0000 7f68caff5700 1 -- 172.21.15.45:0/3422899319 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68d80297d0 con 0x55ca92bca190 2023-10-17T01:17:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.784+0000 7f6900acbb80 10 client.25789 fetch_fsmap learned FSMap version 34 2023-10-17T01:17:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.784+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f68e81a1580 con 0x55ca92bca190 2023-10-17T01:17:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f68caff5700 1 -- 172.21.15.45:0/3422899319 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca92b45cb0 con 0x55ca92bca190 2023-10-17T01:17:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f68caff5700 1 -- 172.21.15.45:0/3422899319 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f68d8020c90 con 0x55ca92bca190 2023-10-17T01:17:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f6900acbb80 10 client.25789 fetch_fsmap finished waiting for FSMap version 34 2023-10-17T01:17:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f6900acbb80 10 client.25789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f6900acbb80 20 client.25789 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92be7640 0x55ca92be9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f6900acbb80 4 client.25789 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.786+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca92b98cd0 con 0x55ca92be7640 2023-10-17T01:17:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.787+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92be7640 0x55ca92be9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.788+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92be7640 0x55ca92be9a60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953552ba6ac1be01 server_cookie=7cd6de1c89c69209 in_seq=0 out_seq=0 2023-10-17T01:17:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.788+0000 7f68caff5700 10 client.25789 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.789+0000 7f68caff5700 1 -- 172.21.15.45:0/3422899319 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ca92b98cd0 con 0x55ca92be7640 2023-10-17T01:17:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.789+0000 7f68caff5700 10 client.25789 handle_command_reply: tid=0 2023-10-17T01:17:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.789+0000 7f6900acbb80 1 client.25789 shutdown 2023-10-17T01:17:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bca190 msgr2=0x55ca92bed170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bca190 0x55ca92bed170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ca92bc95d0 tx=0x7f68d8004750 comp rx=0 tx=0).stop 2023-10-17T01:17:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 shutdown_connections 2023-10-17T01:17:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bf41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92be7640 0x55ca92be9a60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92becc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3422899319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bca190 0x55ca92bed170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f68caff5700 0 client.25789 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 >> 172.21.15.45:0/3422899319 conn(0x55ca92b29070 msgr2=0x55ca92bc3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.790+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 shutdown_connections 2023-10-17T01:17:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.791+0000 7f6900acbb80 1 -- 172.21.15.45:0/3422899319 wait complete. 2023-10-17T01:17:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.791+0000 7f6900acbb80 20 client.25789 trim_cache size 0 max 16384 2023-10-17T01:17:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.821+0000 7f6900acbb80 1 Processor -- start 2023-10-17T01:17:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 -- start start 2023-10-17T01:17:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92be9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bf20c0 0x55ca92bce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92be7450 con 0x55ca92bcad90 2023-10-17T01:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92bbe9e0 con 0x55ca92bf20c0 2023-10-17T01:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.822+0000 7f6900acbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92bec0e0 con 0x55ca92bc9830 2023-10-17T01:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.825+0000 7f68e659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.825+0000 7f68e659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39256/0 (socket says 172.21.15.45:39256) 2023-10-17T01:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.825+0000 7f68e659c700 1 -- 172.21.15.45:0/3188836191 learned_addr learned my addr 172.21.15.45:0/3188836191 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.826+0000 7f68e659c700 1 -- 172.21.15.45:0/3188836191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 msgr2=0x55ca92be9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.826+0000 7f68e659c700 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92be9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.826+0000 7f68e659c700 1 -- 172.21.15.45:0/3188836191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bf20c0 msgr2=0x55ca92bce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.826+0000 7f68e659c700 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bf20c0 0x55ca92bce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.826+0000 7f68e659c700 1 -- 172.21.15.45:0/3188836191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca92931c00 con 0x55ca92bcad90 2023-10-17T01:17:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.829+0000 7f68e659c700 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcdd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f68d8005b40 tx=0x7f68d80335e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c778f4a78e2bcc04 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.836+0000 7f68cdffb700 1 -- 172.21.15.45:0/3188836191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8012b10 con 0x55ca92bcad90 2023-10-17T01:17:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.836+0000 7f68cdffb700 1 -- 172.21.15.45:0/3188836191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68d80336b0 con 0x55ca92bcad90 2023-10-17T01:17:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.836+0000 7f68cdffb700 1 -- 172.21.15.45:0/3188836191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d80045e0 con 0x55ca92bcad90 2023-10-17T01:17:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.837+0000 7f6900acbb80 1 -- 172.21.15.45:0/3188836191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 msgr2=0x55ca92bcdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.837+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcdd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f68d8005b40 tx=0x7f68d80335e0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.838+0000 7f6900acbb80 1 -- 172.21.15.45:0/3188836191 shutdown_connections 2023-10-17T01:17:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.838+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bf20c0 0x55ca92bce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.838+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bc9830 0x55ca92be9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.838+0000 7f6900acbb80 1 --2- 172.21.15.45:0/3188836191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bcad90 0x55ca92bcdd50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.838+0000 7f6900acbb80 1 -- 172.21.15.45:0/3188836191 >> 172.21.15.45:0/3188836191 conn(0x55ca92b29070 msgr2=0x55ca92bd1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.838+0000 7f6900acbb80 1 -- 172.21.15.45:0/3188836191 shutdown_connections 2023-10-17T01:17:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.839+0000 7f6900acbb80 1 -- 172.21.15.45:0/3188836191 wait complete. 2023-10-17T01:17:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.839+0000 7f6900acbb80 1 -- start start 2023-10-17T01:17:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.840+0000 7f6900acbb80 1 Processor -- start 2023-10-17T01:17:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.841+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92bf0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.841+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bcad90 0x55ca92bf0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.841+0000 7f6900acbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bf20c0 0x55ca92bf0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.841+0000 7f6900acbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92bf1e80 con 0x55ca92bf20c0 2023-10-17T01:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.841+0000 7f6900acbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92c52320 con 0x55ca92bc9830 2023-10-17T01:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.841+0000 7f6900acbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca92c52460 con 0x55ca92bcad90 2023-10-17T01:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.842+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92bf0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.842+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92bf0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39868/0 (socket says 172.21.15.45:39868) 2023-10-17T01:17:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.842+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2659659063 learned_addr learned my addr 172.21.15.45:0/2659659063 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.842+0000 7f6900acbb80 10 client.0 _ll_register_callbacks cb 0x7f68d8010c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.842+0000 7f68e659c700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bcad90 0x55ca92bf0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.843+0000 7f68e759e700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bf20c0 0x55ca92bf0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.844+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2659659063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bcad90 msgr2=0x55ca92bf0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.844+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bcad90 0x55ca92bf0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.844+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2659659063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bf20c0 msgr2=0x55ca92bf0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.844+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bf20c0 0x55ca92bf0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.844+0000 7f68e6d9d700 1 -- 172.21.15.45:0/2659659063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e81a1580 con 0x55ca92bc9830 2023-10-17T01:17:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.846+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92bf0200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c5b0 tx=0x7f68e000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133e1810d643fd4e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.846+0000 7f68caff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.846+0000 7f68caff5700 1 -- 172.21.15.45:0/2659659063 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00b1040 con 0x55ca92bc9830 2023-10-17T01:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.846+0000 7f68caff5700 1 -- 172.21.15.45:0/2659659063 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68e00b90b0 con 0x55ca92bc9830 2023-10-17T01:17:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.848+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68e009d7f0 con 0x55ca92bc9830 2023-10-17T01:17:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.848+0000 7f68caff5700 1 -- 172.21.15.45:0/2659659063 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68e00b9270 con 0x55ca92bc9830 2023-10-17T01:17:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.848+0000 7f6900acbb80 10 client.25814 fetch_fsmap learned FSMap version 34 2023-10-17T01:17:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.848+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ca9296f1f0 con 0x55ca92bc9830 2023-10-17T01:17:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f68caff5700 1 -- 172.21.15.45:0/2659659063 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68e009d7f0 con 0x55ca92bc9830 2023-10-17T01:17:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f68caff5700 1 -- 172.21.15.45:0/2659659063 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f68e007e020 con 0x55ca92bc9830 2023-10-17T01:17:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f6900acbb80 10 client.25814 fetch_fsmap finished waiting for FSMap version 34 2023-10-17T01:17:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f6900acbb80 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f6900acbb80 20 client.25814 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92c53a30 0x55ca92c55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f6900acbb80 4 client.25814 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:17:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.849+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca92b13ce0 con 0x55ca92c53a30 2023-10-17T01:17:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.850+0000 7f68e659c700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92c53a30 0x55ca92c55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.853+0000 7f68caff5700 10 client.25814 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.853+0000 7f68e659c700 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92c53a30 0x55ca92c55e50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c922fe410c52b1b server_cookie=b5dc669f5a3c1057 in_seq=0 out_seq=0 2023-10-17T01:17:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f68caff5700 1 -- 172.21.15.45:0/2659659063 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca92b13ce0 con 0x55ca92c53a30 2023-10-17T01:17:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f68caff5700 10 client.25814 handle_command_reply: tid=0 2023-10-17T01:17:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 client.25814 shutdown 2023-10-17T01:17:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 msgr2=0x55ca92bf0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92bf0200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c5b0 tx=0x7f68e000cbd0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 shutdown_connections 2023-10-17T01:17:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca92bf20c0 0x55ca92bf0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ca92c53a30 0x55ca92c55e50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca92bc9830 0x55ca92bf0200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 --2- 172.21.15.45:0/2659659063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ca92bcad90 0x55ca92bf0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 >> 172.21.15.45:0/2659659063 conn(0x55ca92b29070 msgr2=0x55ca92bcdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.870+0000 7f68caff5700 0 client.25814 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.871+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 shutdown_connections 2023-10-17T01:17:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.871+0000 7f6900acbb80 1 -- 172.21.15.45:0/2659659063 wait complete. 2023-10-17T01:17:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.871+0000 7f6900acbb80 20 client.25814 trim_cache size 0 max 16384 2023-10-17T01:17:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:17:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:17:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:17:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:17:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68d00a0b80 msgr2=0x7f68d00a3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68d00a0b80 0x7f68d00a3040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f68d8005fd0 tx=0x7f68d8009310 comp rx=0 tx=0).stop 2023-10-17T01:17:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 msgr2=0x7f68e8198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 0x7f68e8198ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68e000e970 tx=0x7f68e000ec80 comp rx=0 tx=0).stop 2023-10-17T01:17:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 shutdown_connections 2023-10-17T01:17:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68d00a0b80 0x7f68d00a3040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68e81017e0 0x7f68e819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68e80ffc50 0x7f68e8198ed0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 --2- 172.21.15.45:0/4005435248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8100e40 0x7f68e8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 >> 172.21.15.45:0/4005435248 conn(0x7f68e80fb1a0 msgr2=0x7f68e80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 shutdown_connections 2023-10-17T01:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:44.872+0000 7f68ed1ff700 1 -- 172.21.15.45:0/4005435248 wait complete. 2023-10-17T01:17:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 155.0 s, 2982.0 tps, lat 33.815 ms stddev 38.123 2023-10-17T01:17:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:45 smithi181 ceph-mon[134520]: pgmap v411: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 849 op/s 2023-10-17T01:17:46.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:45 smithi045 ceph-mon[98474]: pgmap v411: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 849 op/s 2023-10-17T01:17:46.084 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:45 smithi060 ceph-mon[134136]: pgmap v411: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 849 op/s 2023-10-17T01:17:46.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:46 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:46.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:46 smithi181 ceph-mon[134520]: pgmap v412: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 807 op/s 2023-10-17T01:17:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:46 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:46 smithi045 ceph-mon[98474]: pgmap v412: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 807 op/s 2023-10-17T01:17:47.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:46 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:47.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:46 smithi060 ceph-mon[134136]: pgmap v412: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 807 op/s 2023-10-17T01:17:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:49 smithi181 ceph-mon[134520]: pgmap v413: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 818 op/s 2023-10-17T01:17:50.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:49 smithi045 ceph-mon[98474]: pgmap v413: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 818 op/s 2023-10-17T01:17:50.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:49 smithi060 ceph-mon[134136]: pgmap v413: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 818 op/s 2023-10-17T01:17:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 160.0 s, 2087.5 tps, lat 47.945 ms stddev 59.588 2023-10-17T01:17:51.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:51 smithi181 ceph-mon[134520]: pgmap v414: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2023-10-17T01:17:51.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:51 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:52.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:51 smithi045 ceph-mon[98474]: pgmap v414: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2023-10-17T01:17:52.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:51 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:52.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:51 smithi060 ceph-mon[134136]: pgmap v414: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.05k op/s 2023-10-17T01:17:52.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:51 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:52.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:52 smithi181 ceph-mon[134520]: pgmap v415: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 884 op/s 2023-10-17T01:17:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:52 smithi045 ceph-mon[98474]: pgmap v415: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 884 op/s 2023-10-17T01:17:53.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:52 smithi060 ceph-mon[134136]: pgmap v415: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 884 op/s 2023-10-17T01:17:53.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:53 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:54.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:53 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:54.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:53 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:17:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 165.0 s, 2423.8 tps, lat 41.122 ms stddev 46.298 2023-10-17T01:17:55.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:55 smithi060 ceph-mon[134136]: pgmap v416: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 959 op/s 2023-10-17T01:17:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:55 smithi181 ceph-mon[134520]: pgmap v416: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 959 op/s 2023-10-17T01:17:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:55 smithi045 ceph-mon[98474]: pgmap v416: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 959 op/s 2023-10-17T01:17:57.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:57 smithi060 ceph-mon[134136]: pgmap v417: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 906 op/s 2023-10-17T01:17:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:57 smithi181 ceph-mon[134520]: pgmap v417: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 906 op/s 2023-10-17T01:17:58.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:57 smithi045 ceph-mon[98474]: pgmap v417: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 906 op/s 2023-10-17T01:17:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.125+0000 7fdbde7ff700 1 -- 172.21.15.45:0/2379702281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8102f90 msgr2=0x7fdbd81033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.125+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/2379702281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8102f90 0x7fdbd81033b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80099c0 tx=0x7fdbc8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.126+0000 7fdbde7ff700 1 -- 172.21.15.45:0/2379702281 shutdown_connections 2023-10-17T01:17:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.126+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/2379702281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 0x7fdbd810b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.126+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/2379702281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8102f90 0x7fdbd81033b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.126+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/2379702281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8104180 0x7fdbd81045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.126+0000 7fdbde7ff700 1 -- 172.21.15.45:0/2379702281 >> 172.21.15.45:0/2379702281 conn(0x7fdbd80fe500 msgr2=0x7fdbd8100940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.127+0000 7fdbde7ff700 1 -- 172.21.15.45:0/2379702281 shutdown_connections 2023-10-17T01:17:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.128+0000 7fdbde7ff700 1 -- 172.21.15.45:0/2379702281 wait complete. 2023-10-17T01:17:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.130+0000 7fdbde7ff700 1 Processor -- start 2023-10-17T01:17:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.130+0000 7fdbde7ff700 1 -- start start 2023-10-17T01:17:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbde7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 0x7fdbd8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 0x7fdbd8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 0x7fdbd8198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40040/0 (socket says 172.21.15.45:40040) 2023-10-17T01:17:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbde7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8104180 0x7fdbd81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbde7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 0x7fdbd81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbde7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81a07b0 con 0x7fdbd8104b20 2023-10-17T01:17:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbde7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81a0920 con 0x7fdbd8104180 2023-10-17T01:17:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.131+0000 7fdbde7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81a0ae0 con 0x7fdbd8102f90 2023-10-17T01:17:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.132+0000 7fdbd7fff700 1 -- 172.21.15.45:0/493123592 learned_addr learned my addr 172.21.15.45:0/493123592 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.132+0000 7fdbd7fff700 1 -- 172.21.15.45:0/493123592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8104180 msgr2=0x7fdbd81991b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-17T01:17:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.132+0000 7fdbd7fff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8104180 0x7fdbd81991b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.132+0000 7fdbd7fff700 1 -- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 msgr2=0x7fdbd81a01e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:17:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.133+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 0x7fdbd81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.133+0000 7fdbd7fff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 0x7fdbd81a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.133+0000 7fdbd7fff700 1 -- 172.21.15.45:0/493123592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80095f0 con 0x7fdbd8102f90 2023-10-17T01:17:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.133+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 0x7fdbd81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:17:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.134+0000 7fdbd7fff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 0x7fdbd8198c70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009990 tx=0x7fdbc8003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.134+0000 7fdbd5ffb700 1 -- 172.21.15.45:0/493123592 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc801d070 con 0x7fdbd8102f90 2023-10-17T01:17:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.134+0000 7fdbd5ffb700 1 -- 172.21.15.45:0/493123592 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc8004220 con 0x7fdbd8102f90 2023-10-17T01:17:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.134+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd81a0d60 con 0x7fdbd8102f90 2023-10-17T01:17:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.134+0000 7fdbd5ffb700 1 -- 172.21.15.45:0/493123592 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8021410 con 0x7fdbd8102f90 2023-10-17T01:17:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.134+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd81a1190 con 0x7fdbd8102f90 2023-10-17T01:17:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.140+0000 7fdbd5ffb700 1 -- 172.21.15.45:0/493123592 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdbc800f6d0 con 0x7fdbd8102f90 2023-10-17T01:17:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.143+0000 7fdbf2097b80 1 Processor -- start 2023-10-17T01:17:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.143+0000 7fdbf2097b80 1 -- start start 2023-10-17T01:17:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b2ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2fd90 0x5583c0b30210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0a869b0 con 0x5583c0b2fd90 2023-10-17T01:17:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0a84370 con 0x5583c0b2f190 2023-10-17T01:17:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0a7fc60 con 0x5583c0b2e830 2023-10-17T01:17:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b2ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b2ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40052/0 (socket says 172.21.15.45:40052) 2023-10-17T01:17:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.144+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/3842163519 learned_addr learned my addr 172.21.15.45:0/3842163519 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.145+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/3842163519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 msgr2=0x5583c0b2f790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:17:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.145+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b2f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.145+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/3842163519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2fd90 msgr2=0x5583c0b30210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.145+0000 7fdbcffff700 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2fd90 0x5583c0b30210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.145+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2fd90 0x5583c0b30210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.145+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/3842163519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583c08add80 con 0x5583c0b2e830 2023-10-17T01:17:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.146+0000 7fdbd5ffb700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdbc40a0bf0 0x7fdbc40a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.147+0000 7fdbd7fff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdbc40a0bf0 0x7fdbc40a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.147+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b2ec50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000bb70 tx=0x7fdbd000be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ff86a651c48a4f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.147+0000 7fdbd5ffb700 1 -- 172.21.15.45:0/493123592 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdbc80cd160 con 0x7fdbd8102f90 2023-10-17T01:17:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.148+0000 7fdbce7fc700 1 -- 172.21.15.45:0/3842163519 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000dd00 con 0x5583c0b2e830 2023-10-17T01:17:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.148+0000 7fdbce7fc700 1 -- 172.21.15.45:0/3842163519 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbd000b620 con 0x5583c0b2e830 2023-10-17T01:17:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.148+0000 7fdbce7fc700 1 -- 172.21.15.45:0/3842163519 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000f7b0 con 0x5583c0b2e830 2023-10-17T01:17:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.149+0000 7fdbf2097b80 1 -- 172.21.15.45:0/3842163519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 msgr2=0x5583c0b2ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.149+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b2ec50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000bb70 tx=0x7fdbd000be80 comp rx=0 tx=0).stop 2023-10-17T01:17:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.149+0000 7fdbd7fff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdbc40a0bf0 0x7fdbc40a30b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdbc809a570 tx=0x7fdbc8009cd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.150+0000 7fdbf2097b80 1 -- 172.21.15.45:0/3842163519 shutdown_connections 2023-10-17T01:17:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.150+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2fd90 0x5583c0b30210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.150+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b2ec50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.150+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/3842163519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b2f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.150+0000 7fdbf2097b80 1 -- 172.21.15.45:0/3842163519 >> 172.21.15.45:0/3842163519 conn(0x5583c0a8e070 msgr2=0x5583c0b292c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.151+0000 7fdbf2097b80 1 -- 172.21.15.45:0/3842163519 shutdown_connections 2023-10-17T01:17:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.151+0000 7fdbf2097b80 1 -- 172.21.15.45:0/3842163519 wait complete. 2023-10-17T01:17:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.152+0000 7fdbf2097b80 1 -- start start 2023-10-17T01:17:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.152+0000 7fdbf2097b80 1 Processor -- start 2023-10-17T01:17:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b41100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b48140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0a1c4b0 con 0x5583c0b2f190 2023-10-17T01:17:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b40bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40068/0 (socket says 172.21.15.45:40068) 2023-10-17T01:17:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/1528879478 learned_addr learned my addr 172.21.15.45:0/1528879478 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0a1c150 con 0x5583c0b2fd90 2023-10-17T01:17:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c09038c0 con 0x5583c0b2e830 2023-10-17T01:17:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.153+0000 7fdbf2097b80 10 client.0 _ll_register_callbacks cb 0x5583c0969a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.154+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b48140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.154+0000 7fdbcffff700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b41100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 msgr2=0x5583c0b48140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b48140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/1528879478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 msgr2=0x5583c0b41100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b41100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/1528879478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd000b7a0 con 0x5583c0b2e830 2023-10-17T01:17:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbcffff700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:17:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b40bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000f920 tx=0x7fdbd000b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6749930881838c26 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.155+0000 7fdbb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:17:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbb77fe700 1 -- 172.21.15.45:0/1528879478 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0017cb0 con 0x5583c0b2e830 2023-10-17T01:17:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbb77fe700 1 -- 172.21.15.45:0/1528879478 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdbd0013070 con 0x5583c0b2e830 2023-10-17T01:17:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583c0afdcd0 con 0x5583c0b2e830 2023-10-17T01:17:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbb77fe700 1 -- 172.21.15.45:0/1528879478 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd0004850 con 0x5583c0b2e830 2023-10-17T01:17:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbf2097b80 10 client.25801 fetch_fsmap learned FSMap version 37 2023-10-17T01:17:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5583c0896c00 con 0x5583c0b2e830 2023-10-17T01:17:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbb77fe700 1 -- 172.21.15.45:0/1528879478 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583c0afdcd0 con 0x5583c0b2e830 2023-10-17T01:17:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.157+0000 7fdbb77fe700 1 -- 172.21.15.45:0/1528879478 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdbd002d070 con 0x5583c0b2e830 2023-10-17T01:17:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.158+0000 7fdbf2097b80 10 client.25801 fetch_fsmap finished waiting for FSMap version 37 2023-10-17T01:17:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.158+0000 7fdbf2097b80 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.158+0000 7fdbf2097b80 20 client.25801 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.158+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b4c860 0x5583c0b4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.158+0000 7fdbf2097b80 4 client.25801 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.158+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583c0a78ce0 con 0x5583c0b4c860 2023-10-17T01:17:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.159+0000 7fdbcffff700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b4c860 0x5583c0b4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.161+0000 7fdbcffff700 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b4c860 0x5583c0b4ec80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe96e3b24e4cb95 server_cookie=35139e40e1a7bb88 in_seq=0 out_seq=0 2023-10-17T01:17:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.161+0000 7fdbb77fe700 10 client.25801 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.162+0000 7fdbb77fe700 1 -- 172.21.15.45:0/1528879478 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5583c0a78ce0 con 0x5583c0b4c860 2023-10-17T01:17:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.162+0000 7fdbb77fe700 10 client.25801 handle_command_reply: tid=0 2023-10-17T01:17:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.162+0000 7fdbf2097b80 1 client.25801 shutdown 2023-10-17T01:17:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 msgr2=0x5583c0b40bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b40bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000f920 tx=0x7fdbd000b620 comp rx=0 tx=0).stop 2023-10-17T01:17:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 shutdown_connections 2023-10-17T01:17:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b4c860 0x5583c0b4ec80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b48140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b40bc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/1528879478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 >> 172.21.15.45:0/1528879478 conn(0x5583c0a8e070 msgr2=0x5583c0b28a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.163+0000 7fdbb77fe700 0 client.25801 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.164+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 shutdown_connections 2023-10-17T01:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.164+0000 7fdbf2097b80 1 -- 172.21.15.45:0/1528879478 wait complete. 2023-10-17T01:17:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.164+0000 7fdbf2097b80 20 client.25801 trim_cache size 0 max 16384 2023-10-17T01:17:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.190+0000 7fdbf2097b80 1 Processor -- start 2023-10-17T01:17:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.192+0000 7fdbf2097b80 1 -- start start 2023-10-17T01:17:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b4e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0b36e80 con 0x5583c0b2f190 2023-10-17T01:17:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0b40070 con 0x5583c0b2fd90 2023-10-17T01:17:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbf2097b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0b2a5e0 con 0x5583c0b2e830 2023-10-17T01:17:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b4e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b4e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40074/0 (socket says 172.21.15.45:40074) 2023-10-17T01:17:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/498039333 learned_addr learned my addr 172.21.15.45:0/498039333 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.193+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbcffff700 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/498039333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 msgr2=0x5583c0b366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/498039333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 msgr2=0x5583c0b36160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b36160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/498039333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583c08d41f0 con 0x5583c0b2e830 2023-10-17T01:17:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbcffff700 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:17:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.194+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b4e870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000b770 tx=0x7fdbd000be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55db2b7a789b9f93 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.195+0000 7fdbceffd700 1 -- 172.21.15.45:0/498039333 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0031620 con 0x5583c0b2e830 2023-10-17T01:17:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.195+0000 7fdbceffd700 1 -- 172.21.15.45:0/498039333 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbd0031e50 con 0x5583c0b2e830 2023-10-17T01:17:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.195+0000 7fdbceffd700 1 -- 172.21.15.45:0/498039333 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0005420 con 0x5583c0b2e830 2023-10-17T01:17:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 -- 172.21.15.45:0/498039333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 msgr2=0x5583c0b4e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b4e870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000b770 tx=0x7fdbd000be80 comp rx=0 tx=0).stop 2023-10-17T01:17:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 -- 172.21.15.45:0/498039333 shutdown_connections 2023-10-17T01:17:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2fd90 0x5583c0b366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2e830 0x5583c0b4e870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/498039333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2f190 0x5583c0b36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.197+0000 7fdbf2097b80 1 -- 172.21.15.45:0/498039333 >> 172.21.15.45:0/498039333 conn(0x5583c0a8e070 msgr2=0x5583c0b3f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.198+0000 7fdbf2097b80 1 -- 172.21.15.45:0/498039333 shutdown_connections 2023-10-17T01:17:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.198+0000 7fdbf2097b80 1 -- 172.21.15.45:0/498039333 wait complete. 2023-10-17T01:17:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.199+0000 7fdbf2097b80 1 -- start start 2023-10-17T01:17:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.199+0000 7fdbf2097b80 1 Processor -- start 2023-10-17T01:17:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.200+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 0x5583c0b46000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.200+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b9a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbf2097b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2fd90 0x5583c0b9c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 0x5583c0b46000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbdcd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 0x5583c0b46000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38732/0 (socket says 172.21.15.45:38732) 2023-10-17T01:17:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b9a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b9a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:49976/0 (socket says 172.21.15.45:49976) 2023-10-17T01:17:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/4126731548 learned_addr learned my addr 172.21.15.45:0/4126731548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0b9cf50 con 0x5583c0b2e830 2023-10-17T01:17:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0b9d0c0 con 0x5583c0b2f190 2023-10-17T01:17:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.201+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c0b9d280 con 0x5583c0b2fd90 2023-10-17T01:17:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2fd90 0x5583c0b9c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/4126731548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2fd90 msgr2=0x5583c0b9c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2fd90 0x5583c0b9c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 msgr2=0x5583c0b9a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b9a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbdcd9c700 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583c0896c00 con 0x5583c0b2e830 2023-10-17T01:17:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.202+0000 7fdbcffff700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b9a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbf2097b80 10 client.0 _ll_register_callbacks cb 0x7fdbd000db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583c0aaacb0 con 0x5583c0b2e830 2023-10-17T01:17:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbb77fe700 10 client.15657 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:17:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbdcd9c700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 0x5583c0b46000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000dd90 tx=0x7fdbd000dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d81c12eed728d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbb77fe700 1 -- 172.21.15.45:0/4126731548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00051a0 con 0x5583c0b2e830 2023-10-17T01:17:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbb77fe700 1 -- 172.21.15.45:0/4126731548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdbd0000e40 con 0x5583c0b2e830 2023-10-17T01:17:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbb77fe700 1 -- 172.21.15.45:0/4126731548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd00051a0 con 0x5583c0b2e830 2023-10-17T01:17:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbf2097b80 10 client.15657 fetch_fsmap learned FSMap version 37 2023-10-17T01:17:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.203+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=37}) v3 -- 0x5583c08ac250 con 0x5583c0b2e830 2023-10-17T01:17:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbb77fe700 1 -- 172.21.15.45:0/4126731548 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdbd002d070 con 0x5583c0b2e830 2023-10-17T01:17:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbf2097b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 37 2023-10-17T01:17:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbf2097b80 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbf2097b80 20 client.15657 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b9e4d0 0x5583c0ba08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbf2097b80 4 client.15657 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:17:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583c0a78a00 con 0x5583c0b9e4d0 2023-10-17T01:17:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.204+0000 7fdbcffff700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b9e4d0 0x5583c0ba08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.205+0000 7fdbcffff700 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b9e4d0 0x5583c0ba08f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55e0c180d628e2a server_cookie=b53e5e56ce509140 in_seq=0 out_seq=0 2023-10-17T01:17:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.205+0000 7fdbb77fe700 10 client.15657 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbb77fe700 1 -- 172.21.15.45:0/4126731548 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583c0a78a00 con 0x5583c0b9e4d0 2023-10-17T01:17:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbb77fe700 10 client.15657 handle_command_reply: tid=0 2023-10-17T01:17:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 client.15657 shutdown 2023-10-17T01:17:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 msgr2=0x5583c0b46000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 0x5583c0b46000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000dd90 tx=0x7fdbd000dfd0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 shutdown_connections 2023-10-17T01:17:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583c0b2fd90 0x5583c0b9c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583c0b2e830 0x5583c0b46000 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5583c0b9e4d0 0x5583c0ba08f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 --2- 172.21.15.45:0/4126731548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5583c0b2f190 0x5583c0b9a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 >> 172.21.15.45:0/4126731548 conn(0x5583c0a8e070 msgr2=0x5583c0b4e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.218+0000 7fdbb77fe700 0 client.15657 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.219+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 shutdown_connections 2023-10-17T01:17:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.219+0000 7fdbf2097b80 1 -- 172.21.15.45:0/4126731548 wait complete. 2023-10-17T01:17:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.220+0000 7fdbf2097b80 20 client.15657 trim_cache size 0 max 16384 2023-10-17T01:17:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:17:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:17:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:17:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:17:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdbc40a0bf0 msgr2=0x7fdbc40a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdbc40a0bf0 0x7fdbc40a30b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdbc809a570 tx=0x7fdbc8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 msgr2=0x7fdbd8198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 0x7fdbd8198c70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009990 tx=0x7fdbc8003970 comp rx=0 tx=0).stop 2023-10-17T01:17:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 shutdown_connections 2023-10-17T01:17:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdbc40a0bf0 0x7fdbc40a30b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbd8104b20 0x7fdbd81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbd8102f90 0x7fdbd8198c70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 --2- 172.21.15.45:0/493123592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbd8104180 0x7fdbd81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 >> 172.21.15.45:0/493123592 conn(0x7fdbd80fe500 msgr2=0x7fdbd8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.221+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 shutdown_connections 2023-10-17T01:17:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:17:58.222+0000 7fdbde7ff700 1 -- 172.21.15.45:0/493123592 wait complete. 2023-10-17T01:17:59.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:17:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 -- 172.21.15.45:0/1822462754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8100db0 msgr2=0x7f37d81011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1822462754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8100db0 0x7f37d81011d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37c80099c0 tx=0x7f37c8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 -- 172.21.15.45:0/1822462754 shutdown_connections 2023-10-17T01:17:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1822462754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d8102940 0x7f37d81091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1822462754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8100db0 0x7f37d81011d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1822462754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d8101fa0 0x7f37d8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.536+0000 7f37ddfff700 1 -- 172.21.15.45:0/1822462754 >> 172.21.15.45:0/1822462754 conn(0x7f37d80fc320 msgr2=0x7f37d80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:59.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.537+0000 7f37ddfff700 1 -- 172.21.15.45:0/1822462754 shutdown_connections 2023-10-17T01:17:59.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.537+0000 7f37ddfff700 1 -- 172.21.15.45:0/1822462754 wait complete. 2023-10-17T01:17:59.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.539+0000 7f37ddfff700 1 Processor -- start 2023-10-17T01:17:59.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.539+0000 7f37ddfff700 1 -- start start 2023-10-17T01:17:59.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37ddfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d8100db0 0x7f37d806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37ddfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 0x7f37d806ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37ddfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d8102940 0x7f37d8071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37ddfff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d8072390 con 0x7f37d8102940 2023-10-17T01:17:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37ddfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d8072500 con 0x7f37d8101fa0 2023-10-17T01:17:59.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37ddfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d8072670 con 0x7f37d8100db0 2023-10-17T01:17:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 0x7f37d806ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 0x7f37d806ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:49992/0 (socket says 172.21.15.45:49992) 2023-10-17T01:17:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37d6ffd700 1 -- 172.21.15.45:0/1596965890 learned_addr learned my addr 172.21.15.45:0/1596965890 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:59.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.540+0000 7f37d7fff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d8102940 0x7f37d8071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d6ffd700 1 -- 172.21.15.45:0/1596965890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d8100db0 msgr2=0x7f37d806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d8100db0 0x7f37d806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d6ffd700 1 -- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d8102940 msgr2=0x7f37d8071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d8102940 0x7f37d8071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d6ffd700 1 -- 172.21.15.45:0/1596965890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80095f0 con 0x7f37d8101fa0 2023-10-17T01:17:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 0x7f37d806ed00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00b530 tx=0x7f37cc00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d4ff9700 1 -- 172.21.15.45:0/1596965890 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc011830 con 0x7f37d8101fa0 2023-10-17T01:17:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d4ff9700 1 -- 172.21.15.45:0/1596965890 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37cc011e50 con 0x7f37d8101fa0 2023-10-17T01:17:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37d4ff9700 1 -- 172.21.15.45:0/1596965890 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00f2e0 con 0x7f37d8101fa0 2023-10-17T01:17:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.541+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d8072950 con 0x7f37d8101fa0 2023-10-17T01:17:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.542+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d8198cb0 con 0x7f37d8101fa0 2023-10-17T01:17:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.548+0000 7f37d4ff9700 1 -- 172.21.15.45:0/1596965890 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f37cc0105a0 con 0x7f37d8101fa0 2023-10-17T01:17:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.553+0000 7f37f1a0ab80 1 Processor -- start 2023-10-17T01:17:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.555+0000 7f37d4ff9700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f37c00a0b20 0x7f37c00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.555+0000 7f37d4ff9700 1 -- 172.21.15.45:0/1596965890 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f37cc0cbcb0 con 0x7f37d8101fa0 2023-10-17T01:17:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.557+0000 7f37f1a0ab80 1 -- start start 2023-10-17T01:17:59.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da0b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 0x55d17da0cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d95aa10 con 0x55d17da0b400 2023-10-17T01:17:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d9583d0 con 0x55d17da0cae0 2023-10-17T01:17:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d953cc0 con 0x55d17da0bd60 2023-10-17T01:17:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.558+0000 7f37d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 0x55d17da0cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 0x55d17da0cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50000/0 (socket says 172.21.15.45:50000) 2023-10-17T01:17:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d7fff700 1 -- 172.21.15.45:0/974358616 learned_addr learned my addr 172.21.15.45:0/974358616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:59.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d6ffd700 1 -- 172.21.15.45:0/974358616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 msgr2=0x55d17da0cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 0x55d17da0cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d6ffd700 1 -- 172.21.15.45:0/974358616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 msgr2=0x55d17da0b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da0b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d6ffd700 1 -- 172.21.15.45:0/974358616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17d781de0 con 0x55d17da0bd60 2023-10-17T01:17:59.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d7fff700 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 0x55d17da0cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:17:59.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.559+0000 7f37d7fff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f37c00a0b20 0x7f37c00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.560+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0c500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0c97e0 tx=0x7f37cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb116aed7cc33e3a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:59.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.560+0000 7f37bdffb700 1 -- 172.21.15.45:0/974358616 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc098c90 con 0x55d17da0bd60 2023-10-17T01:17:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.560+0000 7f37bdffb700 1 -- 172.21.15.45:0/974358616 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37cc099a30 con 0x55d17da0bd60 2023-10-17T01:17:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.560+0000 7f37bdffb700 1 -- 172.21.15.45:0/974358616 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc098820 con 0x55d17da0bd60 2023-10-17T01:17:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.561+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/974358616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 msgr2=0x55d17da0c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.561+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0c500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0c97e0 tx=0x7f37cc004750 comp rx=0 tx=0).stop 2023-10-17T01:17:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/974358616 shutdown_connections 2023-10-17T01:17:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0cae0 0x55d17da0cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da0b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/974358616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0c500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/974358616 >> 172.21.15.45:0/974358616 conn(0x55d17d9620d0 msgr2=0x55d17d9fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/974358616 shutdown_connections 2023-10-17T01:17:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.562+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/974358616 wait complete. 2023-10-17T01:17:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.563+0000 7f37d7fff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f37c00a0b20 0x7f37c00a2fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f37d00070b0 tx=0x7f37d00093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.563+0000 7f37f1a0ab80 1 -- start start 2023-10-17T01:17:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.564+0000 7f37f1a0ab80 1 Processor -- start 2023-10-17T01:17:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.565+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 0x55d17da15410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 0x55d17da1c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d8f0510 con 0x55d17da0b400 2023-10-17T01:17:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d8f01b0 con 0x55d17da0bd60 2023-10-17T01:17:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d7d7920 con 0x55d17da0cae0 2023-10-17T01:17:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 0x55d17da15410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 0x55d17da15410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50004/0 (socket says 172.21.15.45:50004) 2023-10-17T01:17:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37d6ffd700 1 -- 172.21.15.45:0/998528385 learned_addr learned my addr 172.21.15.45:0/998528385 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.566+0000 7f37f1a0ab80 10 client.0 _ll_register_callbacks cb 0x55d17d83da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.567+0000 7f37d6ffd700 1 -- 172.21.15.45:0/998528385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 msgr2=0x55d17da1c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.567+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 0x55d17da1c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.567+0000 7f37d6ffd700 1 -- 172.21.15.45:0/998528385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 msgr2=0x55d17da14ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.567+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da14ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.567+0000 7f37d6ffd700 1 -- 172.21.15.45:0/998528385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d0009c30 con 0x55d17da0bd60 2023-10-17T01:17:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.568+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 0x55d17da15410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f37cc099990 tx=0x7f37cc009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21088bb8ebfbc78 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.568+0000 7f37b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:17:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.568+0000 7f37b6ffd700 1 -- 172.21.15.45:0/998528385 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc09d160 con 0x55d17da0bd60 2023-10-17T01:17:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.568+0000 7f37b6ffd700 1 -- 172.21.15.45:0/998528385 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f37cc0a13a0 con 0x55d17da0bd60 2023-10-17T01:17:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.568+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d17d97ed10 con 0x55d17da0bd60 2023-10-17T01:17:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.568+0000 7f37b6ffd700 1 -- 172.21.15.45:0/998528385 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37cc098a30 con 0x55d17da0bd60 2023-10-17T01:17:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37f1a0ab80 10 client.25838 fetch_fsmap learned FSMap version 37 2023-10-17T01:17:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f37d8198cb0 con 0x55d17da0bd60 2023-10-17T01:17:59.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37b6ffd700 1 -- 172.21.15.45:0/998528385 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d17d97ed10 con 0x55d17da0bd60 2023-10-17T01:17:59.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37b6ffd700 1 -- 172.21.15.45:0/998528385 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f37cc095120 con 0x55d17da0bd60 2023-10-17T01:17:59.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37f1a0ab80 10 client.25838 fetch_fsmap finished waiting for FSMap version 37 2023-10-17T01:17:59.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37f1a0ab80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37f1a0ab80 20 client.25838 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.569+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da208a0 0x55d17da22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.570+0000 7f37f1a0ab80 4 client.25838 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:17:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.570+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d17d9d1d30 con 0x55d17da208a0 2023-10-17T01:17:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.571+0000 7f37d77fe700 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da208a0 0x55d17da22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.573+0000 7f37d77fe700 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da208a0 0x55d17da22cc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da3f2c1569f8d44 server_cookie=e542f9cb0b32d2fb in_seq=0 out_seq=0 2023-10-17T01:17:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.573+0000 7f37b6ffd700 10 client.25838 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.574+0000 7f37b6ffd700 1 -- 172.21.15.45:0/998528385 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d17d9d1d30 con 0x55d17da208a0 2023-10-17T01:17:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.574+0000 7f37b6ffd700 10 client.25838 handle_command_reply: tid=0 2023-10-17T01:17:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.574+0000 7f37f1a0ab80 1 client.25838 shutdown 2023-10-17T01:17:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.575+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 msgr2=0x55d17da15410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.575+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 0x55d17da15410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f37cc099990 tx=0x7f37cc009400 comp rx=0 tx=0).stop 2023-10-17T01:17:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 shutdown_connections 2023-10-17T01:17:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 0x55d17da1c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0b400 0x55d17da14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da208a0 0x55d17da22cc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/998528385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0bd60 0x55d17da15410 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 >> 172.21.15.45:0/998528385 conn(0x55d17d9620d0 msgr2=0x55d17d9fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37b6ffd700 0 client.25838 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 shutdown_connections 2023-10-17T01:17:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.576+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/998528385 wait complete. 2023-10-17T01:17:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.577+0000 7f37f1a0ab80 20 client.25838 trim_cache size 0 max 16384 2023-10-17T01:17:59.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.621+0000 7f37f1a0ab80 1 Processor -- start 2023-10-17T01:17:59.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.621+0000 7f37f1a0ab80 1 -- start start 2023-10-17T01:17:59.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.621+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17d9fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.621+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.621+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 0x55d17da228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.622+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d97ed10 con 0x55d17da0cae0 2023-10-17T01:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.622+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17d9f7e60 con 0x55d17da0b400 2023-10-17T01:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.623+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17da160c0 con 0x55d17da0bd60 2023-10-17T01:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.623+0000 7f37d7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 0x55d17da228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.624+0000 7f37d7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 0x55d17da228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38784/0 (socket says 172.21.15.45:38784) 2023-10-17T01:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.624+0000 7f37d7fff700 1 -- 172.21.15.45:0/2820088126 learned_addr learned my addr 172.21.15.45:0/2820088126 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.624+0000 7f37d7fff700 1 -- 172.21.15.45:0/2820088126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 msgr2=0x55d17da0ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.624+0000 7f37d7fff700 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.625+0000 7f37d7fff700 1 -- 172.21.15.45:0/2820088126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 msgr2=0x55d17d9fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.625+0000 7f37d7fff700 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17d9fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.625+0000 7f37d7fff700 1 -- 172.21.15.45:0/2820088126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17d76ac60 con 0x55d17da0cae0 2023-10-17T01:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.627+0000 7f37d7fff700 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 0x55d17da228b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009520 tx=0x7f37d0004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d0277fc863ab11 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.628+0000 7f37be7fc700 1 -- 172.21.15.45:0/2820088126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0014e20 con 0x55d17da0cae0 2023-10-17T01:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.628+0000 7f37be7fc700 1 -- 172.21.15.45:0/2820088126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37d001e040 con 0x55d17da0cae0 2023-10-17T01:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.628+0000 7f37be7fc700 1 -- 172.21.15.45:0/2820088126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0022650 con 0x55d17da0cae0 2023-10-17T01:17:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/2820088126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 msgr2=0x55d17da228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 0x55d17da228b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009520 tx=0x7f37d0004be0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/2820088126 shutdown_connections 2023-10-17T01:17:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0cae0 0x55d17da228b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17d9fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/2820088126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0bd60 0x55d17da0ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/2820088126 >> 172.21.15.45:0/2820088126 conn(0x55d17d9620d0 msgr2=0x55d17da0c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:59.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.630+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/2820088126 shutdown_connections 2023-10-17T01:17:59.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.631+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/2820088126 wait complete. 2023-10-17T01:17:59.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.632+0000 7f37f1a0ab80 1 -- start start 2023-10-17T01:17:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.632+0000 7f37f1a0ab80 1 Processor -- start 2023-10-17T01:17:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17da18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0bd60 0x55d17da19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 0x55d17da19540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17da1a740 con 0x55d17da0bd60 2023-10-17T01:17:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17da72230 con 0x55d17da0b400 2023-10-17T01:17:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17da723a0 con 0x55d17da0cae0 2023-10-17T01:17:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.633+0000 7f37f1a0ab80 10 client.0 _ll_register_callbacks cb 0x7f37d0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:17:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.634+0000 7f37d77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17da18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.634+0000 7f37d77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17da18ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50026/0 (socket says 172.21.15.45:50026) 2023-10-17T01:17:59.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.634+0000 7f37d77fe700 1 -- 172.21.15.45:0/3627400670 learned_addr learned my addr 172.21.15.45:0/3627400670 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:17:59.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.634+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0bd60 0x55d17da19000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.635+0000 7f37d6ffd700 1 -- 172.21.15.45:0/3627400670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 msgr2=0x55d17da19540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.635+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 0x55d17da19540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.635+0000 7f37d6ffd700 1 -- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 msgr2=0x55d17da18ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:17:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.635+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17da18ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.635+0000 7f37d6ffd700 1 -- 172.21.15.45:0/3627400670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d0012040 con 0x55d17da0bd60 2023-10-17T01:17:59.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.636+0000 7f37d6ffd700 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0bd60 0x55d17da19000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f37cc099030 tx=0x7f37cc0109a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7a14ea91feaedf server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:17:59.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.636+0000 7f37b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:17:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.637+0000 7f37b6ffd700 1 -- 172.21.15.45:0/3627400670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00d040 con 0x55d17da0bd60 2023-10-17T01:17:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.637+0000 7f37b6ffd700 1 -- 172.21.15.45:0/3627400670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f37cc009300 con 0x55d17da0bd60 2023-10-17T01:17:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.639+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37cc099a30 con 0x55d17da0bd60 2023-10-17T01:17:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.640+0000 7f37b6ffd700 1 -- 172.21.15.45:0/3627400670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37cc09f710 con 0x55d17da0bd60 2023-10-17T01:17:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.640+0000 7f37f1a0ab80 10 client.15669 fetch_fsmap learned FSMap version 37 2023-10-17T01:17:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.640+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d17d7a8250 con 0x55d17da0bd60 2023-10-17T01:17:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37b6ffd700 1 -- 172.21.15.45:0/3627400670 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37cc099a30 con 0x55d17da0bd60 2023-10-17T01:17:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37b6ffd700 1 -- 172.21.15.45:0/3627400670 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f37cc0a0ff0 con 0x55d17da0bd60 2023-10-17T01:17:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37f1a0ab80 10 client.15669 fetch_fsmap finished waiting for FSMap version 37 2023-10-17T01:17:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37f1a0ab80 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:17:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37f1a0ab80 20 client.15669 populate_metadata read hostname 'smithi045' 2023-10-17T01:17:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da739c0 0x55d17da75de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:17:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37f1a0ab80 4 client.15669 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:17:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.641+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d17d94cd40 con 0x55d17da739c0 2023-10-17T01:17:59.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.643+0000 7f37d77fe700 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da739c0 0x55d17da75de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:17:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.647+0000 7f37d77fe700 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da739c0 0x55d17da75de0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449465efc31052ac server_cookie=39a4086919b9296c in_seq=0 out_seq=0 2023-10-17T01:17:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.647+0000 7f37b6ffd700 10 client.15669 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.649+0000 7f37b6ffd700 1 -- 172.21.15.45:0/3627400670 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d17d94cd40 con 0x55d17da739c0 2023-10-17T01:17:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.649+0000 7f37b6ffd700 10 client.15669 handle_command_reply: tid=0 2023-10-17T01:17:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.650+0000 7f37f1a0ab80 1 client.15669 shutdown 2023-10-17T01:17:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0bd60 msgr2=0x55d17da19000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0bd60 0x55d17da19000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f37cc099030 tx=0x7f37cc0109a0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 shutdown_connections 2023-10-17T01:17:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d17da0cae0 0x55d17da19540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d17da739c0 0x55d17da75de0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d17da0b400 0x55d17da18ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 --2- 172.21.15.45:0/3627400670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17da0bd60 0x55d17da19000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 >> 172.21.15.45:0/3627400670 conn(0x55d17d9620d0 msgr2=0x55d17da0c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37b6ffd700 0 client.15669 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:17:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 shutdown_connections 2023-10-17T01:17:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.653+0000 7f37f1a0ab80 1 -- 172.21.15.45:0/3627400670 wait complete. 2023-10-17T01:17:59.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.654+0000 7f37f1a0ab80 20 client.15669 trim_cache size 0 max 16384 2023-10-17T01:17:59.657 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:17:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:17:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:17:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:17:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:17:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:17:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:17:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:17:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:17:59.659 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:17:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.655+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f37c00a0b20 msgr2=0x7f37c00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.655+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f37c00a0b20 0x7f37c00a2fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f37d00070b0 tx=0x7f37d00093d0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.655+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 msgr2=0x7f37d806ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:17:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.655+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 0x7f37d806ed00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00b530 tx=0x7f37cc00b8f0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 shutdown_connections 2023-10-17T01:17:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f37c00a0b20 0x7f37c00a2fe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d8102940 0x7f37d8071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d8100db0 0x7f37d806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 --2- 172.21.15.45:0/1596965890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37d8101fa0 0x7f37d806ed00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:17:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 >> 172.21.15.45:0/1596965890 conn(0x7f37d80fc320 msgr2=0x7f37d80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:17:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 shutdown_connections 2023-10-17T01:17:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:17:59.656+0000 7f37ddfff700 1 -- 172.21.15.45:0/1596965890 wait complete. 2023-10-17T01:17:59.704 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:17:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:17:59 smithi181 ceph-mon[134520]: pgmap v418: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 865 op/s 2023-10-17T01:18:00.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:17:59 smithi045 ceph-mon[98474]: pgmap v418: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 865 op/s 2023-10-17T01:18:00.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:17:59 smithi060 ceph-mon[134136]: pgmap v418: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 865 op/s 2023-10-17T01:18:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 170.0 s, 2010.4 tps, lat 49.902 ms stddev 108.536 2023-10-17T01:18:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:01 smithi181 ceph-mon[134520]: pgmap v419: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-10-17T01:18:02.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:01 smithi045 ceph-mon[98474]: pgmap v419: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-10-17T01:18:02.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:01 smithi060 ceph-mon[134136]: pgmap v419: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-10-17T01:18:03.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:03 smithi181 ceph-mon[134520]: pgmap v420: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.19k op/s 2023-10-17T01:18:04.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:03 smithi045 ceph-mon[98474]: pgmap v420: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.19k op/s 2023-10-17T01:18:04.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:03 smithi060 ceph-mon[134136]: pgmap v420: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.19k op/s 2023-10-17T01:18:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 175.0 s, 2852.8 tps, lat 35.042 ms stddev 40.096 2023-10-17T01:18:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:05 smithi181 ceph-mon[134520]: pgmap v421: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 42 MiB/s wr, 1.86k op/s 2023-10-17T01:18:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:05 smithi045 ceph-mon[98474]: pgmap v421: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 42 MiB/s wr, 1.86k op/s 2023-10-17T01:18:06.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:05 smithi060 ceph-mon[134136]: pgmap v421: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 42 MiB/s wr, 1.86k op/s 2023-10-17T01:18:07.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:07 smithi181 ceph-mon[134520]: pgmap v422: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 1.78k op/s 2023-10-17T01:18:08.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:07 smithi045 ceph-mon[98474]: pgmap v422: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 1.78k op/s 2023-10-17T01:18:08.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:07 smithi060 ceph-mon[134136]: pgmap v422: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 1.78k op/s 2023-10-17T01:18:08.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:08 smithi181 ceph-mon[134520]: pgmap v423: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.79k op/s 2023-10-17T01:18:09.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:08 smithi045 ceph-mon[98474]: pgmap v423: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.79k op/s 2023-10-17T01:18:09.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:08 smithi060 ceph-mon[134136]: pgmap v423: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.79k op/s 2023-10-17T01:18:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 180.0 s, 2564.2 tps, lat 38.774 ms stddev 43.333 2023-10-17T01:18:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.476+0000 7f4b799ff700 1 -- 172.21.15.45:0/4033320635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74101850 msgr2=0x7f4b74103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.476+0000 7f4b799ff700 1 --2- 172.21.15.45:0/4033320635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74101850 0x7f4b74103c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009990 tx=0x7f4b64009ca0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 -- 172.21.15.45:0/4033320635 shutdown_connections 2023-10-17T01:18:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 --2- 172.21.15.45:0/4033320635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b74106ab0 0x7f4b74108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 --2- 172.21.15.45:0/4033320635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74101850 0x7f4b74103c40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 --2- 172.21.15.45:0/4033320635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74104180 0x7f4b74106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 -- 172.21.15.45:0/4033320635 >> 172.21.15.45:0/4033320635 conn(0x7f4b740fb1a0 msgr2=0x7f4b740fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 -- 172.21.15.45:0/4033320635 shutdown_connections 2023-10-17T01:18:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.477+0000 7f4b799ff700 1 -- 172.21.15.45:0/4033320635 wait complete. 2023-10-17T01:18:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 Processor -- start 2023-10-17T01:18:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 -- start start 2023-10-17T01:18:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 0x7f4b740ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74104180 0x7f4b740ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b74106ab0 0x7f4b74100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740711f0 con 0x7f4b74106ab0 2023-10-17T01:18:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74071360 con 0x7f4b74101850 2023-10-17T01:18:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.479+0000 7f4b799ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74071520 con 0x7f4b74104180 2023-10-17T01:18:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 0x7f4b740ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 0x7f4b740ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:34504/0 (socket says 172.21.15.45:34504) 2023-10-17T01:18:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 -- 172.21.15.45:0/744267366 learned_addr learned my addr 172.21.15.45:0/744267366 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b737fe700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b74106ab0 0x7f4b74100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b727fc700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74104180 0x7f4b740ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 -- 172.21.15.45:0/744267366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74104180 msgr2=0x7f4b740ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74104180 0x7f4b740ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 -- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b74106ab0 msgr2=0x7f4b74100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b74106ab0 0x7f4b74100280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b72ffd700 1 -- 172.21.15.45:0/744267366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640095f0 con 0x7f4b74101850 2023-10-17T01:18:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.480+0000 7f4b727fc700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74104180 0x7f4b740ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:18:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.481+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 0x7f4b740ff800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006010 tx=0x7f4b6400f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.481+0000 7f4b789fd700 1 -- 172.21.15.45:0/744267366 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6401d070 con 0x7f4b74101850 2023-10-17T01:18:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.481+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b740717a0 con 0x7f4b74101850 2023-10-17T01:18:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.481+0000 7f4b789fd700 1 -- 172.21.15.45:0/744267366 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b64004ca0 con 0x7f4b74101850 2023-10-17T01:18:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.481+0000 7f4b789fd700 1 -- 172.21.15.45:0/744267366 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64005590 con 0x7f4b74101850 2023-10-17T01:18:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.481+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b74071c40 con 0x7f4b74101850 2023-10-17T01:18:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.485+0000 7f4b789fd700 1 -- 172.21.15.45:0/744267366 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f4b6400fbf0 con 0x7f4b74101850 2023-10-17T01:18:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.487+0000 7f4b789fd700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b5c0a0bf0 0x7f4b5c0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.487+0000 7f4b789fd700 1 -- 172.21.15.45:0/744267366 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4b640cd440 con 0x7f4b74101850 2023-10-17T01:18:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.488+0000 7f4b8d367b80 1 Processor -- start 2023-10-17T01:18:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.489+0000 7f4b727fc700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b5c0a0bf0 0x7f4b5c0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.490+0000 7f4b727fc700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b5c0a0bf0 0x7f4b5c0a30b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b74100b00 tx=0x7f4b68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.490+0000 7f4b8d367b80 1 -- start start 2023-10-17T01:18:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.491+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x5617126285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.491+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712626e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.491+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712627870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.491+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617125769b0 con 0x5617126281b0 2023-10-17T01:18:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.491+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561712574370 con 0x561712628b10 2023-10-17T01:18:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.491+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171256fc60 con 0x5617126273e0 2023-10-17T01:18:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712626e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712627870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712627870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41386/0 (socket says 172.21.15.45:41386) 2023-10-17T01:18:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 -- 172.21.15.45:0/817286323 learned_addr learned my addr 172.21.15.45:0/817286323 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712626e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:34510/0 (socket says 172.21.15.45:34510) 2023-10-17T01:18:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x5617126285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 -- 172.21.15.45:0/817286323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 msgr2=0x561712626e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712626e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 -- 172.21.15.45:0/817286323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 msgr2=0x5617126285d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x5617126285d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b737fe700 1 -- 172.21.15.45:0/817286323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56171239dd80 con 0x5617126273e0 2023-10-17T01:18:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.492+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x5617126285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:18:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.493+0000 7f4b737fe700 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712627870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009440 tx=0x7f4b6c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed585694cfdb0baa server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.493+0000 7f4b59ffb700 1 -- 172.21.15.45:0/817286323 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00e070 con 0x5617126273e0 2023-10-17T01:18:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.493+0000 7f4b59ffb700 1 -- 172.21.15.45:0/817286323 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6c00f040 con 0x5617126273e0 2023-10-17T01:18:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.493+0000 7f4b59ffb700 1 -- 172.21.15.45:0/817286323 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c015b60 con 0x5617126273e0 2023-10-17T01:18:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 -- 172.21.15.45:0/817286323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 msgr2=0x561712627870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712627870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009440 tx=0x7f4b6c009920 comp rx=0 tx=0).stop 2023-10-17T01:18:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 -- 172.21.15.45:0/817286323 shutdown_connections 2023-10-17T01:18:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712627870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x5617126285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/817286323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712626e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.495+0000 7f4b8d367b80 1 -- 172.21.15.45:0/817286323 >> 172.21.15.45:0/817286323 conn(0x56171257e070 msgr2=0x5617126192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.496+0000 7f4b8d367b80 1 -- 172.21.15.45:0/817286323 shutdown_connections 2023-10-17T01:18:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.496+0000 7f4b8d367b80 1 -- 172.21.15.45:0/817286323 wait complete. 2023-10-17T01:18:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.496+0000 7f4b8d367b80 1 -- start start 2023-10-17T01:18:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.496+0000 7f4b8d367b80 1 Processor -- start 2023-10-17T01:18:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.497+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712630dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.497+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712631300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.497+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712638130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.497+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712631300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.497+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712630dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.497+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712630dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41398/0 (socket says 172.21.15.45:41398) 2023-10-17T01:18:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.498+0000 7f4b72ffd700 1 -- 172.21.15.45:0/4090523317 learned_addr learned my addr 172.21.15.45:0/4090523317 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.498+0000 7f4b737fe700 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712638130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.498+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171250c4b0 con 0x5617126281b0 2023-10-17T01:18:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.498+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171250c150 con 0x561712628b10 2023-10-17T01:18:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.499+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617123f38c0 con 0x5617126273e0 2023-10-17T01:18:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.499+0000 7f4b8d367b80 10 client.0 _ll_register_callbacks cb 0x561712459a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b737fe700 1 -- 172.21.15.45:0/4090523317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 msgr2=0x561712630dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b737fe700 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712630dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b737fe700 1 -- 172.21.15.45:0/4090523317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 msgr2=0x561712631300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b737fe700 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712631300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b737fe700 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c0090f0 con 0x561712628b10 2023-10-17T01:18:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b737fe700 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712638130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c014040 tx=0x7f4b6c016f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a42771a1a659e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.500+0000 7f4b52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:18:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.501+0000 7f4b52ffd700 1 -- 172.21.15.45:0/4090523317 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00e040 con 0x561712628b10 2023-10-17T01:18:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.501+0000 7f4b52ffd700 1 -- 172.21.15.45:0/4090523317 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4b6c004ce0 con 0x561712628b10 2023-10-17T01:18:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.501+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56171259acb0 con 0x561712628b10 2023-10-17T01:18:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.502+0000 7f4b52ffd700 1 -- 172.21.15.45:0/4090523317 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6c013550 con 0x561712628b10 2023-10-17T01:18:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.502+0000 7f4b8d367b80 10 client.25853 fetch_fsmap learned FSMap version 37 2023-10-17T01:18:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.502+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561712386c00 con 0x561712628b10 2023-10-17T01:18:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.503+0000 7f4b52ffd700 1 -- 172.21.15.45:0/4090523317 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56171259acb0 con 0x561712628b10 2023-10-17T01:18:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b52ffd700 1 -- 172.21.15.45:0/4090523317 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4b6c01c070 con 0x561712628b10 2023-10-17T01:18:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b8d367b80 10 client.25853 fetch_fsmap finished waiting for FSMap version 37 2023-10-17T01:18:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b8d367b80 10 client.25853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b8d367b80 20 client.25853 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x56171263ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b8d367b80 4 client.25853 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:18:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617125edcd0 con 0x56171263ca60 2023-10-17T01:18:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.504+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x56171263ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.505+0000 7f4b52ffd700 10 client.25853 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.505+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x56171263ee80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66dd432af58dc2f server_cookie=b161eceb628dc1b7 in_seq=0 out_seq=0 2023-10-17T01:18:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.506+0000 7f4b52ffd700 1 -- 172.21.15.45:0/4090523317 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5617125edcd0 con 0x56171263ca60 2023-10-17T01:18:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.506+0000 7f4b52ffd700 10 client.25853 handle_command_reply: tid=0 2023-10-17T01:18:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.506+0000 7f4b8d367b80 1 client.25853 shutdown 2023-10-17T01:18:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 msgr2=0x561712638130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712638130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c014040 tx=0x7f4b6c016f80 comp rx=0 tx=0).stop 2023-10-17T01:18:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 shutdown_connections 2023-10-17T01:18:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x56171263ee80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b52ffd700 0 client.25853 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712638130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712630dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/4090523317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712631300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.507+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 >> 172.21.15.45:0/4090523317 conn(0x56171257e070 msgr2=0x561712618a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.508+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 shutdown_connections 2023-10-17T01:18:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.508+0000 7f4b8d367b80 1 -- 172.21.15.45:0/4090523317 wait complete. 2023-10-17T01:18:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.508+0000 7f4b8d367b80 20 client.25853 trim_cache size 0 max 16384 2023-10-17T01:18:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.538+0000 7f4b8d367b80 1 Processor -- start 2023-10-17T01:18:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.538+0000 7f4b8d367b80 1 -- start start 2023-10-17T01:18:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712638b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617126281b0 0x561712630dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561712628b10 0x561712631300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561712619dc0 con 0x561712628b10 2023-10-17T01:18:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617126139e0 con 0x5617126281b0 2023-10-17T01:18:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561712630720 con 0x5617126273e0 2023-10-17T01:18:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712638b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617126281b0 0x561712630dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712638b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41404/0 (socket says 172.21.15.45:41404) 2023-10-17T01:18:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.539+0000 7f4b72ffd700 1 -- 172.21.15.45:0/3958324456 learned_addr learned my addr 172.21.15.45:0/3958324456 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b737fe700 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561712628b10 0x561712631300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b727fc700 1 -- 172.21.15.45:0/3958324456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 msgr2=0x561712638b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b727fc700 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712638b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b727fc700 1 -- 172.21.15.45:0/3958324456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561712628b10 msgr2=0x561712631300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b727fc700 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561712628b10 0x561712631300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b727fc700 1 -- 172.21.15.45:0/3958324456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617123c41f0 con 0x5617126281b0 2023-10-17T01:18:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.540+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712638b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:18:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.541+0000 7f4b727fc700 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617126281b0 0x561712630dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b68004540 tx=0x7f4b68013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5583c5c3264cdba server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.541+0000 7f4b5a7fc700 1 -- 172.21.15.45:0/3958324456 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68003880 con 0x5617126281b0 2023-10-17T01:18:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.541+0000 7f4b5a7fc700 1 -- 172.21.15.45:0/3958324456 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b68003ea0 con 0x5617126281b0 2023-10-17T01:18:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.541+0000 7f4b5a7fc700 1 -- 172.21.15.45:0/3958324456 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68021440 con 0x5617126281b0 2023-10-17T01:18:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.542+0000 7f4b8d367b80 1 -- 172.21.15.45:0/3958324456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617126281b0 msgr2=0x561712630dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.542+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617126281b0 0x561712630dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b68004540 tx=0x7f4b68013000 comp rx=0 tx=0).stop 2023-10-17T01:18:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.543+0000 7f4b8d367b80 1 -- 172.21.15.45:0/3958324456 shutdown_connections 2023-10-17T01:18:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.543+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561712628b10 0x561712631300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.543+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712638b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.543+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/3958324456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5617126281b0 0x561712630dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.543+0000 7f4b8d367b80 1 -- 172.21.15.45:0/3958324456 >> 172.21.15.45:0/3958324456 conn(0x56171257e070 msgr2=0x56171262e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.544+0000 7f4b8d367b80 1 -- 172.21.15.45:0/3958324456 shutdown_connections 2023-10-17T01:18:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.544+0000 7f4b8d367b80 1 -- 172.21.15.45:0/3958324456 wait complete. 2023-10-17T01:18:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.544+0000 7f4b8d367b80 1 -- start start 2023-10-17T01:18:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.545+0000 7f4b8d367b80 1 Processor -- start 2023-10-17T01:18:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712635980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712635ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712633f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561712634680 con 0x5617126281b0 2023-10-17T01:18:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617126347f0 con 0x561712628b10 2023-10-17T01:18:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617126349b0 con 0x5617126273e0 2023-10-17T01:18:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b8d367b80 10 client.0 _ll_register_callbacks cb 0x7f4b68016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712633f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712635980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712635980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41406/0 (socket says 172.21.15.45:41406) 2023-10-17T01:18:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.546+0000 7f4b72ffd700 1 -- 172.21.15.45:0/2995154917 learned_addr learned my addr 172.21.15.45:0/2995154917 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b72ffd700 1 -- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 msgr2=0x561712633f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712633f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b72ffd700 1 -- 172.21.15.45:0/2995154917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 msgr2=0x561712635ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b727fc700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712635ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712635ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b72ffd700 1 -- 172.21.15.45:0/2995154917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b680041f0 con 0x5617126273e0 2023-10-17T01:18:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.547+0000 7f4b727fc700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712635ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:18:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.548+0000 7f4b72ffd700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712635980 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b640975c0 tx=0x7f4b640cae40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54c8fdd1dbfd3245 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.548+0000 7f4b52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:18:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.548+0000 7f4b52ffd700 1 -- 172.21.15.45:0/2995154917 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640a2b30 con 0x5617126273e0 2023-10-17T01:18:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.548+0000 7f4b52ffd700 1 -- 172.21.15.45:0/2995154917 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4b64004e10 con 0x5617126273e0 2023-10-17T01:18:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.548+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c00f040 con 0x5617126273e0 2023-10-17T01:18:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b52ffd700 1 -- 172.21.15.45:0/2995154917 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6409adb0 con 0x5617126273e0 2023-10-17T01:18:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 10 client.25825 fetch_fsmap learned FSMap version 37 2023-10-17T01:18:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56171239c250 con 0x5617126273e0 2023-10-17T01:18:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b52ffd700 1 -- 172.21.15.45:0/2995154917 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6c00f040 con 0x5617126273e0 2023-10-17T01:18:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b52ffd700 1 -- 172.21.15.45:0/2995154917 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4b640a26a0 con 0x5617126273e0 2023-10-17T01:18:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 10 client.25825 fetch_fsmap finished waiting for FSMap version 37 2023-10-17T01:18:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 10 client.25825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 20 client.25825 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x5617126914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 4 client.25825 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:18:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.549+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561712568ce0 con 0x56171263ca60 2023-10-17T01:18:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.550+0000 7f4b727fc700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x5617126914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.551+0000 7f4b52ffd700 10 client.25825 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.550+0000 7f4b727fc700 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x5617126914e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a82a90aa645eb70 server_cookie=1c6f3c4023fccaf7 in_seq=0 out_seq=0 2023-10-17T01:18:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.567+0000 7f4b52ffd700 1 -- 172.21.15.45:0/2995154917 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561712568ce0 con 0x56171263ca60 2023-10-17T01:18:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.567+0000 7f4b52ffd700 10 client.25825 handle_command_reply: tid=0 2023-10-17T01:18:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 client.25825 shutdown 2023-10-17T01:18:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 msgr2=0x561712635980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712635980 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b640975c0 tx=0x7f4b640cae40 comp rx=0 tx=0).stop 2023-10-17T01:18:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 shutdown_connections 2023-10-17T01:18:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56171263ca60 0x5617126914e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561712628b10 0x561712633f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617126273e0 0x561712635980 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b8d367b80 1 --2- 172.21.15.45:0/2995154917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617126281b0 0x561712635ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.568+0000 7f4b52ffd700 0 client.25825 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.569+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 >> 172.21.15.45:0/2995154917 conn(0x56171257e070 msgr2=0x561712638b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.569+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 shutdown_connections 2023-10-17T01:18:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.569+0000 7f4b8d367b80 1 -- 172.21.15.45:0/2995154917 wait complete. 2023-10-17T01:18:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.570+0000 7f4b8d367b80 20 client.25825 trim_cache size 0 max 16384 2023-10-17T01:18:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:18:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:18:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:18:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:18:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b5c0a0bf0 msgr2=0x7f4b5c0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b5c0a0bf0 0x7f4b5c0a30b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b74100b00 tx=0x7f4b68009260 comp rx=0 tx=0).stop 2023-10-17T01:18:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 msgr2=0x7f4b740ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 0x7f4b740ff800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006010 tx=0x7f4b6400f460 comp rx=0 tx=0).stop 2023-10-17T01:18:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 shutdown_connections 2023-10-17T01:18:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4b5c0a0bf0 0x7f4b5c0a30b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b74106ab0 0x7f4b74100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b74101850 0x7f4b740ff800 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 --2- 172.21.15.45:0/744267366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b74104180 0x7f4b740ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 >> 172.21.15.45:0/744267366 conn(0x7f4b740fb1a0 msgr2=0x7f4b74102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 shutdown_connections 2023-10-17T01:18:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:11.571+0000 7f4b799ff700 1 -- 172.21.15.45:0/744267366 wait complete. 2023-10-17T01:18:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:11 smithi060 ceph-mon[134136]: pgmap v424: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 MiB/s wr, 1.96k op/s 2023-10-17T01:18:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:11 smithi181 ceph-mon[134520]: pgmap v424: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 MiB/s wr, 1.96k op/s 2023-10-17T01:18:12.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:11 smithi045 ceph-mon[98474]: pgmap v424: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 MiB/s wr, 1.96k op/s 2023-10-17T01:18:13.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:13 smithi181 ceph-mon[134520]: pgmap v425: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 1.68k op/s 2023-10-17T01:18:13.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:13 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:14.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:13 smithi045 ceph-mon[98474]: pgmap v425: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 1.68k op/s 2023-10-17T01:18:14.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:13 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:14.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:13 smithi060 ceph-mon[134136]: pgmap v425: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 1.68k op/s 2023-10-17T01:18:14.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:13 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 185.0 s, 2740.8 tps, lat 36.756 ms stddev 41.089 2023-10-17T01:18:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:15 smithi181 ceph-mon[134520]: pgmap v426: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 1.47k op/s 2023-10-17T01:18:16.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:15 smithi045 ceph-mon[98474]: pgmap v426: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 1.47k op/s 2023-10-17T01:18:16.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:15 smithi060 ceph-mon[134136]: pgmap v426: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 1.47k op/s 2023-10-17T01:18:16.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:16 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:17.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:16 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:17.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:16 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:17.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:17 smithi181 ceph-mon[134520]: pgmap v427: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 MiB/s wr, 768 op/s 2023-10-17T01:18:18.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:17 smithi045 ceph-mon[98474]: pgmap v427: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 MiB/s wr, 768 op/s 2023-10-17T01:18:18.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:17 smithi060 ceph-mon[134136]: pgmap v427: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 18 MiB/s wr, 768 op/s 2023-10-17T01:18:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:19 smithi181 ceph-mon[134520]: pgmap v428: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 755 op/s 2023-10-17T01:18:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:19 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:18:20.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:19 smithi045 ceph-mon[98474]: pgmap v428: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 755 op/s 2023-10-17T01:18:20.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:19 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:18:20.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:19 smithi060 ceph-mon[134136]: pgmap v428: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 755 op/s 2023-10-17T01:18:20.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:19 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:18:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 190.0 s, 1888.8 tps, lat 52.440 ms stddev 66.107 2023-10-17T01:18:20.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:20 smithi181 ceph-mon[134520]: pgmap v429: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 959 op/s 2023-10-17T01:18:21.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:20 smithi045 ceph-mon[98474]: pgmap v429: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 959 op/s 2023-10-17T01:18:21.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:20 smithi060 ceph-mon[134136]: pgmap v429: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 959 op/s 2023-10-17T01:18:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:18:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:18:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:18:22.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:18:22.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:18:22.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:18:22.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:18:22.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:18:22.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:18:22.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:22 smithi181 ceph-mon[134520]: pgmap v430: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 844 op/s 2023-10-17T01:18:23.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:22 smithi045 ceph-mon[98474]: pgmap v430: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 844 op/s 2023-10-17T01:18:23.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:22 smithi060 ceph-mon[134136]: pgmap v430: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 844 op/s 2023-10-17T01:18:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.819+0000 7f5fc93ff700 1 -- 172.21.15.45:0/4265112706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc4102fb0 msgr2=0x7f5fc41033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.819+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/4265112706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc4102fb0 0x7f5fc41033d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40099c0 tx=0x7f5fb4009cd0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 -- 172.21.15.45:0/4265112706 shutdown_connections 2023-10-17T01:18:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/4265112706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4104b40 0x7f5fc410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/4265112706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc4102fb0 0x7f5fc41033d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/4265112706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fc41041a0 0x7f5fc4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 -- 172.21.15.45:0/4265112706 >> 172.21.15.45:0/4265112706 conn(0x7f5fc40fe500 msgr2=0x7f5fc4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 -- 172.21.15.45:0/4265112706 shutdown_connections 2023-10-17T01:18:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.820+0000 7f5fc93ff700 1 -- 172.21.15.45:0/4265112706 wait complete. 2023-10-17T01:18:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.822+0000 7f5fc93ff700 1 Processor -- start 2023-10-17T01:18:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 -- start start 2023-10-17T01:18:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 0x7f5fc4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc41041a0 0x7f5fc41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fc4104b40 0x7f5fc4199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc419a5c0 con 0x7f5fc4102fb0 2023-10-17T01:18:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc419a730 con 0x7f5fc41041a0 2023-10-17T01:18:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.823+0000 7f5fc93ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc419a8f0 con 0x7f5fc4104b40 2023-10-17T01:18:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.824+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 0x7f5fc4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.824+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 0x7f5fc4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54030/0 (socket says 172.21.15.45:54030) 2023-10-17T01:18:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.824+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/843614768 learned_addr learned my addr 172.21.15.45:0/843614768 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.824+0000 7f5fc37fe700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fc4104b40 0x7f5fc4199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.824+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc41041a0 0x7f5fc41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.825+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/843614768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fc4104b40 msgr2=0x7f5fc4199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.825+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fc4104b40 0x7f5fc4199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.825+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/843614768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc41041a0 msgr2=0x7f5fc41990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.825+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc41041a0 0x7f5fc41990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.825+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/843614768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40095f0 con 0x7f5fc4102fb0 2023-10-17T01:18:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.825+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 0x7f5fc4198ba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4003680 tx=0x7f5fb400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.826+0000 7f5fabfff700 1 -- 172.21.15.45:0/843614768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb401d070 con 0x7f5fc4102fb0 2023-10-17T01:18:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.826+0000 7f5fabfff700 1 -- 172.21.15.45:0/843614768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb4005510 con 0x7f5fc4102fb0 2023-10-17T01:18:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.826+0000 7f5fabfff700 1 -- 172.21.15.45:0/843614768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4021410 con 0x7f5fc4102fb0 2023-10-17T01:18:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.826+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc419ab70 con 0x7f5fc4102fb0 2023-10-17T01:18:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.826+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc41a1390 con 0x7f5fc4102fb0 2023-10-17T01:18:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.832+0000 7f5fabfff700 1 -- 172.21.15.45:0/843614768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f5fb4005050 con 0x7f5fc4102fb0 2023-10-17T01:18:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.835+0000 7f5fabfff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5fac0a0c70 0x7f5fac0a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.835+0000 7f5fabfff700 1 -- 172.21.15.45:0/843614768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5fb40cd1f0 con 0x7f5fc4102fb0 2023-10-17T01:18:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.836+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5fac0a0c70 0x7f5fac0a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.838+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5fac0a0c70 0x7f5fac0a3130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009b30 tx=0x7f5fb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.839+0000 7f5fdcc65b80 1 Processor -- start 2023-10-17T01:18:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.839+0000 7f5fdcc65b80 1 -- start start 2023-10-17T01:18:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec414d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7d90 0x55ec414d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4142e9b0 con 0x55ec414d7d90 2023-10-17T01:18:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4142c370 con 0x55ec414d7190 2023-10-17T01:18:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec41427c60 con 0x55ec414d6830 2023-10-17T01:18:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35028/0 (socket says 172.21.15.45:35028) 2023-10-17T01:18:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/2336838055 learned_addr learned my addr 172.21.15.45:0/2336838055 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/2336838055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 msgr2=0x55ec414d7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec414d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec414d7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/2336838055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7d90 msgr2=0x55ec414d8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7d90 0x55ec414d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.840+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/2336838055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec41255d80 con 0x55ec414d6830 2023-10-17T01:18:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.841+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414d6c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4096e00 tx=0x7f5fb409a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bebc83ad9d76ec server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.841+0000 7f5fa97fa700 1 -- 172.21.15.45:0/2336838055 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40ba0f0 con 0x55ec414d6830 2023-10-17T01:18:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.841+0000 7f5fa97fa700 1 -- 172.21.15.45:0/2336838055 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb409b000 con 0x55ec414d6830 2023-10-17T01:18:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.841+0000 7f5fa97fa700 1 -- 172.21.15.45:0/2336838055 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40a2970 con 0x55ec414d6830 2023-10-17T01:18:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.842+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2336838055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 msgr2=0x55ec414d6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.842+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414d6c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4096e00 tx=0x7f5fb409a750 comp rx=0 tx=0).stop 2023-10-17T01:18:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.843+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2336838055 shutdown_connections 2023-10-17T01:18:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.843+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7d90 0x55ec414d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.843+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414d6c50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.843+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2336838055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec414d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.843+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2336838055 >> 172.21.15.45:0/2336838055 conn(0x55ec41436070 msgr2=0x55ec414d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.844+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2336838055 shutdown_connections 2023-10-17T01:18:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.844+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2336838055 wait complete. 2023-10-17T01:18:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.845+0000 7f5fdcc65b80 1 -- start start 2023-10-17T01:18:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.845+0000 7f5fdcc65b80 1 Processor -- start 2023-10-17T01:18:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d6830 0x55ec414e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 0x55ec414e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414efde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec413c44b0 con 0x55ec414d6830 2023-10-17T01:18:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec413c4150 con 0x55ec414d7d90 2023-10-17T01:18:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec412ab8c0 con 0x55ec414d7190 2023-10-17T01:18:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fc27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 0x55ec414e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fdcc65b80 10 client.0 _ll_register_callbacks cb 0x55ec41311a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fc27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 0x55ec414e8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35038/0 (socket says 172.21.15.45:35038) 2023-10-17T01:18:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.846+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2478692701 learned_addr learned my addr 172.21.15.45:0/2478692701 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc37fe700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d6830 0x55ec414e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 msgr2=0x55ec414efde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414efde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2478692701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d6830 msgr2=0x55ec414e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d6830 0x55ec414e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2478692701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc41a1390 con 0x55ec414d7190 2023-10-17T01:18:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc37fe700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414efde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:18:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 0x55ec414e8da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8010d30 tx=0x7f5fb8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=864b304b4b74abf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.847+0000 7f5fa6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:18:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.848+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2478692701 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8012e70 con 0x55ec414d7190 2023-10-17T01:18:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.849+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2478692701 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5fb801f420 con 0x55ec414d7190 2023-10-17T01:18:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.849+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec41452cb0 con 0x55ec414d7190 2023-10-17T01:18:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.850+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2478692701 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fb801f5e0 con 0x55ec414d7190 2023-10-17T01:18:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.850+0000 7f5fdcc65b80 10 client.25837 fetch_fsmap learned FSMap version 39 2023-10-17T01:18:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.850+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ec4123ec00 con 0x55ec414d7190 2023-10-17T01:18:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.850+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2478692701 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec41452cb0 con 0x55ec414d7190 2023-10-17T01:18:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.850+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2478692701 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f5fb8025020 con 0x55ec414d7190 2023-10-17T01:18:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fdcc65b80 10 client.25837 fetch_fsmap finished waiting for FSMap version 39 2023-10-17T01:18:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fdcc65b80 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fdcc65b80 20 client.25837 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec414f4740 0x55ec414f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fdcc65b80 4 client.25837 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:18:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec414a5cd0 con 0x55ec414f4740 2023-10-17T01:18:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.851+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec414f4740 0x55ec414f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.852+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec414f4740 0x55ec414f6b60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f68c102f70c8cb43 server_cookie=deb8f9b08941351d in_seq=0 out_seq=0 2023-10-17T01:18:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.852+0000 7f5fa6ff5700 10 client.25837 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.853+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2478692701 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ec414a5cd0 con 0x55ec414f4740 2023-10-17T01:18:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.853+0000 7f5fa6ff5700 10 client.25837 handle_command_reply: tid=0 2023-10-17T01:18:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.853+0000 7f5fdcc65b80 1 client.25837 shutdown 2023-10-17T01:18:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 msgr2=0x55ec414e8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 0x55ec414e8da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8010d30 tx=0x7f5fb8004750 comp rx=0 tx=0).stop 2023-10-17T01:18:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 shutdown_connections 2023-10-17T01:18:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec414f4740 0x55ec414f6b60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414efde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d6830 0x55ec414e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2478692701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7190 0x55ec414e8da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 >> 172.21.15.45:0/2478692701 conn(0x55ec41436070 msgr2=0x55ec414d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.854+0000 7f5fa6ff5700 0 client.25837 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.855+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 shutdown_connections 2023-10-17T01:18:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.855+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2478692701 wait complete. 2023-10-17T01:18:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.855+0000 7f5fdcc65b80 20 client.25837 trim_cache size 0 max 16384 2023-10-17T01:18:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.881+0000 7f5fdcc65b80 1 Processor -- start 2023-10-17T01:18:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.881+0000 7f5fdcc65b80 1 -- start start 2023-10-17T01:18:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7190 0x55ec414dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec414d1f80 con 0x55ec414d7190 2023-10-17T01:18:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec414cb9e0 con 0x55ec414d7d90 2023-10-17T01:18:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec414e83d0 con 0x55ec414d6830 2023-10-17T01:18:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35052/0 (socket says 172.21.15.45:35052) 2023-10-17T01:18:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/3639482690 learned_addr learned my addr 172.21.15.45:0/3639482690 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fc27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7190 0x55ec414dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.882+0000 7f5fc37fe700 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.883+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/3639482690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 msgr2=0x55ec414db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.883+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414db230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.883+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/3639482690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7190 msgr2=0x55ec414dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.883+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7190 0x55ec414dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.883+0000 7f5fc2ffd700 1 -- 172.21.15.45:0/3639482690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec4127c1f0 con 0x55ec414d6830 2023-10-17T01:18:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.883+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414f05b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cb090 tx=0x7f5fb409e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673a2ec09cc247d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.884+0000 7f5fa9ffb700 1 -- 172.21.15.45:0/3639482690 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4005680 con 0x55ec414d6830 2023-10-17T01:18:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.884+0000 7f5fa9ffb700 1 -- 172.21.15.45:0/3639482690 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb4005300 con 0x55ec414d6830 2023-10-17T01:18:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.884+0000 7f5fa9ffb700 1 -- 172.21.15.45:0/3639482690 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4099df0 con 0x55ec414d6830 2023-10-17T01:18:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.885+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/3639482690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 msgr2=0x55ec414f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.885+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414f05b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cb090 tx=0x7f5fb409e730 comp rx=0 tx=0).stop 2023-10-17T01:18:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/3639482690 shutdown_connections 2023-10-17T01:18:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7d90 0x55ec414db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d6830 0x55ec414f05b0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cb090 tx=0x7f5fb409e730 comp rx=0 tx=0).stop 2023-10-17T01:18:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/3639482690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec414d7190 0x55ec414dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/3639482690 >> 172.21.15.45:0/3639482690 conn(0x55ec41436070 msgr2=0x55ec414f6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/3639482690 shutdown_connections 2023-10-17T01:18:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.886+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/3639482690 wait complete. 2023-10-17T01:18:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.887+0000 7f5fdcc65b80 1 -- start start 2023-10-17T01:18:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.887+0000 7f5fdcc65b80 1 Processor -- start 2023-10-17T01:18:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec415400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 0x55ec415425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41542bd0 0x55ec41545060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec415457a0 con 0x55ec41542bd0 2023-10-17T01:18:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec41545910 con 0x55ec414d7190 2023-10-17T01:18:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec41545ad0 con 0x55ec414d7d90 2023-10-17T01:18:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.889+0000 7f5fdcc65b80 10 client.0 _ll_register_callbacks cb 0x7f5fb8010910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec415400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 0x55ec415425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 0x55ec415425f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35060/0 (socket says 172.21.15.45:35060) 2023-10-17T01:18:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2816336714 learned_addr learned my addr 172.21.15.45:0/2816336714 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec415400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:49996/0 (socket says 172.21.15.45:49996) 2023-10-17T01:18:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 msgr2=0x55ec415400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec415400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.890+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2816336714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41542bd0 msgr2=0x55ec41545060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.891+0000 7f5fc37fe700 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41542bd0 0x55ec41545060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.891+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41542bd0 0x55ec41545060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.891+0000 7f5fc27fc700 1 -- 172.21.15.45:0/2816336714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec41255d80 con 0x55ec414d7d90 2023-10-17T01:18:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.891+0000 7f5fc27fc700 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 0x55ec415425f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8031ad0 tx=0x7f5fb8013ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484936d25c389f72 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.891+0000 7f5fa6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:18:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2816336714 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8005240 con 0x55ec414d7d90 2023-10-17T01:18:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2816336714 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5fb80317b0 con 0x55ec414d7d90 2023-10-17T01:18:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fb409b000 con 0x55ec414d7d90 2023-10-17T01:18:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2816336714 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fb80053a0 con 0x55ec414d7d90 2023-10-17T01:18:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fdcc65b80 10 client.25843 fetch_fsmap learned FSMap version 39 2023-10-17T01:18:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ec41254250 con 0x55ec414d7d90 2023-10-17T01:18:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.892+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2816336714 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb409b000 con 0x55ec414d7d90 2023-10-17T01:18:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2816336714 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f5fb8043020 con 0x55ec414d7d90 2023-10-17T01:18:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fdcc65b80 10 client.25843 fetch_fsmap finished waiting for FSMap version 39 2023-10-17T01:18:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fdcc65b80 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fdcc65b80 20 client.25843 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec415473e0 0x55ec41549800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fdcc65b80 4 client.25843 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:18:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec41420ce0 con 0x55ec415473e0 2023-10-17T01:18:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.893+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec415473e0 0x55ec41549800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.894+0000 7f5fa6ff5700 10 client.25843 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.894+0000 7f5fc2ffd700 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec415473e0 0x55ec41549800 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c65b3fce0823591 server_cookie=7bf647c35beb9069 in_seq=0 out_seq=0 2023-10-17T01:18:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fa6ff5700 1 -- 172.21.15.45:0/2816336714 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec41420ce0 con 0x55ec415473e0 2023-10-17T01:18:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fa6ff5700 10 client.25843 handle_command_reply: tid=0 2023-10-17T01:18:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 client.25843 shutdown 2023-10-17T01:18:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 msgr2=0x55ec415425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 0x55ec415425f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8031ad0 tx=0x7f5fb8013ec0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 shutdown_connections 2023-10-17T01:18:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41542bd0 0x55ec41545060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ec415473e0 0x55ec41549800 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ec414d7190 0x55ec415400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 --2- 172.21.15.45:0/2816336714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ec414d7d90 0x55ec415425f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 >> 172.21.15.45:0/2816336714 conn(0x55ec41436070 msgr2=0x55ec414f05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.932+0000 7f5fa6ff5700 0 client.25843 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.933+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 shutdown_connections 2023-10-17T01:18:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:18:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:18:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:18:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:18:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.933+0000 7f5fdcc65b80 1 -- 172.21.15.45:0/2816336714 wait complete. 2023-10-17T01:18:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.933+0000 7f5fdcc65b80 20 client.25843 trim_cache size 0 max 16384 2023-10-17T01:18:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5fac0a0c70 msgr2=0x7f5fac0a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5fac0a0c70 0x7f5fac0a3130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009b30 tx=0x7f5fb8009260 comp rx=0 tx=0).stop 2023-10-17T01:18:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 msgr2=0x7f5fc4198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 0x7f5fc4198ba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4003680 tx=0x7f5fb400fab0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 shutdown_connections 2023-10-17T01:18:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5fac0a0c70 0x7f5fac0a3130 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fc4104b40 0x7f5fc4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc4102fb0 0x7f5fc4198ba0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 --2- 172.21.15.45:0/843614768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc41041a0 0x7f5fc41990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 >> 172.21.15.45:0/843614768 conn(0x7f5fc40fe500 msgr2=0x7f5fc41074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 shutdown_connections 2023-10-17T01:18:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:24.935+0000 7f5fc93ff700 1 -- 172.21.15.45:0/843614768 wait complete. 2023-10-17T01:18:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 195.0 s, 1837.4 tps, lat 54.395 ms stddev 62.790 2023-10-17T01:18:25.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:25 smithi181 ceph-mon[134520]: pgmap v431: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2023-10-17T01:18:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:25 smithi045 ceph-mon[98474]: pgmap v431: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2023-10-17T01:18:26.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:25 smithi060 ceph-mon[134136]: pgmap v431: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2023-10-17T01:18:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:26 smithi045 ceph-mon[98474]: pgmap v432: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 981 op/s 2023-10-17T01:18:27.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:26 smithi060 ceph-mon[134136]: pgmap v432: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 981 op/s 2023-10-17T01:18:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:26 smithi181 ceph-mon[134520]: pgmap v432: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 981 op/s 2023-10-17T01:18:29.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:18:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:29 smithi181 ceph-mon[134520]: pgmap v433: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 982 op/s 2023-10-17T01:18:30.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:29 smithi045 ceph-mon[98474]: pgmap v433: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 982 op/s 2023-10-17T01:18:30.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:29 smithi060 ceph-mon[134136]: pgmap v433: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 22 MiB/s wr, 982 op/s 2023-10-17T01:18:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.077+0000 7f25d13ff700 1 -- 172.21.15.45:0/1894079353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc1017c0 msgr2=0x7f25cc103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.077+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1894079353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc1017c0 0x7f25cc103bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009990 tx=0x7f25bc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 -- 172.21.15.45:0/1894079353 shutdown_connections 2023-10-17T01:18:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1894079353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc106a20 0x7f25cc108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1894079353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc1017c0 0x7f25cc103bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1894079353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 0x7f25cc1064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 -- 172.21.15.45:0/1894079353 >> 172.21.15.45:0/1894079353 conn(0x7f25cc0fb1b0 msgr2=0x7f25cc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 -- 172.21.15.45:0/1894079353 shutdown_connections 2023-10-17T01:18:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.078+0000 7f25d13ff700 1 -- 172.21.15.45:0/1894079353 wait complete. 2023-10-17T01:18:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.080+0000 7f25d13ff700 1 Processor -- start 2023-10-17T01:18:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.080+0000 7f25d13ff700 1 -- start start 2023-10-17T01:18:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.080+0000 7f25d13ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 0x7f25cc101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.080+0000 7f25d13ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 0x7f25cc0ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25d13ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc106a20 0x7f25cc0ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25d13ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1a6e50 con 0x7f25cc1040f0 2023-10-17T01:18:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25d13ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1001f0 con 0x7f25cc106a20 2023-10-17T01:18:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25d13ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1003b0 con 0x7f25cc1017c0 2023-10-17T01:18:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 0x7f25cc101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 0x7f25cc101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43348/0 (socket says 172.21.15.45:43348) 2023-10-17T01:18:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 -- 172.21.15.45:0/1019768119 learned_addr learned my addr 172.21.15.45:0/1019768119 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 0x7f25cc0ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc106a20 0x7f25cc0ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 -- 172.21.15.45:0/1019768119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc106a20 msgr2=0x7f25cc0ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc106a20 0x7f25cc0ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 -- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 msgr2=0x7f25cc0ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 0x7f25cc0ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.081+0000 7f25caffd700 1 -- 172.21.15.45:0/1019768119 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0095f0 con 0x7f25cc1017c0 2023-10-17T01:18:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 0x7f25cc0ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:18:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25caffd700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 0x7f25cc101130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005910 tx=0x7f25bc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25c3fff700 1 -- 172.21.15.45:0/1019768119 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc01d070 con 0x7f25cc1017c0 2023-10-17T01:18:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25c3fff700 1 -- 172.21.15.45:0/1019768119 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25bc00fb80 con 0x7f25cc1017c0 2023-10-17T01:18:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25c3fff700 1 -- 172.21.15.45:0/1019768119 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc017650 con 0x7f25cc1017c0 2023-10-17T01:18:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc071170 con 0x7f25cc1017c0 2023-10-17T01:18:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.082+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc071620 con 0x7f25cc1017c0 2023-10-17T01:18:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.088+0000 7f25c3fff700 1 -- 172.21.15.45:0/1019768119 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f25bc00fcf0 con 0x7f25cc1017c0 2023-10-17T01:18:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.089+0000 7f25e4cffb80 1 Processor -- start 2023-10-17T01:18:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 -- start start 2023-10-17T01:18:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 0x55d4ede8bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8d2b0 0x55d4ede8d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edde3a10 con 0x55d4ede8bba0 2023-10-17T01:18:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edde13d0 con 0x55d4ede8c500 2023-10-17T01:18:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.090+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edddccc0 con 0x55d4ede8d2b0 2023-10-17T01:18:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.091+0000 7f25cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 0x55d4ede8bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.091+0000 7f25cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 0x55d4ede8bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49248/0 (socket says 172.21.15.45:49248) 2023-10-17T01:18:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.091+0000 7f25cb7fe700 1 -- 172.21.15.45:0/690315128 learned_addr learned my addr 172.21.15.45:0/690315128 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25caffd700 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25cb7fe700 1 -- 172.21.15.45:0/690315128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8d2b0 msgr2=0x55d4ede8d730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:18:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8d2b0 0x55d4ede8d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25cb7fe700 1 -- 172.21.15.45:0/690315128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 msgr2=0x55d4ede8cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25c3fff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25b00a0ba0 0x7f25b00a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede8cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25cb7fe700 1 -- 172.21.15.45:0/690315128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4edc0ade0 con 0x55d4ede8bba0 2023-10-17T01:18:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25caffd700 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:18:30.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.092+0000 7f25c3fff700 1 -- 172.21.15.45:0/1019768119 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f25bc0cd3f0 con 0x7f25cc1017c0 2023-10-17T01:18:30.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.093+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 0x55d4ede8bfc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f25c4003fc0 tx=0x7f25c4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6ef20873ed1038 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:30.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.093+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25b00a0ba0 0x7f25b00a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.093+0000 7f25c17fa700 1 -- 172.21.15.45:0/690315128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c401a030 con 0x55d4ede8bba0 2023-10-17T01:18:30.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.094+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25b00a0ba0 0x7f25b00a3060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009fd0 tx=0x7f25c4015f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:30.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.094+0000 7f25c17fa700 1 -- 172.21.15.45:0/690315128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25c400f040 con 0x55d4ede8bba0 2023-10-17T01:18:30.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.094+0000 7f25c17fa700 1 -- 172.21.15.45:0/690315128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4015ca0 con 0x55d4ede8bba0 2023-10-17T01:18:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 -- 172.21.15.45:0/690315128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 msgr2=0x55d4ede8bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 0x55d4ede8bfc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f25c4003fc0 tx=0x7f25c4009ba0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 -- 172.21.15.45:0/690315128 shutdown_connections 2023-10-17T01:18:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8d2b0 0x55d4ede8d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8bba0 0x55d4ede8bfc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/690315128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede8cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.095+0000 7f25e4cffb80 1 -- 172.21.15.45:0/690315128 >> 172.21.15.45:0/690315128 conn(0x55d4eddeb0d0 msgr2=0x55d4ede86950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.096+0000 7f25e4cffb80 1 -- 172.21.15.45:0/690315128 shutdown_connections 2023-10-17T01:18:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.096+0000 7f25e4cffb80 1 -- 172.21.15.45:0/690315128 wait complete. 2023-10-17T01:18:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.096+0000 7f25e4cffb80 1 -- start start 2023-10-17T01:18:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.097+0000 7f25e4cffb80 1 Processor -- start 2023-10-17T01:18:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.097+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.097+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.097+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4ede9ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edd79510 con 0x55d4ede8d2b0 2023-10-17T01:18:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edd791b0 con 0x55d4ede8c500 2023-10-17T01:18:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edc60920 con 0x55d4ede8bba0 2023-10-17T01:18:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25e4cffb80 10 client.0 _ll_register_callbacks cb 0x55d4edcc6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25ca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25ca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33080/0 (socket says 172.21.15.45:33080) 2023-10-17T01:18:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25ca7fc700 1 -- 172.21.15.45:0/2426809201 learned_addr learned my addr 172.21.15.45:0/2426809201 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.098+0000 7f25caffd700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25ca7fc700 1 -- 172.21.15.45:0/2426809201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 msgr2=0x55d4ede958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25ca7fc700 1 -- 172.21.15.45:0/2426809201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 msgr2=0x55d4ede9ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4ede9ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25ca7fc700 1 -- 172.21.15.45:0/2426809201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c4003bf0 con 0x55d4ede8c500 2023-10-17T01:18:30.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25caffd700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:18:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.099+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25b80046e0 tx=0x7f25b800ea70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969b6a3510483859 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.100+0000 7f25ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:18:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.100+0000 7f25ae7fc700 1 -- 172.21.15.45:0/2426809201 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8010840 con 0x55d4ede8c500 2023-10-17T01:18:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.100+0000 7f25ae7fc700 1 -- 172.21.15.45:0/2426809201 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f25b80109a0 con 0x55d4ede8c500 2023-10-17T01:18:30.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.100+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4ede07d10 con 0x55d4ede8c500 2023-10-17T01:18:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.100+0000 7f25ae7fc700 1 -- 172.21.15.45:0/2426809201 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b8010b60 con 0x55d4ede8c500 2023-10-17T01:18:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 10 client.25883 fetch_fsmap learned FSMap version 39 2023-10-17T01:18:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d4edbf3c60 con 0x55d4ede8c500 2023-10-17T01:18:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25ae7fc700 1 -- 172.21.15.45:0/2426809201 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4ede07d10 con 0x55d4ede8c500 2023-10-17T01:18:30.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25ae7fc700 1 -- 172.21.15.45:0/2426809201 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f25b800f420 con 0x55d4ede8c500 2023-10-17T01:18:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 10 client.25883 fetch_fsmap finished waiting for FSMap version 39 2023-10-17T01:18:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 20 client.25883 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:30.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edea1640 0x55d4edea3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 4 client.25883 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:18:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.101+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4ede5ad30 con 0x55d4edea1640 2023-10-17T01:18:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.102+0000 7f25caffd700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edea1640 0x55d4edea3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.103+0000 7f25caffd700 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edea1640 0x55d4edea3a60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dde49adc1f771b7 server_cookie=f83076238a7a217a in_seq=0 out_seq=0 2023-10-17T01:18:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.103+0000 7f25ae7fc700 10 client.25883 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:30.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.103+0000 7f25ae7fc700 1 -- 172.21.15.45:0/2426809201 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d4ede5ad30 con 0x55d4edea1640 2023-10-17T01:18:30.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.103+0000 7f25ae7fc700 10 client.25883 handle_command_reply: tid=0 2023-10-17T01:18:30.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 client.25883 shutdown 2023-10-17T01:18:30.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 msgr2=0x55d4ede95e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25b80046e0 tx=0x7f25b800ea70 comp rx=0 tx=0).stop 2023-10-17T01:18:30.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 shutdown_connections 2023-10-17T01:18:30.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4ede9ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edea1640 0x55d4edea3a60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/2426809201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 >> 172.21.15.45:0/2426809201 conn(0x55d4eddeb0d0 msgr2=0x55d4ede86020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:30.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.104+0000 7f25ae7fc700 0 client.25883 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:30.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.105+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 shutdown_connections 2023-10-17T01:18:30.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.105+0000 7f25e4cffb80 1 -- 172.21.15.45:0/2426809201 wait complete. 2023-10-17T01:18:30.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.105+0000 7f25e4cffb80 20 client.25883 trim_cache size 0 max 16384 2023-10-17T01:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.138+0000 7f25e4cffb80 1 Processor -- start 2023-10-17T01:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.138+0000 7f25e4cffb80 1 -- start start 2023-10-17T01:18:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4ede90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ede8cbb0 con 0x55d4ede8d2b0 2023-10-17T01:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ede9d280 con 0x55d4ede8c500 2023-10-17T01:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ede94d90 con 0x55d4ede8bba0 2023-10-17T01:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43370/0 (socket says 172.21.15.45:43370) 2023-10-17T01:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25caffd700 1 -- 172.21.15.45:0/892346727 learned_addr learned my addr 172.21.15.45:0/892346727 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.140+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25caffd700 1 -- 172.21.15.45:0/892346727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 msgr2=0x55d4ede95e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25caffd700 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25caffd700 1 -- 172.21.15.45:0/892346727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 msgr2=0x55d4ede90260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25caffd700 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4ede90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25caffd700 1 -- 172.21.15.45:0/892346727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4edc31250 con 0x55d4ede8bba0 2023-10-17T01:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25caffd700 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00c010 tx=0x7f25bc0ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a08549e49dca6a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.141+0000 7f25c1ffb700 1 -- 172.21.15.45:0/892346727 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc09a460 con 0x55d4ede8bba0 2023-10-17T01:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.142+0000 7f25c1ffb700 1 -- 172.21.15.45:0/892346727 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25bc09b200 con 0x55d4ede8bba0 2023-10-17T01:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.142+0000 7f25c1ffb700 1 -- 172.21.15.45:0/892346727 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0d0d60 con 0x55d4ede8bba0 2023-10-17T01:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 -- 172.21.15.45:0/892346727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 msgr2=0x55d4ede958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00c010 tx=0x7f25bc0ba040 comp rx=0 tx=0).stop 2023-10-17T01:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 -- 172.21.15.45:0/892346727 shutdown_connections 2023-10-17T01:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4ede90260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4ede958e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/892346727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4ede95e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 -- 172.21.15.45:0/892346727 >> 172.21.15.45:0/892346727 conn(0x55d4eddeb0d0 msgr2=0x55d4ede9d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 -- 172.21.15.45:0/892346727 shutdown_connections 2023-10-17T01:18:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 -- 172.21.15.45:0/892346727 wait complete. 2023-10-17T01:18:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.143+0000 7f25e4cffb80 1 -- start start 2023-10-17T01:18:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 Processor -- start 2023-10-17T01:18:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4edeeb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4edeeb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4edef26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edef2bf0 con 0x55d4ede8d2b0 2023-10-17T01:18:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edef2d60 con 0x55d4ede8c500 2023-10-17T01:18:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.144+0000 7f25e4cffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4edef2f20 con 0x55d4ede8bba0 2023-10-17T01:18:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.145+0000 7f25e4cffb80 10 client.0 _ll_register_callbacks cb 0x55d4edcc6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.145+0000 7f25ca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4edeeb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.145+0000 7f25ca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4edeeb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33094/0 (socket says 172.21.15.45:33094) 2023-10-17T01:18:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.145+0000 7f25ca7fc700 1 -- 172.21.15.45:0/733686014 learned_addr learned my addr 172.21.15.45:0/733686014 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.145+0000 7f25caffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4edeeb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25cb7fe700 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4edef26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25caffd700 1 -- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 msgr2=0x55d4edeeb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25caffd700 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4edeeb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25caffd700 1 -- 172.21.15.45:0/733686014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 msgr2=0x55d4edef26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25caffd700 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4edef26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25caffd700 1 -- 172.21.15.45:0/733686014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc071620 con 0x55d4ede8bba0 2023-10-17T01:18:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.146+0000 7f25caffd700 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4edeeb1e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0ce000 tx=0x7f25bc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918043f631dce7e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.147+0000 7f25ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:18:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.147+0000 7f25ae7fc700 1 -- 172.21.15.45:0/733686014 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc09a1f0 con 0x55d4ede8bba0 2023-10-17T01:18:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.148+0000 7f25ae7fc700 1 -- 172.21.15.45:0/733686014 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f25bc09e930 con 0x55d4ede8bba0 2023-10-17T01:18:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.148+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25c400f040 con 0x55d4ede8bba0 2023-10-17T01:18:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25ae7fc700 1 -- 172.21.15.45:0/733686014 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25bc09ab00 con 0x55d4ede8bba0 2023-10-17T01:18:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 10 client.25858 fetch_fsmap learned FSMap version 39 2023-10-17T01:18:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d4edc092b0 con 0x55d4ede8bba0 2023-10-17T01:18:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25ae7fc700 1 -- 172.21.15.45:0/733686014 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25c400f040 con 0x55d4ede8bba0 2023-10-17T01:18:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25ae7fc700 1 -- 172.21.15.45:0/733686014 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f25bc0b7070 con 0x55d4ede8bba0 2023-10-17T01:18:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 10 client.25858 fetch_fsmap finished waiting for FSMap version 39 2023-10-17T01:18:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 20 client.25858 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edef44a0 0x55d4edef68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 4 client.25858 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:18:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.149+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4eddd5d40 con 0x55d4edef44a0 2023-10-17T01:18:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.150+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edef44a0 0x55d4edef68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:30.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25ca7fc700 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edef44a0 0x55d4edef68c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322c69b33aefeac server_cookie=434f31cd7b61d98b in_seq=0 out_seq=0 2023-10-17T01:18:30.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25ae7fc700 10 client.25858 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:30.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25ae7fc700 1 -- 172.21.15.45:0/733686014 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d4eddd5d40 con 0x55d4edef44a0 2023-10-17T01:18:30.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25ae7fc700 10 client.25858 handle_command_reply: tid=0 2023-10-17T01:18:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25e4cffb80 1 client.25858 shutdown 2023-10-17T01:18:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 msgr2=0x55d4edeeb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.151+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4edeeb1e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0ce000 tx=0x7f25bc009310 comp rx=0 tx=0).stop 2023-10-17T01:18:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 shutdown_connections 2023-10-17T01:18:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ede8d2b0 0x55d4edef26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d4ede8bba0 0x55d4edeeb1e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d4edef44a0 0x55d4edef68c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 --2- 172.21.15.45:0/733686014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4ede8c500 0x55d4edeeb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 >> 172.21.15.45:0/733686014 conn(0x55d4eddeb0d0 msgr2=0x55d4ede95880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25ae7fc700 0 client.25858 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 shutdown_connections 2023-10-17T01:18:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 1 -- 172.21.15.45:0/733686014 wait complete. 2023-10-17T01:18:30.163 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:18:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:18:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:18:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:18:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:18:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:18:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:18:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:18:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:18:30.165 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:18:30.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.152+0000 7f25e4cffb80 20 client.25858 trim_cache size 0 max 16384 2023-10-17T01:18:30.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25b00a0ba0 msgr2=0x7f25b00a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25b00a0ba0 0x7f25b00a3060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009fd0 tx=0x7f25c4015f90 comp rx=0 tx=0).stop 2023-10-17T01:18:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 msgr2=0x7f25cc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 0x7f25cc101130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005910 tx=0x7f25bc00f690 comp rx=0 tx=0).stop 2023-10-17T01:18:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 shutdown_connections 2023-10-17T01:18:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f25b00a0ba0 0x7f25b00a3060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc106a20 0x7f25cc0ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25cc1017c0 0x7f25cc101130 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005910 tx=0x7f25bc00f690 comp rx=0 tx=0).stop 2023-10-17T01:18:30.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 --2- 172.21.15.45:0/1019768119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25cc1040f0 0x7f25cc0ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:30.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.153+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 >> 172.21.15.45:0/1019768119 conn(0x7f25cc0fb1b0 msgr2=0x7f25cc102430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:30.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.154+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 shutdown_connections 2023-10-17T01:18:30.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:18:30.154+0000 7f25d13ff700 1 -- 172.21.15.45:0/1019768119 wait complete. 2023-10-17T01:18:30.184 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:18:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 200.0 s, 1782.6 tps, lat 55.383 ms stddev 59.629 2023-10-17T01:18:31.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:30 smithi045 ceph-mon[98474]: pgmap v434: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 26 MiB/s wr, 1.14k op/s 2023-10-17T01:18:31.085 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:30 smithi060 ceph-mon[134136]: pgmap v434: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 26 MiB/s wr, 1.14k op/s 2023-10-17T01:18:31.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:30 smithi181 ceph-mon[134520]: pgmap v434: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 26 MiB/s wr, 1.14k op/s 2023-10-17T01:18:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:33 smithi181 ceph-mon[134520]: pgmap v435: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.24k op/s 2023-10-17T01:18:34.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:33 smithi045 ceph-mon[98474]: pgmap v435: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.24k op/s 2023-10-17T01:18:34.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:33 smithi060 ceph-mon[134136]: pgmap v435: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.24k op/s 2023-10-17T01:18:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 205.0 s, 1949.7 tps, lat 52.274 ms stddev 89.485 2023-10-17T01:18:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:35 smithi181 ceph-mon[134520]: pgmap v436: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-10-17T01:18:36.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:35 smithi045 ceph-mon[98474]: pgmap v436: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-10-17T01:18:36.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:35 smithi060 ceph-mon[134136]: pgmap v436: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-10-17T01:18:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:36 smithi045 ceph-mon[98474]: pgmap v437: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 41 MiB/s wr, 1.97k op/s 2023-10-17T01:18:37.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:36 smithi060 ceph-mon[134136]: pgmap v437: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 41 MiB/s wr, 1.97k op/s 2023-10-17T01:18:37.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:36 smithi181 ceph-mon[134520]: pgmap v437: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 41 MiB/s wr, 1.97k op/s 2023-10-17T01:18:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.191+0000 7f27133ff700 1 -- 172.21.15.45:0/1784373219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 msgr2=0x7f270c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.191+0000 7f27133ff700 1 --2- 172.21.15.45:0/1784373219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 0x7f270c073180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f270400b0a0 tx=0x7f270400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.192+0000 7f27133ff700 1 -- 172.21.15.45:0/1784373219 shutdown_connections 2023-10-17T01:18:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.192+0000 7f27133ff700 1 --2- 172.21.15.45:0/1784373219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c108c80 0x7f270c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.192+0000 7f27133ff700 1 --2- 172.21.15.45:0/1784373219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 0x7f270c073180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.192+0000 7f27133ff700 1 --2- 172.21.15.45:0/1784373219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f270c073750 0x7f270c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.192+0000 7f27133ff700 1 -- 172.21.15.45:0/1784373219 >> 172.21.15.45:0/1784373219 conn(0x7f270c0fc300 msgr2=0x7f270c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.192+0000 7f27133ff700 1 -- 172.21.15.45:0/1784373219 shutdown_connections 2023-10-17T01:18:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.193+0000 7f27133ff700 1 -- 172.21.15.45:0/1784373219 wait complete. 2023-10-17T01:18:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.194+0000 7f27133ff700 1 Processor -- start 2023-10-17T01:18:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.194+0000 7f27133ff700 1 -- start start 2023-10-17T01:18:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f27133ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 0x7f270c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f27133ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 0x7f270c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f27133ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f270c108c80 0x7f270c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f27133ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c19a920 con 0x7f270c108c80 2023-10-17T01:18:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f27133ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c19aa90 con 0x7f270c073750 2023-10-17T01:18:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f27133ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c19ac50 con 0x7f270c074d30 2023-10-17T01:18:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f271119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 0x7f270c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f271119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 0x7f270c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46200/0 (socket says 172.21.15.45:46200) 2023-10-17T01:18:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f271119b700 1 -- 172.21.15.45:0/1015578381 learned_addr learned my addr 172.21.15.45:0/1015578381 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.195+0000 7f271099a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 0x7f270c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271199c700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f270c108c80 0x7f270c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271119b700 1 -- 172.21.15.45:0/1015578381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 msgr2=0x7f270c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271119b700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 0x7f270c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271119b700 1 -- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f270c108c80 msgr2=0x7f270c19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271119b700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f270c108c80 0x7f270c19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271119b700 1 -- 172.21.15.45:0/1015578381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704009d00 con 0x7f270c073750 2023-10-17T01:18:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.196+0000 7f271119b700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 0x7f270c198f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2704007ab0 tx=0x7f2704004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.197+0000 7f26fe7fc700 1 -- 172.21.15.45:0/1015578381 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270400e040 con 0x7f270c073750 2023-10-17T01:18:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.197+0000 7f26fe7fc700 1 -- 172.21.15.45:0/1015578381 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2704005710 con 0x7f270c073750 2023-10-17T01:18:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.197+0000 7f26fe7fc700 1 -- 172.21.15.45:0/1015578381 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270401dbc0 con 0x7f270c073750 2023-10-17T01:18:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.197+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c19aed0 con 0x7f270c073750 2023-10-17T01:18:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.197+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c100a20 con 0x7f270c073750 2023-10-17T01:18:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.205+0000 7f2726e22b80 1 Processor -- start 2023-10-17T01:18:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.205+0000 7f26fe7fc700 1 -- 172.21.15.45:0/1015578381 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2704005880 con 0x7f270c073750 2023-10-17T01:18:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.206+0000 7f2726e22b80 1 -- start start 2023-10-17T01:18:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.206+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 0x55982e53ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.206+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53b190 0x55982e53b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.207+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e53c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.207+0000 7f2726e22b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e4929b0 con 0x55982e53b190 2023-10-17T01:18:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.207+0000 7f2726e22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e490370 con 0x55982e53bd90 2023-10-17T01:18:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.207+0000 7f2726e22b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e48bc60 con 0x55982e53a830 2023-10-17T01:18:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 0x55982e53ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 0x55982e53ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47728/0 (socket says 172.21.15.45:47728) 2023-10-17T01:18:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 -- 172.21.15.45:0/3219628746 learned_addr learned my addr 172.21.15.45:0/3219628746 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271119b700 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53b190 0x55982e53b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 -- 172.21.15.45:0/3219628746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 msgr2=0x55982e53c210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:18:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e53c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 -- 172.21.15.45:0/3219628746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53b190 msgr2=0x55982e53b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53b190 0x55982e53b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f26fe7fc700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26f80a0ba0 0x7f26f80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271199c700 1 -- 172.21.15.45:0/3219628746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982e2b9d80 con 0x55982e53a830 2023-10-17T01:18:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.208+0000 7f271119b700 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53b190 0x55982e53b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:18:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.209+0000 7f26fe7fc700 1 -- 172.21.15.45:0/1015578381 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f27040cd9c0 con 0x7f270c073750 2023-10-17T01:18:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.209+0000 7f271199c700 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 0x55982e53ac50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f270000b810 tx=0x7f270000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f6e53ac6ad8657 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.209+0000 7f26f77fe700 1 -- 172.21.15.45:0/3219628746 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700014620 con 0x55982e53a830 2023-10-17T01:18:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.209+0000 7f26f77fe700 1 -- 172.21.15.45:0/3219628746 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2700014c40 con 0x55982e53a830 2023-10-17T01:18:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.209+0000 7f26f77fe700 1 -- 172.21.15.45:0/3219628746 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700012d10 con 0x55982e53a830 2023-10-17T01:18:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.209+0000 7f271099a700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26f80a0ba0 0x7f26f80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 -- 172.21.15.45:0/3219628746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 msgr2=0x55982e53ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f271099a700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26f80a0ba0 0x7f26f80a3060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55982e53c750 tx=0x7f2708017040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 0x55982e53ac50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f270000b810 tx=0x7f270000bb20 comp rx=0 tx=0).stop 2023-10-17T01:18:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 -- 172.21.15.45:0/3219628746 shutdown_connections 2023-10-17T01:18:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e53c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53a830 0x55982e53ac50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 --2- 172.21.15.45:0/3219628746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53b190 0x55982e53b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 -- 172.21.15.45:0/3219628746 >> 172.21.15.45:0/3219628746 conn(0x55982e49a070 msgr2=0x55982e5352c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 -- 172.21.15.45:0/3219628746 shutdown_connections 2023-10-17T01:18:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.210+0000 7f2726e22b80 1 -- 172.21.15.45:0/3219628746 wait complete. 2023-10-17T01:18:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.211+0000 7f2726e22b80 1 -- start start 2023-10-17T01:18:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.212+0000 7f2726e22b80 1 Processor -- start 2023-10-17T01:18:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53a830 0x55982e544380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e5448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e54b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e4284b0 con 0x55982e53a830 2023-10-17T01:18:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e428150 con 0x55982e53bd90 2023-10-17T01:18:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e30f8c0 con 0x55982e53b190 2023-10-17T01:18:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f2726e22b80 10 client.0 _ll_register_callbacks cb 0x55982e375a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f271199c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e54b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f271099a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e5448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f271199c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e54b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46208/0 (socket says 172.21.15.45:46208) 2023-10-17T01:18:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f271099a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e5448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47734/0 (socket says 172.21.15.45:47734) 2023-10-17T01:18:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.213+0000 7f271199c700 1 -- 172.21.15.45:0/1365028586 learned_addr learned my addr 172.21.15.45:0/1365028586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271119b700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53a830 0x55982e544380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271199c700 1 -- 172.21.15.45:0/1365028586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 msgr2=0x55982e5448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271199c700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e5448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271199c700 1 -- 172.21.15.45:0/1365028586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53a830 msgr2=0x55982e544380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271199c700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53a830 0x55982e544380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271199c700 1 -- 172.21.15.45:0/1365028586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270000e040 con 0x55982e53bd90 2023-10-17T01:18:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271119b700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53a830 0x55982e544380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:18:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.214+0000 7f271199c700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e54b900 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2700012480 tx=0x7f27000124b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ab05679d363c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.215+0000 7f26f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:18:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.216+0000 7f26f4ff9700 1 -- 172.21.15.45:0/1365028586 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700012e20 con 0x55982e53bd90 2023-10-17T01:18:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.216+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55982e4b6cb0 con 0x55982e53bd90 2023-10-17T01:18:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.216+0000 7f26f4ff9700 1 -- 172.21.15.45:0/1365028586 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2700014620 con 0x55982e53bd90 2023-10-17T01:18:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.217+0000 7f26f4ff9700 1 -- 172.21.15.45:0/1365028586 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27000147e0 con 0x55982e53bd90 2023-10-17T01:18:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.217+0000 7f2726e22b80 10 client.25895 fetch_fsmap learned FSMap version 39 2023-10-17T01:18:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.217+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55982e2a2c00 con 0x55982e53bd90 2023-10-17T01:18:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.218+0000 7f26f4ff9700 1 -- 172.21.15.45:0/1365028586 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55982e4b6cb0 con 0x55982e53bd90 2023-10-17T01:18:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.218+0000 7f26f4ff9700 1 -- 172.21.15.45:0/1365028586 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f270001b070 con 0x55982e53bd90 2023-10-17T01:18:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f2726e22b80 10 client.25895 fetch_fsmap finished waiting for FSMap version 39 2023-10-17T01:18:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f2726e22b80 10 client.25895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f2726e22b80 20 client.25895 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f2726e22b80 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e561080 0x55982e5634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f2726e22b80 4 client.25895 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:18:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55982e509cd0 con 0x55982e561080 2023-10-17T01:18:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.219+0000 7f271119b700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e561080 0x55982e5634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.220+0000 7f26f4ff9700 10 client.25895 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.220+0000 7f271119b700 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e561080 0x55982e5634a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c8b698d576d657 server_cookie=e1c91e562eac8098 in_seq=0 out_seq=0 2023-10-17T01:18:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.221+0000 7f26f4ff9700 1 -- 172.21.15.45:0/1365028586 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55982e509cd0 con 0x55982e561080 2023-10-17T01:18:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.221+0000 7f26f4ff9700 10 client.25895 handle_command_reply: tid=0 2023-10-17T01:18:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.221+0000 7f2726e22b80 1 client.25895 shutdown 2023-10-17T01:18:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.222+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 msgr2=0x55982e54b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.222+0000 7f2726e22b80 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e54b900 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2700012480 tx=0x7f27000124b0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 shutdown_connections 2023-10-17T01:18:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e561080 0x55982e5634a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53bd90 0x55982e54b900 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53a830 0x55982e544380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 --2- 172.21.15.45:0/1365028586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e5448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 >> 172.21.15.45:0/1365028586 conn(0x55982e49a070 msgr2=0x55982e534a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f26f4ff9700 0 client.25895 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.223+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 shutdown_connections 2023-10-17T01:18:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.224+0000 7f2726e22b80 1 -- 172.21.15.45:0/1365028586 wait complete. 2023-10-17T01:18:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.224+0000 7f2726e22b80 20 client.25895 trim_cache size 0 max 16384 2023-10-17T01:18:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.257+0000 7f2726e22b80 1 Processor -- start 2023-10-17T01:18:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.258+0000 7f2726e22b80 1 -- start start 2023-10-17T01:18:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.260+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e536360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.260+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e543030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.260+0000 7f271119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e536360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.260+0000 7f271119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e536360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46218/0 (socket says 172.21.15.45:46218) 2023-10-17T01:18:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e53ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f2726e22b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e532010 con 0x55982e53bd90 2023-10-17T01:18:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f2726e22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e542c60 con 0x55982e53a830 2023-10-17T01:18:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f2726e22b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e542970 con 0x55982e53b190 2023-10-17T01:18:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f271119b700 1 -- 172.21.15.45:0/468800145 learned_addr learned my addr 172.21.15.45:0/468800145 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f271119b700 1 -- 172.21.15.45:0/468800145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 msgr2=0x55982e543030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f271119b700 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e543030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.261+0000 7f271119b700 1 -- 172.21.15.45:0/468800145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 msgr2=0x55982e53ed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:18:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.262+0000 7f271119b700 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e53ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.263+0000 7f271119b700 1 -- 172.21.15.45:0/468800145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982e2e01f0 con 0x55982e53a830 2023-10-17T01:18:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.263+0000 7f271119b700 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e536360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f270400b830 tx=0x7f27040cb3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642e6690b3e74646 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.264+0000 7f26f7fff700 1 -- 172.21.15.45:0/468800145 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270409b7d0 con 0x55982e53a830 2023-10-17T01:18:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.264+0000 7f26f7fff700 1 -- 172.21.15.45:0/468800145 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f270409b0d0 con 0x55982e53a830 2023-10-17T01:18:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.264+0000 7f26f7fff700 1 -- 172.21.15.45:0/468800145 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704099470 con 0x55982e53a830 2023-10-17T01:18:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 -- 172.21.15.45:0/468800145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 msgr2=0x55982e536360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e536360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f270400b830 tx=0x7f27040cb3c0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 -- 172.21.15.45:0/468800145 shutdown_connections 2023-10-17T01:18:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e53ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e536360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 --2- 172.21.15.45:0/468800145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e543030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.266+0000 7f2726e22b80 1 -- 172.21.15.45:0/468800145 >> 172.21.15.45:0/468800145 conn(0x55982e49a070 msgr2=0x55982e542260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.268+0000 7f2726e22b80 1 -- 172.21.15.45:0/468800145 shutdown_connections 2023-10-17T01:18:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.269+0000 7f2726e22b80 1 -- 172.21.15.45:0/468800145 wait complete. 2023-10-17T01:18:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.269+0000 7f2726e22b80 1 -- start start 2023-10-17T01:18:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.270+0000 7f2726e22b80 1 Processor -- start 2023-10-17T01:18:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.271+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e548fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.272+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e549520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.272+0000 7f2726e22b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e5b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.272+0000 7f271119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e548fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.272+0000 7f271119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e548fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:46222/0 (socket says 172.21.15.45:46222) 2023-10-17T01:18:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.272+0000 7f271099a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e549520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.272+0000 7f271099a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e549520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47764/0 (socket says 172.21.15.45:47764) 2023-10-17T01:18:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.273+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e5b1610 con 0x55982e53bd90 2023-10-17T01:18:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.273+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e5b1750 con 0x55982e53a830 2023-10-17T01:18:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.273+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982e5b18e0 con 0x55982e53b190 2023-10-17T01:18:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.273+0000 7f271119b700 1 -- 172.21.15.45:0/2329031664 learned_addr learned my addr 172.21.15.45:0/2329031664 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.273+0000 7f2726e22b80 10 client.0 _ll_register_callbacks cb 0x7f270000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.273+0000 7f271199c700 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e5b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f271119b700 1 -- 172.21.15.45:0/2329031664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 msgr2=0x55982e549520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f271119b700 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e549520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f271119b700 1 -- 172.21.15.45:0/2329031664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 msgr2=0x55982e5b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f271119b700 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e5b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f271119b700 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c100a20 con 0x55982e53a830 2023-10-17T01:18:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f271119b700 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e548fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2704005b30 tx=0x7f27040cb480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8fad4aa1357e632 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.274+0000 7f26f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:18:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.275+0000 7f26f4ff9700 1 -- 172.21.15.45:0/2329031664 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704005450 con 0x55982e53a830 2023-10-17T01:18:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.275+0000 7f26f4ff9700 1 -- 172.21.15.45:0/2329031664 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f270409b7d0 con 0x55982e53a830 2023-10-17T01:18:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.275+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2700014c40 con 0x55982e53a830 2023-10-17T01:18:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.275+0000 7f26f4ff9700 1 -- 172.21.15.45:0/2329031664 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f270409b330 con 0x55982e53a830 2023-10-17T01:18:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.275+0000 7f2726e22b80 10 client.25901 fetch_fsmap learned FSMap version 39 2023-10-17T01:18:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.276+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55982e2b8250 con 0x55982e53a830 2023-10-17T01:18:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.276+0000 7f26f4ff9700 1 -- 172.21.15.45:0/2329031664 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2700014c40 con 0x55982e53a830 2023-10-17T01:18:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.276+0000 7f26f4ff9700 1 -- 172.21.15.45:0/2329031664 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f270401dd20 con 0x55982e53a830 2023-10-17T01:18:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f2726e22b80 10 client.25901 fetch_fsmap finished waiting for FSMap version 39 2023-10-17T01:18:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f2726e22b80 10 client.25901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f2726e22b80 20 client.25901 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f2726e22b80 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e5b2df0 0x55982e5b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f2726e22b80 4 client.25901 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:18:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55982e484ce0 con 0x55982e5b2df0 2023-10-17T01:18:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.277+0000 7f271099a700 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e5b2df0 0x55982e5b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.278+0000 7f271099a700 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e5b2df0 0x55982e5b51e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b17121002fdf7f server_cookie=71e38ac05bc85a36 in_seq=0 out_seq=0 2023-10-17T01:18:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.278+0000 7f26f4ff9700 10 client.25901 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.294+0000 7f26f4ff9700 1 -- 172.21.15.45:0/2329031664 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55982e484ce0 con 0x55982e5b2df0 2023-10-17T01:18:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.294+0000 7f26f4ff9700 10 client.25901 handle_command_reply: tid=0 2023-10-17T01:18:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.295+0000 7f2726e22b80 1 client.25901 shutdown 2023-10-17T01:18:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.295+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 msgr2=0x55982e548fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.295+0000 7f2726e22b80 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e548fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2704005b30 tx=0x7f27040cb480 comp rx=0 tx=0).stop 2023-10-17T01:18:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 shutdown_connections 2023-10-17T01:18:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55982e53bd90 0x55982e5b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55982e5b2df0 0x55982e5b51e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55982e53a830 0x55982e548fe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 --2- 172.21.15.45:0/2329031664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55982e53b190 0x55982e549520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 >> 172.21.15.45:0/2329031664 conn(0x55982e49a070 msgr2=0x55982e543030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f26f4ff9700 0 client.25901 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 shutdown_connections 2023-10-17T01:18:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.296+0000 7f2726e22b80 1 -- 172.21.15.45:0/2329031664 wait complete. 2023-10-17T01:18:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.297+0000 7f2726e22b80 20 client.25901 trim_cache size 0 max 16384 2023-10-17T01:18:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:18:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:18:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:18:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:18:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.297+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26f80a0ba0 msgr2=0x7f26f80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26f80a0ba0 0x7f26f80a3060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55982e53c750 tx=0x7f2708017040 comp rx=0 tx=0).stop 2023-10-17T01:18:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 msgr2=0x7f270c198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 0x7f270c198f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2704007ab0 tx=0x7f2704004d00 comp rx=0 tx=0).stop 2023-10-17T01:18:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 shutdown_connections 2023-10-17T01:18:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26f80a0ba0 0x7f26f80a3060 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f270c108c80 0x7f270c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f270c073750 0x7f270c198f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 --2- 172.21.15.45:0/1015578381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c074d30 0x7f270c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 >> 172.21.15.45:0/1015578381 conn(0x7f270c0fc300 msgr2=0x7f270c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 shutdown_connections 2023-10-17T01:18:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:38.298+0000 7f27133ff700 1 -- 172.21.15.45:0/1015578381 wait complete. 2023-10-17T01:18:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:39 smithi181 ceph-mon[134520]: pgmap v438: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-10-17T01:18:40.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:39 smithi045 ceph-mon[98474]: pgmap v438: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-10-17T01:18:40.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:39 smithi060 ceph-mon[134136]: pgmap v438: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.92k op/s 2023-10-17T01:18:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 210.0 s, 2401.3 tps, lat 41.607 ms stddev 46.908 2023-10-17T01:18:41.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:40 smithi045 ceph-mon[98474]: pgmap v439: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.15k op/s 2023-10-17T01:18:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:40 smithi060 ceph-mon[134136]: pgmap v439: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.15k op/s 2023-10-17T01:18:41.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:40 smithi181 ceph-mon[134520]: pgmap v439: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.15k op/s 2023-10-17T01:18:43.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:42 smithi045 ceph-mon[98474]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 2.06k op/s 2023-10-17T01:18:43.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:42 smithi060 ceph-mon[134136]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 2.06k op/s 2023-10-17T01:18:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:42 smithi181 ceph-mon[134520]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 2.06k op/s 2023-10-17T01:18:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 215.0 s, 2337.6 tps, lat 42.603 ms stddev 52.925 2023-10-17T01:18:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:45 smithi181 ceph-mon[134520]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 1.81k op/s 2023-10-17T01:18:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:45 smithi045 ceph-mon[98474]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 1.81k op/s 2023-10-17T01:18:46.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:45 smithi060 ceph-mon[134136]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 1.81k op/s 2023-10-17T01:18:46.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:46 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:46.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:46 smithi181 ceph-mon[134520]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.07k op/s 2023-10-17T01:18:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:46 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:46 smithi045 ceph-mon[98474]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.07k op/s 2023-10-17T01:18:47.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:46 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:18:47.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:46 smithi060 ceph-mon[134136]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 MiB/s wr, 1.07k op/s 2023-10-17T01:18:49.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:49 smithi181 ceph-mon[134520]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 796 op/s 2023-10-17T01:18:50.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:49 smithi045 ceph-mon[98474]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 796 op/s 2023-10-17T01:18:50.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:49 smithi060 ceph-mon[134136]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 796 op/s 2023-10-17T01:18:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 220.0 s, 2424.0 tps, lat 41.311 ms stddev 47.315 2023-10-17T01:18:51.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:50 smithi045 ceph-mon[98474]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 941 op/s 2023-10-17T01:18:51.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:50 smithi060 ceph-mon[134136]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 941 op/s 2023-10-17T01:18:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:50 smithi181 ceph-mon[134520]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 941 op/s 2023-10-17T01:18:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.558+0000 7f79b87ff700 1 -- 172.21.15.45:0/3021972019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0108f40 msgr2=0x7f79b010f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.558+0000 7f79b87ff700 1 --2- 172.21.15.45:0/3021972019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0108f40 0x7f79b010f7f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f90 tx=0x7f79a800b330 comp rx=0 tx=0).stop 2023-10-17T01:18:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.559+0000 7f79b87ff700 1 -- 172.21.15.45:0/3021972019 shutdown_connections 2023-10-17T01:18:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.559+0000 7f79b87ff700 1 --2- 172.21.15.45:0/3021972019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0108f40 0x7f79b010f7f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.559+0000 7f79b87ff700 1 --2- 172.21.15.45:0/3021972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b01073b0 0x7f79b01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.559+0000 7f79b87ff700 1 --2- 172.21.15.45:0/3021972019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b01085a0 0x7f79b0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.559+0000 7f79b87ff700 1 -- 172.21.15.45:0/3021972019 >> 172.21.15.45:0/3021972019 conn(0x7f79b0076330 msgr2=0x7f79b0078790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.559+0000 7f79b87ff700 1 -- 172.21.15.45:0/3021972019 shutdown_connections 2023-10-17T01:18:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.561+0000 7f79b87ff700 1 -- 172.21.15.45:0/3021972019 wait complete. 2023-10-17T01:18:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.563+0000 7f79b87ff700 1 Processor -- start 2023-10-17T01:18:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.563+0000 7f79b87ff700 1 -- start start 2023-10-17T01:18:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b87ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b01073b0 0x7f79b019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b87ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b01085a0 0x7f79b019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b87ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 0x7f79b019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b87ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b019ed10 con 0x7f79b01073b0 2023-10-17T01:18:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b87ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b019ee80 con 0x7f79b0108f40 2023-10-17T01:18:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b87ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b019f040 con 0x7f79b01085a0 2023-10-17T01:18:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.564+0000 7f79b6d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 0x7f79b019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 0x7f79b019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51728/0 (socket says 172.21.15.45:51728) 2023-10-17T01:18:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 -- 172.21.15.45:0/1603719839 learned_addr learned my addr 172.21.15.45:0/1603719839 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b01085a0 0x7f79b019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 -- 172.21.15.45:0/1603719839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b01085a0 msgr2=0x7f79b019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b01085a0 0x7f79b019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 -- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b01073b0 msgr2=0x7f79b019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b01073b0 0x7f79b019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.565+0000 7f79b6d9c700 1 -- 172.21.15.45:0/1603719839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009c40 con 0x7f79b0108f40 2023-10-17T01:18:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.566+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b01085a0 0x7f79b019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:18:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.566+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 0x7f79b019e6b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f60 tx=0x7f79a8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.567+0000 7f79a37fe700 1 -- 172.21.15.45:0/1603719839 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800e040 con 0x7f79b0108f40 2023-10-17T01:18:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.567+0000 7f79a37fe700 1 -- 172.21.15.45:0/1603719839 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a800be00 con 0x7f79b0108f40 2023-10-17T01:18:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.567+0000 7f79a37fe700 1 -- 172.21.15.45:0/1603719839 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80053a0 con 0x7f79b0108f40 2023-10-17T01:18:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.567+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b019f2c0 con 0x7f79b0108f40 2023-10-17T01:18:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.567+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01a5ae0 con 0x7f79b0108f40 2023-10-17T01:18:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.572+0000 7f79a37fe700 1 -- 172.21.15.45:0/1603719839 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f79a801d470 con 0x7f79b0108f40 2023-10-17T01:18:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.576+0000 7f79a37fe700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f799c0a0c70 0x7f799c0a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.576+0000 7f79a37fe700 1 -- 172.21.15.45:0/1603719839 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f79a80cd7b0 con 0x7f79b0108f40 2023-10-17T01:18:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.577+0000 7f79b659b700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f799c0a0c70 0x7f799c0a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.580+0000 7f79b659b700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f799c0a0c70 0x7f799c0a3130 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f79b0108400 tx=0x7f79a40096f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.582+0000 7f79cc0d7b80 1 Processor -- start 2023-10-17T01:18:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.584+0000 7f79cc0d7b80 1 -- start start 2023-10-17T01:18:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.584+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c064c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.584+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c065790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.584+0000 7f79b659b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c064c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51740/0 (socket says 172.21.15.45:51740) 2023-10-17T01:18:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 -- 172.21.15.45:0/2931192443 learned_addr learned my addr 172.21.15.45:0/2931192443 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 0x55870c066210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870bfbc9b0 con 0x55870c065d90 2023-10-17T01:18:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870bfba370 con 0x55870c064830 2023-10-17T01:18:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870bfb5c60 con 0x55870c065190 2023-10-17T01:18:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 -- 172.21.15.45:0/2931192443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 msgr2=0x55870c065790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c065790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c065790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 -- 172.21.15.45:0/2931192443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 msgr2=0x55870c066210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 0x55870c066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.585+0000 7f79b659b700 1 -- 172.21.15.45:0/2931192443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870bde3d80 con 0x55870c064830 2023-10-17T01:18:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.586+0000 7f79b659b700 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c064c50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79a400ebe0 tx=0x7f79a4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ed04f4180c6d549 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.586+0000 7f79a0ff9700 1 -- 172.21.15.45:0/2931192443 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4010ac0 con 0x55870c064830 2023-10-17T01:18:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.587+0000 7f79a0ff9700 1 -- 172.21.15.45:0/2931192443 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a4010c20 con 0x55870c064830 2023-10-17T01:18:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.587+0000 7f79a0ff9700 1 -- 172.21.15.45:0/2931192443 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400f620 con 0x55870c064830 2023-10-17T01:18:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.588+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 msgr2=0x55870c064c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.588+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c064c50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79a400ebe0 tx=0x7f79a4011000 comp rx=0 tx=0).stop 2023-10-17T01:18:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.589+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 shutdown_connections 2023-10-17T01:18:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.589+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 0x55870c066210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.589+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c064c50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.589+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2931192443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c065790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.589+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 >> 172.21.15.45:0/2931192443 conn(0x55870bfc4070 msgr2=0x55870c05f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.590+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 shutdown_connections 2023-10-17T01:18:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.590+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2931192443 wait complete. 2023-10-17T01:18:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.590+0000 7f79cc0d7b80 1 -- start start 2023-10-17T01:18:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.591+0000 7f79cc0d7b80 1 Processor -- start 2023-10-17T01:18:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c064830 0x55870c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c065d90 0x55870c075a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870bf524b0 con 0x55870c065190 2023-10-17T01:18:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870bf52150 con 0x55870c065d90 2023-10-17T01:18:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870be398c0 con 0x55870c064830 2023-10-17T01:18:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.594+0000 7f79cc0d7b80 10 client.0 _ll_register_callbacks cb 0x55870be9fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.595+0000 7f79b659b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c064830 0x55870c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.595+0000 7f79b659b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c064830 0x55870c06e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44174/0 (socket says 172.21.15.45:44174) 2023-10-17T01:18:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.595+0000 7f79b659b700 1 -- 172.21.15.45:0/4240381989 learned_addr learned my addr 172.21.15.45:0/4240381989 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.595+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c065d90 0x55870c075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f79b6d9c700 1 -- 172.21.15.45:0/4240381989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c064830 msgr2=0x55870c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c064830 0x55870c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f79b6d9c700 1 -- 172.21.15.45:0/4240381989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 msgr2=0x55870c06e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f79b6d9c700 1 -- 172.21.15.45:0/4240381989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a400e8c0 con 0x55870c065d90 2023-10-17T01:18:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c065d90 0x55870c075a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55870c066750 tx=0x7f79a809ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05c0f4ff6775db server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f7999ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:18:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.596+0000 7f7999ffb700 1 -- 172.21.15.45:0/4240381989 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a809b2c0 con 0x55870c065d90 2023-10-17T01:18:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.597+0000 7f7999ffb700 1 -- 172.21.15.45:0/4240381989 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f79a809ae60 con 0x55870c065d90 2023-10-17T01:18:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.597+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55870bfe0cb0 con 0x55870c065d90 2023-10-17T01:18:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.597+0000 7f7999ffb700 1 -- 172.21.15.45:0/4240381989 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a809ec90 con 0x55870c065d90 2023-10-17T01:18:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.597+0000 7f79cc0d7b80 10 client.25913 fetch_fsmap learned FSMap version 40 2023-10-17T01:18:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.597+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55870bdccc00 con 0x55870c065d90 2023-10-17T01:18:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f7999ffb700 1 -- 172.21.15.45:0/4240381989 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55870bfe0cb0 con 0x55870c065d90 2023-10-17T01:18:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f7999ffb700 1 -- 172.21.15.45:0/4240381989 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f79a80a29b0 con 0x55870c065d90 2023-10-17T01:18:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f79cc0d7b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:18:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f79cc0d7b80 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f79cc0d7b80 20 client.25913 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c079fc0 0x55870c07c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f79cc0d7b80 4 client.25913 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:18:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.598+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55870c033cd0 con 0x55870c079fc0 2023-10-17T01:18:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.599+0000 7f79b659b700 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c079fc0 0x55870c07c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.599+0000 7f7999ffb700 10 client.25913 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.601+0000 7f79b659b700 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c079fc0 0x55870c07c3b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae07e8ca5a81f93 server_cookie=128eb87865a75105 in_seq=0 out_seq=0 2023-10-17T01:18:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.602+0000 7f7999ffb700 1 -- 172.21.15.45:0/4240381989 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55870c033cd0 con 0x55870c079fc0 2023-10-17T01:18:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.602+0000 7f7999ffb700 10 client.25913 handle_command_reply: tid=0 2023-10-17T01:18:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 client.25913 shutdown 2023-10-17T01:18:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c065d90 msgr2=0x55870c075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c065d90 0x55870c075a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55870c066750 tx=0x7f79a809ace0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 shutdown_connections 2023-10-17T01:18:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c079fc0 0x55870c07c3b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c065d90 0x55870c075a10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c064830 0x55870c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/4240381989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c06e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 >> 172.21.15.45:0/4240381989 conn(0x55870bfc4070 msgr2=0x55870c05ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.603+0000 7f7999ffb700 0 client.25913 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.606+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 shutdown_connections 2023-10-17T01:18:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.606+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/4240381989 wait complete. 2023-10-17T01:18:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.607+0000 7f79cc0d7b80 20 client.25913 trim_cache size 0 max 16384 2023-10-17T01:18:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.648+0000 7f79cc0d7b80 1 Processor -- start 2023-10-17T01:18:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.649+0000 7f79cc0d7b80 1 -- start start 2023-10-17T01:18:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.650+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c06c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.650+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c05ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.650+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065d90 0x55870c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.650+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870c06d940 con 0x55870c065190 2023-10-17T01:18:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.650+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870c06e000 con 0x55870c064830 2023-10-17T01:18:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.650+0000 7f79b6d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065d90 0x55870c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.651+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870c079b60 con 0x55870c065d90 2023-10-17T01:18:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.651+0000 7f79b659b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c06c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.651+0000 7f79b659b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c06c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51756/0 (socket says 172.21.15.45:51756) 2023-10-17T01:18:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.651+0000 7f79b659b700 1 -- 172.21.15.45:0/2334627969 learned_addr learned my addr 172.21.15.45:0/2334627969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.651+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c05ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.653+0000 7f79b6d9c700 1 -- 172.21.15.45:0/2334627969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 msgr2=0x55870c06c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.654+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c06c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.654+0000 7f79b6d9c700 1 -- 172.21.15.45:0/2334627969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 msgr2=0x55870c05ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.654+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c05ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.654+0000 7f79b6d9c700 1 -- 172.21.15.45:0/2334627969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870be0a1f0 con 0x55870c065d90 2023-10-17T01:18:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.654+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c05ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:18:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.654+0000 7f79b659b700 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c06c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:18:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.655+0000 7f79b6d9c700 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065d90 0x55870c06d090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f79a8007930 tx=0x7f79a8099410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda2d136a2237534 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.655+0000 7f79a17fa700 1 -- 172.21.15.45:0/2334627969 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a809ae60 con 0x55870c065d90 2023-10-17T01:18:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.656+0000 7f79a17fa700 1 -- 172.21.15.45:0/2334627969 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a809ec90 con 0x55870c065d90 2023-10-17T01:18:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.656+0000 7f79a17fa700 1 -- 172.21.15.45:0/2334627969 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a809b0c0 con 0x55870c065d90 2023-10-17T01:18:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.657+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2334627969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065d90 msgr2=0x55870c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.657+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065d90 0x55870c06d090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f79a8007930 tx=0x7f79a8099410 comp rx=0 tx=0).stop 2023-10-17T01:18:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2334627969 shutdown_connections 2023-10-17T01:18:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065d90 0x55870c06d090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c06c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/2334627969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065190 0x55870c05ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2334627969 >> 172.21.15.45:0/2334627969 conn(0x55870bfc4070 msgr2=0x55870c05b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2334627969 shutdown_connections 2023-10-17T01:18:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.660+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/2334627969 wait complete. 2023-10-17T01:18:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.661+0000 7f79cc0d7b80 1 -- start start 2023-10-17T01:18:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.661+0000 7f79cc0d7b80 1 Processor -- start 2023-10-17T01:18:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c072c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 0x55870c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870c0ca150 con 0x55870c065d90 2023-10-17T01:18:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870c0ca290 con 0x55870c064830 2023-10-17T01:18:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870c0ca420 con 0x55870c065190 2023-10-17T01:18:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79b5d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79b5d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44202/0 (socket says 172.21.15.45:44202) 2023-10-17T01:18:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79b5d9a700 1 -- 172.21.15.45:0/3932067754 learned_addr learned my addr 172.21.15.45:0/3932067754 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:18:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79cc0d7b80 10 client.0 _ll_register_callbacks cb 0x55870be9fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:18:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.662+0000 7f79b659b700 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79b5d9a700 1 -- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 msgr2=0x55870c072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c072c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79b5d9a700 1 -- 172.21.15.45:0/3932067754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 msgr2=0x55870c0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:18:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 0x55870c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79b5d9a700 1 -- 172.21.15.45:0/3932067754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55870bde3d80 con 0x55870c065190 2023-10-17T01:18:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79b5d9a700 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c0731b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0136e0 tx=0x7f79ac013aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8600e3c930d5d3b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:18:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f7999ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:18:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f7999ffb700 1 -- 172.21.15.45:0/3932067754 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac004240 con 0x55870c065190 2023-10-17T01:18:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f7999ffb700 1 -- 172.21.15.45:0/3932067754 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f79ac0043a0 con 0x55870c065190 2023-10-17T01:18:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.663+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79a4010c20 con 0x55870c065190 2023-10-17T01:18:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.664+0000 7f7999ffb700 1 -- 172.21.15.45:0/3932067754 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79ac004560 con 0x55870c065190 2023-10-17T01:18:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.665+0000 7f79cc0d7b80 10 client.25888 fetch_fsmap learned FSMap version 40 2023-10-17T01:18:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.665+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f79ac00e690 con 0x55870c065190 2023-10-17T01:18:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.665+0000 7f7999ffb700 1 -- 172.21.15.45:0/3932067754 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a4010c20 con 0x55870c065190 2023-10-17T01:18:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.666+0000 7f7999ffb700 1 -- 172.21.15.45:0/3932067754 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f79ac011ab0 con 0x55870c065190 2023-10-17T01:18:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.667+0000 7f79cc0d7b80 10 client.25888 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:18:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.667+0000 7f79cc0d7b80 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:18:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.667+0000 7f79cc0d7b80 20 client.25888 populate_metadata read hostname 'smithi045' 2023-10-17T01:18:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.667+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c0cbe40 0x55870c0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:18:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.667+0000 7f79cc0d7b80 4 client.25888 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:18:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.667+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55870bfaece0 con 0x55870c0cbe40 2023-10-17T01:18:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.668+0000 7f79b659b700 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c0cbe40 0x55870c0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:18:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.669+0000 7f79b659b700 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c0cbe40 0x55870c0ce260 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a689f40abbca888 server_cookie=6b7056038a7e5df6 in_seq=0 out_seq=0 2023-10-17T01:18:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.669+0000 7f7999ffb700 10 client.25888 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.739+0000 7f7999ffb700 1 -- 172.21.15.45:0/3932067754 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55870bfaece0 con 0x55870c0cbe40 2023-10-17T01:18:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.739+0000 7f7999ffb700 10 client.25888 handle_command_reply: tid=0 2023-10-17T01:18:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.739+0000 7f79cc0d7b80 1 client.25888 shutdown 2023-10-17T01:18:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.740+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 msgr2=0x55870c0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.740+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c0731b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0136e0 tx=0x7f79ac013aa0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.740+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 shutdown_connections 2023-10-17T01:18:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.740+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55870c065d90 0x55870c0c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.740+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55870c0cbe40 0x55870c0ce260 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.741+0000 7f7999ffb700 0 client.25888 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:18:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.741+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55870c064830 0x55870c072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.741+0000 7f79cc0d7b80 1 --2- 172.21.15.45:0/3932067754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55870c065190 0x55870c0731b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.741+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 >> 172.21.15.45:0/3932067754 conn(0x55870bfc4070 msgr2=0x55870c06c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.741+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 shutdown_connections 2023-10-17T01:18:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.741+0000 7f79cc0d7b80 1 -- 172.21.15.45:0/3932067754 wait complete. 2023-10-17T01:18:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.742+0000 7f79cc0d7b80 20 client.25888 trim_cache size 0 max 16384 2023-10-17T01:18:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:18:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:18:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:18:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:18:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f799c0a0c70 msgr2=0x7f799c0a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f799c0a0c70 0x7f799c0a3130 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f79b0108400 tx=0x7f79a40096f0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 msgr2=0x7f79b019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:18:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 0x7f79b019e6b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f60 tx=0x7f79a8004c80 comp rx=0 tx=0).stop 2023-10-17T01:18:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 shutdown_connections 2023-10-17T01:18:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f799c0a0c70 0x7f799c0a3130 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0108f40 0x7f79b019e6b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b01073b0 0x7f79b019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 --2- 172.21.15.45:0/1603719839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b01085a0 0x7f79b019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:18:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.743+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 >> 172.21.15.45:0/1603719839 conn(0x7f79b0076330 msgr2=0x7f79b010b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:18:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.744+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 shutdown_connections 2023-10-17T01:18:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:18:51.744+0000 7f79b87ff700 1 -- 172.21.15.45:0/1603719839 wait complete. 2023-10-17T01:18:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:52 smithi045 ceph-mon[98474]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 778 op/s 2023-10-17T01:18:53.086 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:52 smithi060 ceph-mon[134136]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 778 op/s 2023-10-17T01:18:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:52 smithi181 ceph-mon[134520]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 778 op/s 2023-10-17T01:18:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 225.0 s, 1344.8 tps, lat 73.172 ms stddev 83.638 2023-10-17T01:18:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:55 smithi181 ceph-mon[134520]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 MiB/s wr, 870 op/s 2023-10-17T01:18:56.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:55 smithi045 ceph-mon[98474]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 MiB/s wr, 870 op/s 2023-10-17T01:18:56.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:55 smithi060 ceph-mon[134136]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 MiB/s wr, 870 op/s 2023-10-17T01:18:57.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:57 smithi181 ceph-mon[134520]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 870 op/s 2023-10-17T01:18:58.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:57 smithi045 ceph-mon[98474]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 870 op/s 2023-10-17T01:18:58.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:57 smithi060 ceph-mon[134136]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 870 op/s 2023-10-17T01:18:59.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:18:59 smithi181 ceph-mon[134520]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 898 op/s 2023-10-17T01:19:00.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:18:59 smithi045 ceph-mon[98474]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 898 op/s 2023-10-17T01:19:00.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:18:59 smithi060 ceph-mon[134136]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 898 op/s 2023-10-17T01:19:00.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:19:00.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 -- 172.21.15.45:0/4277318021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 msgr2=0x7f68c00ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 --2- 172.21.15.45:0/4277318021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c00ff9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f68b00098f0 tx=0x7f68b0009c00 comp rx=0 tx=0).stop 2023-10-17T01:19:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 -- 172.21.15.45:0/4277318021 shutdown_connections 2023-10-17T01:19:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 --2- 172.21.15.45:0/4277318021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c01008d0 0x7f68c0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 --2- 172.21.15.45:0/4277318021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c00ff9d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 --2- 172.21.15.45:0/4277318021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c00fff10 0x7f68c0100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.552+0000 7f68c4bff700 1 -- 172.21.15.45:0/4277318021 >> 172.21.15.45:0/4277318021 conn(0x7f68c00fb1b0 msgr2=0x7f68c00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.553+0000 7f68c4bff700 1 -- 172.21.15.45:0/4277318021 shutdown_connections 2023-10-17T01:19:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.553+0000 7f68c4bff700 1 -- 172.21.15.45:0/4277318021 wait complete. 2023-10-17T01:19:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.554+0000 7f68c4bff700 1 Processor -- start 2023-10-17T01:19:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 -- start start 2023-10-17T01:19:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00fff10 0x7f68c0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c01008d0 0x7f68c01a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01a0830 con 0x7f68c01008d0 2023-10-17T01:19:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01a09a0 con 0x7f68c00ff5b0 2023-10-17T01:19:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.555+0000 7f68c4bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01a0b60 con 0x7f68c00fff10 2023-10-17T01:19:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.556+0000 7f68be59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.556+0000 7f68be59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39142/0 (socket says 172.21.15.45:39142) 2023-10-17T01:19:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.556+0000 7f68be59c700 1 -- 172.21.15.45:0/3114684220 learned_addr learned my addr 172.21.15.45:0/3114684220 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:00.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.556+0000 7f68be59c700 1 -- 172.21.15.45:0/3114684220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00fff10 msgr2=0x7f68c0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.556+0000 7f68b7fff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00fff10 0x7f68c0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c01008d0 0x7f68c01a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68be59c700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00fff10 0x7f68c0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68be59c700 1 -- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c01008d0 msgr2=0x7f68c01a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68be59c700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c01008d0 0x7f68c01a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68be59c700 1 -- 172.21.15.45:0/3114684220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8009d00 con 0x7f68c00ff5b0 2023-10-17T01:19:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68b7fff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00fff10 0x7f68c0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:19:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.557+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c01008d0 0x7f68c01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:19:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.558+0000 7f68be59c700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c0198cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009fd0 tx=0x7f68b000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.558+0000 7f68b77fe700 1 -- 172.21.15.45:0/3114684220 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b001c070 con 0x7f68c00ff5b0 2023-10-17T01:19:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.558+0000 7f68b77fe700 1 -- 172.21.15.45:0/3114684220 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b000f820 con 0x7f68c00ff5b0 2023-10-17T01:19:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.558+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b0009520 con 0x7f68c00ff5b0 2023-10-17T01:19:00.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.559+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c01a1070 con 0x7f68c00ff5b0 2023-10-17T01:19:00.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.559+0000 7f68b77fe700 1 -- 172.21.15.45:0/3114684220 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0004c70 con 0x7f68c00ff5b0 2023-10-17T01:19:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.565+0000 7f68b77fe700 1 -- 172.21.15.45:0/3114684220 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f68b0020070 con 0x7f68c00ff5b0 2023-10-17T01:19:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.567+0000 7f68d851db80 1 Processor -- start 2023-10-17T01:19:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.568+0000 7f68d851db80 1 -- start start 2023-10-17T01:19:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68d851db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec4753b0 0x555eec4757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68d851db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec475d10 0x555eec476530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68d851db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec476f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68d851db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec3c4a10 con 0x555eec475d10 2023-10-17T01:19:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68d851db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec3c23d0 con 0x555eec4753b0 2023-10-17T01:19:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68d851db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec3bdcc0 con 0x555eec476ac0 2023-10-17T01:19:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec4753b0 0x555eec4757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.569+0000 7f68b77fe700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68ac0a0cf0 0x7f68ac0a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.570+0000 7f68b77fe700 1 -- 172.21.15.45:0/3114684220 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68b00ccf40 con 0x7f68c00ff5b0 2023-10-17T01:19:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.570+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec476f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.570+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec476f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34594/0 (socket says 172.21.15.45:34594) 2023-10-17T01:19:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.570+0000 7f68b7fff700 1 -- 172.21.15.45:0/3855978010 learned_addr learned my addr 172.21.15.45:0/3855978010 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68ac0a0cf0 0x7f68ac0a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 -- 172.21.15.45:0/3855978010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 msgr2=0x555eec476f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec476f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68be59c700 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec475d10 0x555eec476530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 -- 172.21.15.45:0/3855978010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec475d10 msgr2=0x555eec476530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec475d10 0x555eec476530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 -- 172.21.15.45:0/3855978010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eec1ebde0 con 0x555eec4753b0 2023-10-17T01:19:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68be59c700 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec475d10 0x555eec476530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:19:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.571+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec4753b0 0x555eec4757d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f68b8007770 tx=0x7f68b8007a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6855bf3009dbc3f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.572+0000 7f68b4ff9700 1 -- 172.21.15.45:0/3855978010 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8017690 con 0x555eec4753b0 2023-10-17T01:19:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.572+0000 7f68b4ff9700 1 -- 172.21.15.45:0/3855978010 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b800f030 con 0x555eec4753b0 2023-10-17T01:19:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.572+0000 7f68b4ff9700 1 -- 172.21.15.45:0/3855978010 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8021990 con 0x555eec4753b0 2023-10-17T01:19:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 -- 172.21.15.45:0/3855978010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec4753b0 msgr2=0x555eec4757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec4753b0 0x555eec4757d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f68b8007770 tx=0x7f68b8007a80 comp rx=0 tx=0).stop 2023-10-17T01:19:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68bed9d700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68ac0a0cf0 0x7f68ac0a31b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68b8020000 tx=0x7f68b801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 -- 172.21.15.45:0/3855978010 shutdown_connections 2023-10-17T01:19:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec476f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec4753b0 0x555eec4757d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 --2- 172.21.15.45:0/3855978010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec475d10 0x555eec476530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.574+0000 7f68d851db80 1 -- 172.21.15.45:0/3855978010 >> 172.21.15.45:0/3855978010 conn(0x555eec3cc0d0 msgr2=0x555eec467950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:00.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.575+0000 7f68d851db80 1 -- 172.21.15.45:0/3855978010 shutdown_connections 2023-10-17T01:19:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.576+0000 7f68d851db80 1 -- 172.21.15.45:0/3855978010 wait complete. 2023-10-17T01:19:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.577+0000 7f68d851db80 1 -- start start 2023-10-17T01:19:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.579+0000 7f68d851db80 1 Processor -- start 2023-10-17T01:19:00.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.580+0000 7f68d851db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec47f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68d851db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec475d10 0x555eec47f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68d851db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec476ac0 0x555eec486650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68d851db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec35a510 con 0x555eec4753b0 2023-10-17T01:19:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68d851db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec35a1b0 con 0x555eec476ac0 2023-10-17T01:19:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68d851db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec241920 con 0x555eec475d10 2023-10-17T01:19:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec475d10 0x555eec47f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68d851db80 10 client.0 _ll_register_callbacks cb 0x555eec2a7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec47f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec476ac0 0x555eec486650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec47f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39676/0 (socket says 172.21.15.45:39676) 2023-10-17T01:19:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.581+0000 7f68be59c700 1 -- 172.21.15.45:0/3292796599 learned_addr learned my addr 172.21.15.45:0/3292796599 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.582+0000 7f68b7fff700 1 -- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec476ac0 msgr2=0x555eec486650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.582+0000 7f68b7fff700 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec476ac0 0x555eec486650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.582+0000 7f68b7fff700 1 -- 172.21.15.45:0/3292796599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 msgr2=0x555eec47f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.582+0000 7f68b7fff700 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec47f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.582+0000 7f68b7fff700 1 -- 172.21.15.45:0/3292796599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8009390 con 0x555eec475d10 2023-10-17T01:19:00.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.584+0000 7f68b7fff700 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec475d10 0x555eec47f610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005fd0 tx=0x7f68a8011710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e68cce22b23e44a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:00.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.584+0000 7f68a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:00.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.585+0000 7f68a5ffb700 1 -- 172.21.15.45:0/3292796599 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8004220 con 0x555eec475d10 2023-10-17T01:19:00.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.586+0000 7f68a5ffb700 1 -- 172.21.15.45:0/3292796599 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68a8004380 con 0x555eec475d10 2023-10-17T01:19:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.586+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eec3e8d10 con 0x555eec475d10 2023-10-17T01:19:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.587+0000 7f68a5ffb700 1 -- 172.21.15.45:0/3292796599 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a8004540 con 0x555eec475d10 2023-10-17T01:19:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.587+0000 7f68d851db80 10 client.25897 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.587+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555eec1d4c60 con 0x555eec475d10 2023-10-17T01:19:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.587+0000 7f68a5ffb700 1 -- 172.21.15.45:0/3292796599 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555eec3e8d10 con 0x555eec475d10 2023-10-17T01:19:00.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68a5ffb700 1 -- 172.21.15.45:0/3292796599 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f68a800c070 con 0x555eec475d10 2023-10-17T01:19:00.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68d851db80 10 client.25897 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68d851db80 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68d851db80 20 client.25897 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68d851db80 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec48ad80 0x555eec48d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68d851db80 4 client.25897 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eec43bd30 con 0x555eec48ad80 2023-10-17T01:19:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.589+0000 7f68be59c700 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec48ad80 0x555eec48d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.590+0000 7f68be59c700 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec48ad80 0x555eec48d1a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6292e219ab7f84 server_cookie=db7d5f441a0108ef in_seq=0 out_seq=0 2023-10-17T01:19:00.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.590+0000 7f68a5ffb700 10 client.25897 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:00.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.591+0000 7f68a5ffb700 1 -- 172.21.15.45:0/3292796599 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x555eec43bd30 con 0x555eec48ad80 2023-10-17T01:19:00.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.591+0000 7f68a5ffb700 10 client.25897 handle_command_reply: tid=0 2023-10-17T01:19:00.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.591+0000 7f68d851db80 1 client.25897 shutdown 2023-10-17T01:19:00.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.592+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec475d10 msgr2=0x555eec47f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.592+0000 7f68d851db80 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec475d10 0x555eec47f610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005fd0 tx=0x7f68a8011710 comp rx=0 tx=0).stop 2023-10-17T01:19:00.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 shutdown_connections 2023-10-17T01:19:00.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec48ad80 0x555eec48d1a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec476ac0 0x555eec486650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec47f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 --2- 172.21.15.45:0/3292796599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec475d10 0x555eec47f610 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 >> 172.21.15.45:0/3292796599 conn(0x555eec3cc0d0 msgr2=0x555eec467020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:00.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68a5ffb700 0 client.25897 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:00.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.593+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 shutdown_connections 2023-10-17T01:19:00.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.594+0000 7f68d851db80 1 -- 172.21.15.45:0/3292796599 wait complete. 2023-10-17T01:19:00.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.594+0000 7f68d851db80 20 client.25897 trim_cache size 0 max 16384 2023-10-17T01:19:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 Processor -- start 2023-10-17T01:19:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 -- start start 2023-10-17T01:19:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec4753b0 0x555eec4799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec479f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec476ac0 0x555eec47ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec47ec40 con 0x555eec476ac0 2023-10-17T01:19:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec464420 con 0x555eec475d10 2023-10-17T01:19:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.641+0000 7f68d851db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec47dbf0 con 0x555eec4753b0 2023-10-17T01:19:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.642+0000 7f68bed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec476ac0 0x555eec47ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.642+0000 7f68bed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec476ac0 0x555eec47ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39688/0 (socket says 172.21.15.45:39688) 2023-10-17T01:19:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.642+0000 7f68bed9d700 1 -- 172.21.15.45:0/2287632837 learned_addr learned my addr 172.21.15.45:0/2287632837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68be59c700 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec4753b0 0x555eec4799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68b7fff700 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec479f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68be59c700 1 -- 172.21.15.45:0/2287632837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 msgr2=0x555eec479f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68be59c700 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec479f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68be59c700 1 -- 172.21.15.45:0/2287632837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec476ac0 msgr2=0x555eec47ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68be59c700 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec476ac0 0x555eec47ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.643+0000 7f68be59c700 1 -- 172.21.15.45:0/2287632837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eec212250 con 0x555eec4753b0 2023-10-17T01:19:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.644+0000 7f68be59c700 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec4753b0 0x555eec4799d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68b009a2f0 tx=0x7f68b0019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb66759399ec594 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.644+0000 7f68b57fa700 1 -- 172.21.15.45:0/2287632837 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b009e420 con 0x555eec4753b0 2023-10-17T01:19:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.644+0000 7f68b57fa700 1 -- 172.21.15.45:0/2287632837 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b00a25d0 con 0x555eec4753b0 2023-10-17T01:19:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.644+0000 7f68b57fa700 1 -- 172.21.15.45:0/2287632837 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b009e420 con 0x555eec4753b0 2023-10-17T01:19:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 230.0 s, 1768.2 tps, lat 57.018 ms stddev 62.945 2023-10-17T01:19:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 -- 172.21.15.45:0/2287632837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec4753b0 msgr2=0x555eec4799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec4753b0 0x555eec4799d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68b009a2f0 tx=0x7f68b0019040 comp rx=0 tx=0).stop 2023-10-17T01:19:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 -- 172.21.15.45:0/2287632837 shutdown_connections 2023-10-17T01:19:00.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec476ac0 0x555eec47ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec4753b0 0x555eec4799d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 --2- 172.21.15.45:0/2287632837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec479f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.647+0000 7f68d851db80 1 -- 172.21.15.45:0/2287632837 >> 172.21.15.45:0/2287632837 conn(0x555eec3cc0d0 msgr2=0x555eec47ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:00.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.648+0000 7f68d851db80 1 -- 172.21.15.45:0/2287632837 shutdown_connections 2023-10-17T01:19:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.648+0000 7f68d851db80 1 -- 172.21.15.45:0/2287632837 wait complete. 2023-10-17T01:19:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.648+0000 7f68d851db80 1 -- start start 2023-10-17T01:19:00.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.649+0000 7f68d851db80 1 Processor -- start 2023-10-17T01:19:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68d851db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec484140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68d851db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec482750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68d851db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec482c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68d851db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec4831d0 con 0x555eec4753b0 2023-10-17T01:19:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68d851db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec483340 con 0x555eec475d10 2023-10-17T01:19:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68d851db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eec483500 con 0x555eec476ac0 2023-10-17T01:19:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec482750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec482750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39176/0 (socket says 172.21.15.45:39176) 2023-10-17T01:19:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68b7fff700 1 -- 172.21.15.45:0/664151652 learned_addr learned my addr 172.21.15.45:0/664151652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.651+0000 7f68d851db80 10 client.0 _ll_register_callbacks cb 0x7f68a8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.650+0000 7f68bed9d700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec482c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.651+0000 7f68be59c700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec484140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.652+0000 7f68b7fff700 1 -- 172.21.15.45:0/664151652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 msgr2=0x555eec482c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.652+0000 7f68b7fff700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec482c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.652+0000 7f68b7fff700 1 -- 172.21.15.45:0/664151652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 msgr2=0x555eec484140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.652+0000 7f68b7fff700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec484140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.653+0000 7f68b7fff700 1 -- 172.21.15.45:0/664151652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b00a0040 con 0x555eec475d10 2023-10-17T01:19:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.653+0000 7f68bed9d700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec482c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:19:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.653+0000 7f68b7fff700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec482750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009c60 tx=0x7f68a8011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86dcdde881687496 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.653+0000 7f68a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:19:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.656+0000 7f68a5ffb700 1 -- 172.21.15.45:0/664151652 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8005510 con 0x555eec475d10 2023-10-17T01:19:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.656+0000 7f68a5ffb700 1 -- 172.21.15.45:0/664151652 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f68a8000e40 con 0x555eec475d10 2023-10-17T01:19:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.656+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68b800f030 con 0x555eec475d10 2023-10-17T01:19:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.656+0000 7f68a5ffb700 1 -- 172.21.15.45:0/664151652 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a8005670 con 0x555eec475d10 2023-10-17T01:19:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.657+0000 7f68d851db80 10 client.25937 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.657+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555eec1ea2b0 con 0x555eec475d10 2023-10-17T01:19:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.657+0000 7f68a5ffb700 1 -- 172.21.15.45:0/664151652 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b800f030 con 0x555eec475d10 2023-10-17T01:19:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.657+0000 7f68a5ffb700 1 -- 172.21.15.45:0/664151652 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f68a8011430 con 0x555eec475d10 2023-10-17T01:19:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68d851db80 10 client.25937 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:00.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68d851db80 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68d851db80 20 client.25937 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68d851db80 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec4eedf0 0x555eec4f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68d851db80 4 client.25937 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:19:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555eec3b6d40 con 0x555eec4eedf0 2023-10-17T01:19:00.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.658+0000 7f68be59c700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec4eedf0 0x555eec4f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.659+0000 7f68be59c700 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec4eedf0 0x555eec4f11e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c08c5aa57decfb server_cookie=9caea463a89d1d2b in_seq=0 out_seq=0 2023-10-17T01:19:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.659+0000 7f68a5ffb700 10 client.25937 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:00.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.659+0000 7f68a5ffb700 1 -- 172.21.15.45:0/664151652 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555eec3b6d40 con 0x555eec4eedf0 2023-10-17T01:19:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.659+0000 7f68a5ffb700 10 client.25937 handle_command_reply: tid=0 2023-10-17T01:19:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.659+0000 7f68d851db80 1 client.25937 shutdown 2023-10-17T01:19:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 msgr2=0x555eec482750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec482750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009c60 tx=0x7f68a8011bf0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 shutdown_connections 2023-10-17T01:19:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555eec476ac0 0x555eec482c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555eec4753b0 0x555eec484140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555eec4eedf0 0x555eec4f11e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 --2- 172.21.15.45:0/664151652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555eec475d10 0x555eec482750 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 >> 172.21.15.45:0/664151652 conn(0x555eec3cc0d0 msgr2=0x555eec47ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68a5ffb700 0 client.25937 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.660+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 shutdown_connections 2023-10-17T01:19:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.661+0000 7f68d851db80 1 -- 172.21.15.45:0/664151652 wait complete. 2023-10-17T01:19:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.661+0000 7f68d851db80 20 client.25937 trim_cache size 0 max 16384 2023-10-17T01:19:00.671 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:19:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:19:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:19:00.671 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:19:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:19:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:19:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:19:00.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:19:00.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:19:00.673 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:19:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68ac0a0cf0 msgr2=0x7f68ac0a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68ac0a0cf0 0x7f68ac0a31b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68b8020000 tx=0x7f68b801d000 comp rx=0 tx=0).stop 2023-10-17T01:19:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 msgr2=0x7f68c0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c0198cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009fd0 tx=0x7f68b000b840 comp rx=0 tx=0).stop 2023-10-17T01:19:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 shutdown_connections 2023-10-17T01:19:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f68ac0a0cf0 0x7f68ac0a31b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c01008d0 0x7f68c01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68c00ff5b0 0x7f68c0198cf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 --2- 172.21.15.45:0/3114684220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68c00fff10 0x7f68c0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.663+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 >> 172.21.15.45:0/3114684220 conn(0x7f68c00fb1b0 msgr2=0x7f68c00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.664+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 shutdown_connections 2023-10-17T01:19:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:00.664+0000 7f68c4bff700 1 -- 172.21.15.45:0/3114684220 wait complete. 2023-10-17T01:19:00.699 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:19:01.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:01 smithi181 ceph-mon[134520]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 1.07k op/s 2023-10-17T01:19:02.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:01 smithi045 ceph-mon[98474]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 1.07k op/s 2023-10-17T01:19:02.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:01 smithi060 ceph-mon[134136]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 1.07k op/s 2023-10-17T01:19:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:03 smithi181 ceph-mon[134520]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:19:04.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:03 smithi045 ceph-mon[98474]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:19:04.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:03 smithi060 ceph-mon[134136]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:19:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 -- 172.21.15.45:0/3522098946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40501017f0 msgr2=0x7f4050103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 --2- 172.21.15.45:0/3522098946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40501017f0 0x7f4050103be0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f404c009990 tx=0x7f404c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 -- 172.21.15.45:0/3522098946 shutdown_connections 2023-10-17T01:19:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 --2- 172.21.15.45:0/3522098946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050106a50 0x7f4050108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 --2- 172.21.15.45:0/3522098946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40501017f0 0x7f4050103be0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 --2- 172.21.15.45:0/3522098946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050104120 0x7f4050106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 -- 172.21.15.45:0/3522098946 >> 172.21.15.45:0/3522098946 conn(0x7f40500fb1a0 msgr2=0x7f40500fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 -- 172.21.15.45:0/3522098946 shutdown_connections 2023-10-17T01:19:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.025+0000 7f40579ff700 1 -- 172.21.15.45:0/3522098946 wait complete. 2023-10-17T01:19:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.027+0000 7f40579ff700 1 Processor -- start 2023-10-17T01:19:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 -- start start 2023-10-17T01:19:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 0x7f4050196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050104120 0x7f4050197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4050106a50 0x7f4050197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050198570 con 0x7f4050104120 2023-10-17T01:19:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501986e0 con 0x7f4050106a50 2023-10-17T01:19:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f40579ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501988a0 con 0x7f40501017f0 2023-10-17T01:19:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f405579b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 0x7f4050196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f405579b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 0x7f4050196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34642/0 (socket says 172.21.15.45:34642) 2023-10-17T01:19:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f405579b700 1 -- 172.21.15.45:0/4021583285 learned_addr learned my addr 172.21.15.45:0/4021583285 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f4055f9c700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4050106a50 0x7f4050197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.028+0000 7f4054f9a700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050104120 0x7f4050197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f4055f9c700 1 -- 172.21.15.45:0/4021583285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 msgr2=0x7f4050196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f4055f9c700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 0x7f4050196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f4055f9c700 1 -- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050104120 msgr2=0x7f4050197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f4055f9c700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050104120 0x7f4050197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f4055f9c700 1 -- 172.21.15.45:0/4021583285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c0095f0 con 0x7f4050106a50 2023-10-17T01:19:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f405579b700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 0x7f4050196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f4055f9c700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4050106a50 0x7f4050197f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f404800efc0 tx=0x7f404800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.029+0000 7f40467fc700 1 -- 172.21.15.45:0/4021583285 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048018070 con 0x7f4050106a50 2023-10-17T01:19:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.030+0000 7f40467fc700 1 -- 172.21.15.45:0/4021583285 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404800f040 con 0x7f4050106a50 2023-10-17T01:19:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.030+0000 7f40467fc700 1 -- 172.21.15.45:0/4021583285 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800cd40 con 0x7f4050106a50 2023-10-17T01:19:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.030+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4050198b20 con 0x7f4050106a50 2023-10-17T01:19:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.030+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405019f340 con 0x7f4050106a50 2023-10-17T01:19:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.033+0000 7f40467fc700 1 -- 172.21.15.45:0/4021583285 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f4048003a00 con 0x7f4050106a50 2023-10-17T01:19:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.036+0000 7f406b29db80 1 Processor -- start 2023-10-17T01:19:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.036+0000 7f40467fc700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f403c0a0c70 0x7f403c0a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.036+0000 7f40467fc700 1 -- 172.21.15.45:0/4021583285 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f40480cbc00 con 0x7f4050106a50 2023-10-17T01:19:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.036+0000 7f406b29db80 1 -- start start 2023-10-17T01:19:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f405579b700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f403c0a0c70 0x7f403c0a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f406b29db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x55680983b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f406b29db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x556809831510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f406b29db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809831f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f406b29db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568097899b0 con 0x556809831aa0 2023-10-17T01:19:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f406b29db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556809787370 con 0x55680983b300 2023-10-17T01:19:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f406b29db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556809782c60 con 0x55680983bc60 2023-10-17T01:19:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.037+0000 7f405579b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x55680983b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809831f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f405579b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x55680983b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39210/0 (socket says 172.21.15.45:39210) 2023-10-17T01:19:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f405579b700 1 -- 172.21.15.45:0/260984841 learned_addr learned my addr 172.21.15.45:0/260984841 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4054f9a700 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x556809831510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 -- 172.21.15.45:0/260984841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 msgr2=0x556809831510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x556809831510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 -- 172.21.15.45:0/260984841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 msgr2=0x55680983b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x55680983b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 -- 172.21.15.45:0/260984841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568095b0d80 con 0x556809831aa0 2023-10-17T01:19:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f405579b700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f403c0a0c70 0x7f403c0a3130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f404c005120 tx=0x7f404c005150 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f4055f9c700 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809831f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f40480c96a0 tx=0x7f40480956c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d219b2c061e99525 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.038+0000 7f403b7fe700 1 -- 172.21.15.45:0/260984841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048098df0 con 0x556809831aa0 2023-10-17T01:19:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f403b7fe700 1 -- 172.21.15.45:0/260984841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4048098c10 con 0x556809831aa0 2023-10-17T01:19:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f403b7fe700 1 -- 172.21.15.45:0/260984841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480a1290 con 0x556809831aa0 2023-10-17T01:19:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 -- 172.21.15.45:0/260984841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 msgr2=0x556809831f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809831f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f40480c96a0 tx=0x7f40480956c0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 -- 172.21.15.45:0/260984841 shutdown_connections 2023-10-17T01:19:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809831f00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x55680983b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 --2- 172.21.15.45:0/260984841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x556809831510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.039+0000 7f406b29db80 1 -- 172.21.15.45:0/260984841 >> 172.21.15.45:0/260984841 conn(0x556809791070 msgr2=0x55680982c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.040+0000 7f406b29db80 1 -- 172.21.15.45:0/260984841 shutdown_connections 2023-10-17T01:19:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.040+0000 7f406b29db80 1 -- 172.21.15.45:0/260984841 wait complete. 2023-10-17T01:19:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.040+0000 7f406b29db80 1 -- start start 2023-10-17T01:19:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.040+0000 7f406b29db80 1 Processor -- start 2023-10-17T01:19:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809843d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x55680984b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55680971f4b0 con 0x556809831aa0 2023-10-17T01:19:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55680971f150 con 0x55680983b300 2023-10-17T01:19:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568096068c0 con 0x55680983bc60 2023-10-17T01:19:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f406b29db80 10 client.0 _ll_register_callbacks cb 0x55680966ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f4054f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f405579b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809843d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f4054f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39218/0 (socket says 172.21.15.45:39218) 2023-10-17T01:19:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f4054f9a700 1 -- 172.21.15.45:0/726263056 learned_addr learned my addr 172.21.15.45:0/726263056 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.041+0000 7f4055f9c700 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x55680984b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4055f9c700 1 -- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 msgr2=0x5568098442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4055f9c700 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4055f9c700 1 -- 172.21.15.45:0/726263056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 msgr2=0x556809843d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4055f9c700 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809843d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4055f9c700 1 -- 172.21.15.45:0/726263056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405019f340 con 0x55680983bc60 2023-10-17T01:19:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4055f9c700 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x55680984b100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f404800ef90 tx=0x7f40480997f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef1b8b063658c2a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.042+0000 7f4038ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.043+0000 7f4038ff9700 1 -- 172.21.15.45:0/726263056 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048004ad0 con 0x55680983bc60 2023-10-17T01:19:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.043+0000 7f4038ff9700 1 -- 172.21.15.45:0/726263056 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f40480999b0 con 0x55680983bc60 2023-10-17T01:19:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.043+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568097adcb0 con 0x55680983bc60 2023-10-17T01:19:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f4038ff9700 1 -- 172.21.15.45:0/726263056 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40480a1290 con 0x55680983bc60 2023-10-17T01:19:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 10 client.25915 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556809599c00 con 0x55680983bc60 2023-10-17T01:19:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f4038ff9700 1 -- 172.21.15.45:0/726263056 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40480b7020 con 0x55680983bc60 2023-10-17T01:19:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f4038ff9700 1 -- 172.21.15.45:0/726263056 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f40480a0e00 con 0x55680983bc60 2023-10-17T01:19:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 10 client.25915 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 20 client.25915 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55680984f7f0 0x556809851c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 4 client.25915 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.044+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556809800cd0 con 0x55680984f7f0 2023-10-17T01:19:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.045+0000 7f405579b700 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55680984f7f0 0x556809851c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.045+0000 7f405579b700 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55680984f7f0 0x556809851c10 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da2af9d50e6c944 server_cookie=32df364bed4f4d73 in_seq=0 out_seq=0 2023-10-17T01:19:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.045+0000 7f4038ff9700 10 client.25915 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f4038ff9700 1 -- 172.21.15.45:0/726263056 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x556809800cd0 con 0x55680984f7f0 2023-10-17T01:19:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f4038ff9700 10 client.25915 handle_command_reply: tid=0 2023-10-17T01:19:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 client.25915 shutdown 2023-10-17T01:19:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 msgr2=0x55680984b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x55680984b100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f404800ef90 tx=0x7f40480997f0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 shutdown_connections 2023-10-17T01:19:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x55680984b100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809843d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55680984f7f0 0x556809851c10 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 --2- 172.21.15.45:0/726263056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 >> 172.21.15.45:0/726263056 conn(0x556809791070 msgr2=0x55680982ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.046+0000 7f4038ff9700 0 client.25915 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.047+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 shutdown_connections 2023-10-17T01:19:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.047+0000 7f406b29db80 1 -- 172.21.15.45:0/726263056 wait complete. 2023-10-17T01:19:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.047+0000 7f406b29db80 20 client.25915 trim_cache size 0 max 16384 2023-10-17T01:19:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.066+0000 7f406b29db80 1 Processor -- start 2023-10-17T01:19:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.066+0000 7f406b29db80 1 -- start start 2023-10-17T01:19:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f406b29db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556809831aa0 0x556809831510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f406b29db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55680983b300 0x556809831ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f406b29db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983bc60 0x55680983e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f406b29db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55680982cf60 con 0x55680983b300 2023-10-17T01:19:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f406b29db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568098417b0 con 0x55680983bc60 2023-10-17T01:19:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f406b29db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55680983f0a0 con 0x556809831aa0 2023-10-17T01:19:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f405579b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556809831aa0 0x556809831510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4055f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983bc60 0x55680983e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f405579b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556809831aa0 0x556809831510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34672/0 (socket says 172.21.15.45:34672) 2023-10-17T01:19:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f405579b700 1 -- 172.21.15.45:0/2395670304 learned_addr learned my addr 172.21.15.45:0/2395670304 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4054f9a700 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55680983b300 0x556809831ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4055f9c700 1 -- 172.21.15.45:0/2395670304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556809831aa0 msgr2=0x556809831510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4055f9c700 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556809831aa0 0x556809831510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4055f9c700 1 -- 172.21.15.45:0/2395670304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55680983b300 msgr2=0x556809831ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4055f9c700 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55680983b300 0x556809831ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.067+0000 7f4055f9c700 1 -- 172.21.15.45:0/2395670304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568095d71f0 con 0x55680983bc60 2023-10-17T01:19:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.068+0000 7f4055f9c700 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983bc60 0x55680983e510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40480957b0 tx=0x7f4048004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf344305036a0169 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.068+0000 7f403bfff700 1 -- 172.21.15.45:0/2395670304 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048098df0 con 0x55680983bc60 2023-10-17T01:19:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.068+0000 7f403bfff700 1 -- 172.21.15.45:0/2395670304 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40480b7030 con 0x55680983bc60 2023-10-17T01:19:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.068+0000 7f403bfff700 1 -- 172.21.15.45:0/2395670304 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404809f0b0 con 0x55680983bc60 2023-10-17T01:19:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 -- 172.21.15.45:0/2395670304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983bc60 msgr2=0x55680983e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983bc60 0x55680983e510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40480957b0 tx=0x7f4048004dc0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 -- 172.21.15.45:0/2395670304 shutdown_connections 2023-10-17T01:19:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983bc60 0x55680983e510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556809831aa0 0x556809831510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 --2- 172.21.15.45:0/2395670304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55680983b300 0x556809831ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 -- 172.21.15.45:0/2395670304 >> 172.21.15.45:0/2395670304 conn(0x556809791070 msgr2=0x556809841a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 -- 172.21.15.45:0/2395670304 shutdown_connections 2023-10-17T01:19:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 -- 172.21.15.45:0/2395670304 wait complete. 2023-10-17T01:19:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.069+0000 7f406b29db80 1 -- start start 2023-10-17T01:19:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.070+0000 7f406b29db80 1 Processor -- start 2023-10-17T01:19:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809846e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x5568098483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556809848900 con 0x556809831aa0 2023-10-17T01:19:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556809848a70 con 0x55680983b300 2023-10-17T01:19:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556809848c30 con 0x55680983bc60 2023-10-17T01:19:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f406b29db80 10 client.0 _ll_register_callbacks cb 0x7f404c00bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4054f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4054f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39234/0 (socket says 172.21.15.45:39234) 2023-10-17T01:19:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4055f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x5568098483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f405579b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809846e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4054f9a700 1 -- 172.21.15.45:0/2992220544 learned_addr learned my addr 172.21.15.45:0/2992220544 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4055f9c700 1 -- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 msgr2=0x5568098473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4055f9c700 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4055f9c700 1 -- 172.21.15.45:0/2992220544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 msgr2=0x556809846e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4055f9c700 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809846e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f4055f9c700 1 -- 172.21.15.45:0/2992220544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556809599c00 con 0x55680983bc60 2023-10-17T01:19:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.071+0000 7f405579b700 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809846e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f4055f9c700 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x5568098483c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x556809841940 tx=0x7f40480c9750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b682986015bbf426 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f4038ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f4038ff9700 1 -- 172.21.15.45:0/2992220544 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480b73a0 con 0x55680983bc60 2023-10-17T01:19:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4048098c10 con 0x55680983bc60 2023-10-17T01:19:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f4038ff9700 1 -- 172.21.15.45:0/2992220544 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4048004ad0 con 0x55680983bc60 2023-10-17T01:19:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f4038ff9700 1 -- 172.21.15.45:0/2992220544 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40480999b0 con 0x55680983bc60 2023-10-17T01:19:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f406b29db80 10 client.25921 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5568095af250 con 0x55680983bc60 2023-10-17T01:19:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.072+0000 7f4038ff9700 1 -- 172.21.15.45:0/2992220544 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4048098c10 con 0x55680983bc60 2023-10-17T01:19:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f4038ff9700 1 -- 172.21.15.45:0/2992220544 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f40480950a0 con 0x55680983bc60 2023-10-17T01:19:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f406b29db80 10 client.25921 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f406b29db80 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f406b29db80 20 client.25921 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f406b29db80 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5568098a1b40 0x5568098a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f406b29db80 4 client.25921 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:19:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55680977bce0 con 0x5568098a1b40 2023-10-17T01:19:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f405579b700 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5568098a1b40 0x5568098a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f405579b700 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5568098a1b40 0x5568098a3f30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcbb6846f4aff41 server_cookie=ef2c2bc9bd308c36 in_seq=0 out_seq=0 2023-10-17T01:19:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.073+0000 7f4038ff9700 10 client.25921 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.087+0000 7f4038ff9700 1 -- 172.21.15.45:0/2992220544 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55680977bce0 con 0x5568098a1b40 2023-10-17T01:19:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.087+0000 7f4038ff9700 10 client.25921 handle_command_reply: tid=0 2023-10-17T01:19:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.087+0000 7f406b29db80 1 client.25921 shutdown 2023-10-17T01:19:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.087+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 msgr2=0x5568098483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.087+0000 7f406b29db80 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x5568098483c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x556809841940 tx=0x7f40480c9750 comp rx=0 tx=0).stop 2023-10-17T01:19:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 shutdown_connections 2023-10-17T01:19:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55680983bc60 0x5568098483c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556809831aa0 0x556809846e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5568098a1b40 0x5568098a3f30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 --2- 172.21.15.45:0/2992220544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55680983b300 0x5568098473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 >> 172.21.15.45:0/2992220544 conn(0x556809791070 msgr2=0x556809831e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f4038ff9700 0 client.25921 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.088+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 shutdown_connections 2023-10-17T01:19:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.089+0000 7f406b29db80 1 -- 172.21.15.45:0/2992220544 wait complete. 2023-10-17T01:19:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.089+0000 7f406b29db80 20 client.25921 trim_cache size 0 max 16384 2023-10-17T01:19:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:19:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:19:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:19:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:19:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f403c0a0c70 msgr2=0x7f403c0a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f403c0a0c70 0x7f403c0a3130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f404c005120 tx=0x7f404c005150 comp rx=0 tx=0).stop 2023-10-17T01:19:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4050106a50 msgr2=0x7f4050197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4050106a50 0x7f4050197f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f404800efc0 tx=0x7f404800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 shutdown_connections 2023-10-17T01:19:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f403c0a0c70 0x7f403c0a3130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4050106a50 0x7f4050197f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40501017f0 0x7f4050196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 --2- 172.21.15.45:0/4021583285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050104120 0x7f4050197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 >> 172.21.15.45:0/4021583285 conn(0x7f40500fb1a0 msgr2=0x7f4050102460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 shutdown_connections 2023-10-17T01:19:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:05.091+0000 7f40579ff700 1 -- 172.21.15.45:0/4021583285 wait complete. 2023-10-17T01:19:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 235.0 s, 1325.0 tps, lat 75.702 ms stddev 141.088 2023-10-17T01:19:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:05 smithi181 ceph-mon[134520]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 1.15k op/s 2023-10-17T01:19:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:05 smithi045 ceph-mon[98474]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 1.15k op/s 2023-10-17T01:19:06.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:05 smithi060 ceph-mon[134136]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 1.15k op/s 2023-10-17T01:19:06.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:06 smithi181 ceph-mon[134520]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 1.33k op/s 2023-10-17T01:19:07.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:06 smithi045 ceph-mon[98474]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 1.33k op/s 2023-10-17T01:19:07.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:06 smithi060 ceph-mon[134136]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 1.33k op/s 2023-10-17T01:19:09.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:09 smithi181 ceph-mon[134520]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 1.49k op/s 2023-10-17T01:19:10.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:09 smithi045 ceph-mon[98474]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 1.49k op/s 2023-10-17T01:19:10.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:09 smithi060 ceph-mon[134136]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 1.49k op/s 2023-10-17T01:19:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 240.0 s, 2004.4 tps, lat 50.318 ms stddev 64.422 2023-10-17T01:19:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:10 smithi045 ceph-mon[98474]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 MiB/s wr, 2.23k op/s 2023-10-17T01:19:11.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:10 smithi060 ceph-mon[134136]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 MiB/s wr, 2.23k op/s 2023-10-17T01:19:11.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:10 smithi181 ceph-mon[134520]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 MiB/s wr, 2.23k op/s 2023-10-17T01:19:13.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:13 smithi181 ceph-mon[134520]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 MiB/s wr, 2.16k op/s 2023-10-17T01:19:14.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:13 smithi045 ceph-mon[98474]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 MiB/s wr, 2.16k op/s 2023-10-17T01:19:14.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:13 smithi060 ceph-mon[134136]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 MiB/s wr, 2.16k op/s 2023-10-17T01:19:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 245.0 s, 2209.0 tps, lat 44.789 ms stddev 50.619 2023-10-17T01:19:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:15 smithi181 ceph-mon[134520]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 2.15k op/s 2023-10-17T01:19:16.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:15 smithi045 ceph-mon[98474]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 2.15k op/s 2023-10-17T01:19:16.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:15 smithi060 ceph-mon[134136]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 2.15k op/s 2023-10-17T01:19:17.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:17 smithi181 ceph-mon[134520]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 1.96k op/s 2023-10-17T01:19:18.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:17 smithi045 ceph-mon[98474]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 1.96k op/s 2023-10-17T01:19:18.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:17 smithi060 ceph-mon[134136]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 1.96k op/s 2023-10-17T01:19:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 -- 172.21.15.45:0/3941605764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 msgr2=0x7f157810f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 --2- 172.21.15.45:0/3941605764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f157810f090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1568009990 tx=0x7f1568009ca0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 -- 172.21.15.45:0/3941605764 shutdown_connections 2023-10-17T01:19:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 --2- 172.21.15.45:0/3941605764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157810f5d0 0x7f15781119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 --2- 172.21.15.45:0/3941605764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1578107d50 0x7f1578108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 --2- 172.21.15.45:0/3941605764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f157810f090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.348+0000 7f157cfff700 1 -- 172.21.15.45:0/3941605764 >> 172.21.15.45:0/3941605764 conn(0x7f157806db80 msgr2=0x7f157806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.349+0000 7f157cfff700 1 -- 172.21.15.45:0/3941605764 shutdown_connections 2023-10-17T01:19:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.349+0000 7f157cfff700 1 -- 172.21.15.45:0/3941605764 wait complete. 2023-10-17T01:19:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.352+0000 7f157cfff700 1 Processor -- start 2023-10-17T01:19:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.354+0000 7f157cfff700 1 -- start start 2023-10-17T01:19:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.354+0000 7f157cfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1578107d50 0x7f15781a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.354+0000 7f157cfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f15781a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f157cfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157810f5d0 0x7f15781a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f157cfff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781a2f00 con 0x7f157810f5d0 2023-10-17T01:19:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f157cfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781a3070 con 0x7f15781086b0 2023-10-17T01:19:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f157cfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781a3230 con 0x7f1578107d50 2023-10-17T01:19:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f15781a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f157659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1578107d50 0x7f15781a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f15781a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47398/0 (socket says 172.21.15.45:47398) 2023-10-17T01:19:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 -- 172.21.15.45:0/267988745 learned_addr learned my addr 172.21.15.45:0/267988745 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 -- 172.21.15.45:0/267988745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1578107d50 msgr2=0x7f15781a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1578107d50 0x7f15781a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 -- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157810f5d0 msgr2=0x7f15781a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157810f5d0 0x7f15781a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.355+0000 7f1575d9b700 1 -- 172.21.15.45:0/267988745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009c40 con 0x7f15781086b0 2023-10-17T01:19:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.356+0000 7f1575d9b700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f15781a1a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f156800b680 tx=0x7f156800f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.357+0000 7f15677fe700 1 -- 172.21.15.45:0/267988745 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156801d070 con 0x7f15781086b0 2023-10-17T01:19:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.357+0000 7f15677fe700 1 -- 172.21.15.45:0/267988745 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156800fc30 con 0x7f15781086b0 2023-10-17T01:19:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.357+0000 7f15677fe700 1 -- 172.21.15.45:0/267988745 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568017650 con 0x7f15781086b0 2023-10-17T01:19:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.357+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15680095f0 con 0x7f15781086b0 2023-10-17T01:19:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.361+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15781a36f0 con 0x7f15781086b0 2023-10-17T01:19:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.371+0000 7f15677fe700 1 -- 172.21.15.45:0/267988745 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f1568022470 con 0x7f15781086b0 2023-10-17T01:19:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.372+0000 7f15909f0b80 1 Processor -- start 2023-10-17T01:19:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.373+0000 7f15909f0b80 1 -- start start 2023-10-17T01:19:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.374+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a564c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.374+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a565790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.374+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a566210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.374+0000 7f15909f0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a4bc9b0 con 0x56427a564830 2023-10-17T01:19:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.374+0000 7f15909f0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a4ba370 con 0x56427a565d90 2023-10-17T01:19:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.374+0000 7f15909f0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a4b5c60 con 0x56427a565190 2023-10-17T01:19:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a565790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a566210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a565790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38238/0 (socket says 172.21.15.45:38238) 2023-10-17T01:19:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 -- 172.21.15.45:0/3881949705 learned_addr learned my addr 172.21.15.45:0/3881949705 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 -- 172.21.15.45:0/3881949705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 msgr2=0x56427a566210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a566210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 -- 172.21.15.45:0/3881949705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 msgr2=0x56427a564c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:19:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a564c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f157659c700 1 -- 172.21.15.45:0/3881949705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56427a2e3d80 con 0x56427a565190 2023-10-17T01:19:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.375+0000 7f1575d9b700 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a566210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.376+0000 7f157659c700 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a565790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15700094c0 tx=0x7f1570009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0229dac80b38ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.376+0000 7f1564ff9700 1 -- 172.21.15.45:0/3881949705 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570013780 con 0x56427a565190 2023-10-17T01:19:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.376+0000 7f1564ff9700 1 -- 172.21.15.45:0/3881949705 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f157000f070 con 0x56427a565190 2023-10-17T01:19:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.376+0000 7f1564ff9700 1 -- 172.21.15.45:0/3881949705 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570014a80 con 0x56427a565190 2023-10-17T01:19:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.377+0000 7f15677fe700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15600a0cf0 0x7f15600a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.377+0000 7f15677fe700 1 -- 172.21.15.45:0/267988745 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f15680cd6b0 con 0x7f15781086b0 2023-10-17T01:19:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 -- 172.21.15.45:0/3881949705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 msgr2=0x56427a565790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a565790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15700094c0 tx=0x7f1570009920 comp rx=0 tx=0).stop 2023-10-17T01:19:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 -- 172.21.15.45:0/3881949705 shutdown_connections 2023-10-17T01:19:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a566210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a564c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3881949705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a565790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 -- 172.21.15.45:0/3881949705 >> 172.21.15.45:0/3881949705 conn(0x56427a4c4070 msgr2=0x56427a55f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.378+0000 7f15909f0b80 1 -- 172.21.15.45:0/3881949705 shutdown_connections 2023-10-17T01:19:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.379+0000 7f1576d9d700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15600a0cf0 0x7f15600a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.379+0000 7f15909f0b80 1 -- 172.21.15.45:0/3881949705 wait complete. 2023-10-17T01:19:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.380+0000 7f15909f0b80 1 -- start start 2023-10-17T01:19:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.380+0000 7f15909f0b80 1 Processor -- start 2023-10-17T01:19:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.382+0000 7f1576d9d700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15600a0cf0 0x7f15600a31b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1578108d30 tx=0x7f156c005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.383+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a576be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.383+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a577120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.383+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a57e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.383+0000 7f15909f0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a4524b0 con 0x56427a564830 2023-10-17T01:19:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.383+0000 7f15909f0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a452150 con 0x56427a565190 2023-10-17T01:19:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.383+0000 7f15909f0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a3398c0 con 0x56427a565d90 2023-10-17T01:19:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.384+0000 7f15909f0b80 10 client.0 _ll_register_callbacks cb 0x56427a39fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.384+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a577120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.384+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a577120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47422/0 (socket says 172.21.15.45:47422) 2023-10-17T01:19:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.384+0000 7f1575d9b700 1 -- 172.21.15.45:0/3273141583 learned_addr learned my addr 172.21.15.45:0/3273141583 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.384+0000 7f1576d9d700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a57e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.384+0000 7f157659c700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a576be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.385+0000 7f1576d9d700 1 -- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 msgr2=0x56427a577120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.385+0000 7f1576d9d700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a577120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.385+0000 7f1576d9d700 1 -- 172.21.15.45:0/3273141583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 msgr2=0x56427a576be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.385+0000 7f1576d9d700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a576be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.385+0000 7f1576d9d700 1 -- 172.21.15.45:0/3273141583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15781a36f0 con 0x56427a565d90 2023-10-17T01:19:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.385+0000 7f157659c700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a576be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.387+0000 7f1576d9d700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a57e160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c5b0 tx=0x7f156c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a78e00561cc8ad server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.387+0000 7f155dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.387+0000 7f155dffb700 1 -- 172.21.15.45:0/3273141583 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00ae60 con 0x56427a565d90 2023-10-17T01:19:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.387+0000 7f155dffb700 1 -- 172.21.15.45:0/3273141583 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f156c004830 con 0x56427a565d90 2023-10-17T01:19:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.387+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56427a4e0cb0 con 0x56427a565d90 2023-10-17T01:19:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f155dffb700 1 -- 172.21.15.45:0/3273141583 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156c0049f0 con 0x56427a565d90 2023-10-17T01:19:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 10 client.25930 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f1570009170 con 0x56427a565d90 2023-10-17T01:19:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f155dffb700 1 -- 172.21.15.45:0/3273141583 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c02b020 con 0x56427a565d90 2023-10-17T01:19:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f155dffb700 1 -- 172.21.15.45:0/3273141583 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f156c025450 con 0x56427a565d90 2023-10-17T01:19:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 20 client.25930 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a582a60 0x56427a584e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 4 client.25930 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.388+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56427a533cd0 con 0x56427a582a60 2023-10-17T01:19:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.389+0000 7f157659c700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a582a60 0x56427a584e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.390+0000 7f157659c700 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a582a60 0x56427a584e80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607bf5fdd2b77efd server_cookie=4f918b707a6e383c in_seq=0 out_seq=0 2023-10-17T01:19:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.390+0000 7f155dffb700 10 client.25930 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.392+0000 7f155dffb700 1 -- 172.21.15.45:0/3273141583 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x56427a533cd0 con 0x56427a582a60 2023-10-17T01:19:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.392+0000 7f155dffb700 10 client.25930 handle_command_reply: tid=0 2023-10-17T01:19:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.392+0000 7f15909f0b80 1 client.25930 shutdown 2023-10-17T01:19:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.392+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 msgr2=0x56427a57e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.392+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a57e160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c5b0 tx=0x7f156c004750 comp rx=0 tx=0).stop 2023-10-17T01:19:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 shutdown_connections 2023-10-17T01:19:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a57e160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a576be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a582a60 0x56427a584e80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 --2- 172.21.15.45:0/3273141583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a577120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 >> 172.21.15.45:0/3273141583 conn(0x56427a4c4070 msgr2=0x56427a55ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f155dffb700 0 client.25930 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 shutdown_connections 2023-10-17T01:19:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.393+0000 7f15909f0b80 1 -- 172.21.15.45:0/3273141583 wait complete. 2023-10-17T01:19:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.394+0000 7f15909f0b80 20 client.25930 trim_cache size 0 max 16384 2023-10-17T01:19:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.420+0000 7f15909f0b80 1 Processor -- start 2023-10-17T01:19:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.420+0000 7f15909f0b80 1 -- start start 2023-10-17T01:19:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a584e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a55ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a56c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f15909f0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a533cd0 con 0x56427a564830 2023-10-17T01:19:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a56c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f15909f0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a577dd0 con 0x56427a565d90 2023-10-17T01:19:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f15909f0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a5599e0 con 0x56427a565190 2023-10-17T01:19:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a55ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a56c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47424/0 (socket says 172.21.15.45:47424) 2023-10-17T01:19:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f1576d9d700 1 -- 172.21.15.45:0/313232244 learned_addr learned my addr 172.21.15.45:0/313232244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.421+0000 7f157659c700 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a584e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.422+0000 7f1576d9d700 1 -- 172.21.15.45:0/313232244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 msgr2=0x56427a55ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.422+0000 7f1576d9d700 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a55ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.422+0000 7f1576d9d700 1 -- 172.21.15.45:0/313232244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 msgr2=0x56427a584e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.422+0000 7f1576d9d700 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a584e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.422+0000 7f1576d9d700 1 -- 172.21.15.45:0/313232244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56427a2ccc00 con 0x56427a565d90 2023-10-17T01:19:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.422+0000 7f1576d9d700 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a56c220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f156c025830 tx=0x7f156c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3209890957e22fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.423+0000 7f15657fa700 1 -- 172.21.15.45:0/313232244 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00fe70 con 0x56427a565d90 2023-10-17T01:19:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.423+0000 7f15657fa700 1 -- 172.21.15.45:0/313232244 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c02b030 con 0x56427a565d90 2023-10-17T01:19:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.423+0000 7f15657fa700 1 -- 172.21.15.45:0/313232244 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c005040 con 0x56427a565d90 2023-10-17T01:19:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.424+0000 7f15909f0b80 1 -- 172.21.15.45:0/313232244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 msgr2=0x56427a56c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.424+0000 7f15909f0b80 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a56c220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f156c025830 tx=0x7f156c005fd0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 -- 172.21.15.45:0/313232244 shutdown_connections 2023-10-17T01:19:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565d90 0x56427a56c220 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a584e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 --2- 172.21.15.45:0/313232244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565190 0x56427a55ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 -- 172.21.15.45:0/313232244 >> 172.21.15.45:0/313232244 conn(0x56427a4c4070 msgr2=0x56427a55f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 -- 172.21.15.45:0/313232244 shutdown_connections 2023-10-17T01:19:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.425+0000 7f15909f0b80 1 -- 172.21.15.45:0/313232244 wait complete. 2023-10-17T01:19:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.426+0000 7f15909f0b80 1 -- start start 2023-10-17T01:19:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.427+0000 7f15909f0b80 1 Processor -- start 2023-10-17T01:19:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a57b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a57bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a5d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a5d2f50 con 0x56427a564830 2023-10-17T01:19:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a5d30c0 con 0x56427a565190 2023-10-17T01:19:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a5d3280 con 0x56427a565d90 2023-10-17T01:19:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f15909f0b80 10 client.0 _ll_register_callbacks cb 0x7f156c00cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.428+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a57bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.429+0000 7f157659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a57b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.429+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a5d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.429+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a57bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47432/0 (socket says 172.21.15.45:47432) 2023-10-17T01:19:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.429+0000 7f1575d9b700 1 -- 172.21.15.45:0/2097280090 learned_addr learned my addr 172.21.15.45:0/2097280090 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.429+0000 7f157659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a57b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52170/0 (socket says 172.21.15.45:52170) 2023-10-17T01:19:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.429+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a5d29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38266/0 (socket says 172.21.15.45:38266) 2023-10-17T01:19:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f1575d9b700 1 -- 172.21.15.45:0/2097280090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 msgr2=0x56427a5d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f1575d9b700 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a5d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f1575d9b700 1 -- 172.21.15.45:0/2097280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 msgr2=0x56427a57b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f1575d9b700 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a57b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f1575d9b700 1 -- 172.21.15.45:0/2097280090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1570009170 con 0x56427a565190 2023-10-17T01:19:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f157659c700 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a57b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f1575d9b700 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a57bc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1568009ce0 tx=0x7f15680d3190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec323805f9330826 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.430+0000 7f155dffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:19:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.431+0000 7f155dffb700 1 -- 172.21.15.45:0/2097280090 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156809a7e0 con 0x56427a565190 2023-10-17T01:19:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.431+0000 7f155dffb700 1 -- 172.21.15.45:0/2097280090 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f156809b580 con 0x56427a565190 2023-10-17T01:19:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.431+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f157000f070 con 0x56427a565190 2023-10-17T01:19:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.431+0000 7f155dffb700 1 -- 172.21.15.45:0/2097280090 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156809ae80 con 0x56427a565190 2023-10-17T01:19:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.431+0000 7f15909f0b80 10 client.25973 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.431+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56427a30a1f0 con 0x56427a565190 2023-10-17T01:19:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f155dffb700 1 -- 172.21.15.45:0/2097280090 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15680b7080 con 0x56427a565190 2023-10-17T01:19:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f155dffb700 1 -- 172.21.15.45:0/2097280090 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f156809a370 con 0x56427a565190 2023-10-17T01:19:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f15909f0b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f15909f0b80 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f15909f0b80 20 client.25973 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f15909f0b80 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a5d49d0 0x56427a5d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f15909f0b80 4 client.25973 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:19:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.432+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56427a4aece0 con 0x56427a5d49d0 2023-10-17T01:19:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.433+0000 7f157659c700 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a5d49d0 0x56427a5d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.434+0000 7f155dffb700 10 client.25973 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.434+0000 7f157659c700 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a5d49d0 0x56427a5d6df0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8974788555593dfb server_cookie=9708f827bf969a08 in_seq=0 out_seq=0 2023-10-17T01:19:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.449+0000 7f155dffb700 1 -- 172.21.15.45:0/2097280090 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56427a4aece0 con 0x56427a5d49d0 2023-10-17T01:19:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.449+0000 7f155dffb700 10 client.25973 handle_command_reply: tid=0 2023-10-17T01:19:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.449+0000 7f15909f0b80 1 client.25973 shutdown 2023-10-17T01:19:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 msgr2=0x56427a57bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a57bc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1568009ce0 tx=0x7f15680d3190 comp rx=0 tx=0).stop 2023-10-17T01:19:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 shutdown_connections 2023-10-17T01:19:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56427a565d90 0x56427a5d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56427a564830 0x56427a57b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56427a5d49d0 0x56427a5d6df0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 --2- 172.21.15.45:0/2097280090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56427a565190 0x56427a57bc40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 >> 172.21.15.45:0/2097280090 conn(0x56427a4c4070 msgr2=0x56427a56d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.450+0000 7f155dffb700 0 client.25973 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.451+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 shutdown_connections 2023-10-17T01:19:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.451+0000 7f15909f0b80 1 -- 172.21.15.45:0/2097280090 wait complete. 2023-10-17T01:19:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.451+0000 7f15909f0b80 20 client.25973 trim_cache size 0 max 16384 2023-10-17T01:19:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:19:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:19:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:19:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:19:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.452+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15600a0cf0 msgr2=0x7f15600a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.452+0000 7f157cfff700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15600a0cf0 0x7f15600a31b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1578108d30 tx=0x7f156c005ce0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.452+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 msgr2=0x7f15781a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.452+0000 7f157cfff700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f15781a1a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f156800b680 tx=0x7f156800f460 comp rx=0 tx=0).stop 2023-10-17T01:19:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.453+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 shutdown_connections 2023-10-17T01:19:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.453+0000 7f157cfff700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f15600a0cf0 0x7f15600a31b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.453+0000 7f157cfff700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157810f5d0 0x7f15781a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.453+0000 7f157cfff700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1578107d50 0x7f15781a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.453+0000 7f157cfff700 1 --2- 172.21.15.45:0/267988745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15781086b0 0x7f15781a1a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.453+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 >> 172.21.15.45:0/267988745 conn(0x7f157806db80 msgr2=0x7f1578108e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.454+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 shutdown_connections 2023-10-17T01:19:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:18.454+0000 7f157cfff700 1 -- 172.21.15.45:0/267988745 wait complete. 2023-10-17T01:19:19.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:19 smithi181 ceph-mon[134520]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.59k op/s 2023-10-17T01:19:20.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:19 smithi045 ceph-mon[98474]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.59k op/s 2023-10-17T01:19:20.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:19 smithi060 ceph-mon[134136]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.59k op/s 2023-10-17T01:19:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 250.0 s, 2063.2 tps, lat 48.888 ms stddev 54.813 2023-10-17T01:19:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:21 smithi181 ceph-mon[134520]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 1.50k op/s 2023-10-17T01:19:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:21 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:19:22.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:21 smithi045 ceph-mon[98474]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 1.50k op/s 2023-10-17T01:19:22.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:21 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:19:22.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:21 smithi060 ceph-mon[134136]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 1.50k op/s 2023-10-17T01:19:22.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:21 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:19:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:23 smithi181 ceph-mon[134520]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 782 op/s 2023-10-17T01:19:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:19:23.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:19:23.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:19:24.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:23 smithi045 ceph-mon[98474]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 782 op/s 2023-10-17T01:19:24.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:19:24.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:19:24.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:19:24.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:23 smithi060 ceph-mon[134136]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 782 op/s 2023-10-17T01:19:24.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:19:24.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:19:24.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:19:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 255.0 s, 2323.6 tps, lat 43.021 ms stddev 47.328 2023-10-17T01:19:25.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:25 smithi181 ceph-mon[134520]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 766 op/s 2023-10-17T01:19:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:25 smithi045 ceph-mon[98474]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 766 op/s 2023-10-17T01:19:26.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:25 smithi060 ceph-mon[134136]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 766 op/s 2023-10-17T01:19:27.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:27 smithi181 ceph-mon[134520]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 690 op/s 2023-10-17T01:19:28.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:27 smithi045 ceph-mon[98474]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 690 op/s 2023-10-17T01:19:28.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:27 smithi060 ceph-mon[134136]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 690 op/s 2023-10-17T01:19:29.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:29 smithi181 ceph-mon[134520]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 679 op/s 2023-10-17T01:19:30.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:29 smithi045 ceph-mon[98474]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 679 op/s 2023-10-17T01:19:30.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:29 smithi060 ceph-mon[134136]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 679 op/s 2023-10-17T01:19:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 260.0 s, 1312.2 tps, lat 75.700 ms stddev 81.604 2023-10-17T01:19:30.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:19:31.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.074+0000 7ff7527ff700 1 -- 172.21.15.45:0/1683090148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c0737b0 msgr2=0x7ff74c073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.074+0000 7ff7527ff700 1 --2- 172.21.15.45:0/1683090148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c0737b0 0x7ff74c073c30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff740009990 tx=0x7ff740009ca0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 -- 172.21.15.45:0/1683090148 shutdown_connections 2023-10-17T01:19:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 --2- 172.21.15.45:0/1683090148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c108c00 0x7ff74c10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 --2- 172.21.15.45:0/1683090148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c074d90 0x7ff74c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 --2- 172.21.15.45:0/1683090148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c0737b0 0x7ff74c073c30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 -- 172.21.15.45:0/1683090148 >> 172.21.15.45:0/1683090148 conn(0x7ff74c0fc290 msgr2=0x7ff74c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 -- 172.21.15.45:0/1683090148 shutdown_connections 2023-10-17T01:19:31.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.076+0000 7ff7527ff700 1 -- 172.21.15.45:0/1683090148 wait complete. 2023-10-17T01:19:31.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.080+0000 7ff7527ff700 1 Processor -- start 2023-10-17T01:19:31.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.080+0000 7ff7527ff700 1 -- start start 2023-10-17T01:19:31.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff7527ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 0x7ff74c19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff7527ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c074d90 0x7ff74c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff7527ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c108c00 0x7ff74c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff7527ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1a4c30 con 0x7ff74c074d90 2023-10-17T01:19:31.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff7527ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1a4d70 con 0x7ff74c0737b0 2023-10-17T01:19:31.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff7527ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1a4f00 con 0x7ff74c108c00 2023-10-17T01:19:31.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 0x7ff74c19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 0x7ff74c19d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40308/0 (socket says 172.21.15.45:40308) 2023-10-17T01:19:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.083+0000 7ff74bfff700 1 -- 172.21.15.45:0/2735011354 learned_addr learned my addr 172.21.15.45:0/2735011354 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.084+0000 7ff74bfff700 1 -- 172.21.15.45:0/2735011354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c108c00 msgr2=0x7ff74c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.084+0000 7ff74bfff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c108c00 0x7ff74c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.084+0000 7ff74bfff700 1 -- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c074d90 msgr2=0x7ff74c19d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.084+0000 7ff74bfff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c074d90 0x7ff74c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.084+0000 7ff74bfff700 1 -- 172.21.15.45:0/2735011354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7400095f0 con 0x7ff74c0737b0 2023-10-17T01:19:31.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.084+0000 7ff74bfff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 0x7ff74c19d0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e9a0 tx=0x7ff73c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.087+0000 7ff7497fa700 1 -- 172.21.15.45:0/2735011354 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c009790 con 0x7ff74c0737b0 2023-10-17T01:19:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.087+0000 7ff7497fa700 1 -- 172.21.15.45:0/2735011354 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff73c00cbd0 con 0x7ff74c0737b0 2023-10-17T01:19:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.087+0000 7ff7497fa700 1 -- 172.21.15.45:0/2735011354 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c018680 con 0x7ff74c0737b0 2023-10-17T01:19:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.087+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1a51e0 con 0x7ff74c0737b0 2023-10-17T01:19:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.087+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a56f0 con 0x7ff74c0737b0 2023-10-17T01:19:31.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.095+0000 7ff76603fb80 1 Processor -- start 2023-10-17T01:19:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.096+0000 7ff7497fa700 1 -- 172.21.15.45:0/2735011354 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff73c010430 con 0x7ff74c0737b0 2023-10-17T01:19:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 -- start start 2023-10-17T01:19:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae95fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6ae96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6ae97730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6adeda10 con 0x55aa6ae95ba0 2023-10-17T01:19:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6adeb3d0 con 0x55aa6ae96500 2023-10-17T01:19:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.097+0000 7ff76603fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ade6cc0 con 0x55aa6ae972b0 2023-10-17T01:19:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6ae97730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6ae97730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44596/0 (socket says 172.21.15.45:44596) 2023-10-17T01:19:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 -- 172.21.15.45:0/4002467153 learned_addr learned my addr 172.21.15.45:0/4002467153 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 -- 172.21.15.45:0/4002467153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 msgr2=0x55aa6ae96d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:19:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6ae96d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 -- 172.21.15.45:0/4002467153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 msgr2=0x55aa6ae95fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:19:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae95fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.098+0000 7ff74b7fe700 1 -- 172.21.15.45:0/4002467153 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa6ac14de0 con 0x55aa6ae972b0 2023-10-17T01:19:31.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.099+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6ae97730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7400095b0 tx=0x7ff740005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8f2518255c3d010 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.102+0000 7ff7327fc700 1 -- 172.21.15.45:0/4002467153 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74001d070 con 0x55aa6ae972b0 2023-10-17T01:19:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.102+0000 7ff7327fc700 1 -- 172.21.15.45:0/4002467153 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff74000bdd0 con 0x55aa6ae972b0 2023-10-17T01:19:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.102+0000 7ff7327fc700 1 -- 172.21.15.45:0/4002467153 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74000fd60 con 0x55aa6ae972b0 2023-10-17T01:19:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.103+0000 7ff76603fb80 1 -- 172.21.15.45:0/4002467153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 msgr2=0x55aa6ae97730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.103+0000 7ff76603fb80 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6ae97730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7400095b0 tx=0x7ff740005c00 comp rx=0 tx=0).stop 2023-10-17T01:19:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.103+0000 7ff7497fa700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff7340a0ba0 0x7ff7340a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.103+0000 7ff7497fa700 1 -- 172.21.15.45:0/2735011354 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff73c014070 con 0x7ff74c0737b0 2023-10-17T01:19:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.104+0000 7ff76603fb80 1 -- 172.21.15.45:0/4002467153 shutdown_connections 2023-10-17T01:19:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.104+0000 7ff76603fb80 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6ae97730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.104+0000 7ff76603fb80 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae95fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.104+0000 7ff76603fb80 1 --2- 172.21.15.45:0/4002467153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6ae96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.104+0000 7ff76603fb80 1 -- 172.21.15.45:0/4002467153 >> 172.21.15.45:0/4002467153 conn(0x55aa6adf50d0 msgr2=0x55aa6ae90950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.105+0000 7ff76603fb80 1 -- 172.21.15.45:0/4002467153 shutdown_connections 2023-10-17T01:19:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.105+0000 7ff76603fb80 1 -- 172.21.15.45:0/4002467153 wait complete. 2023-10-17T01:19:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.106+0000 7ff76603fb80 1 -- start start 2023-10-17T01:19:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.106+0000 7ff76603fb80 1 Processor -- start 2023-10-17T01:19:31.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.108+0000 7ff750d9c700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff7340a0ba0 0x7ff7340a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.110+0000 7ff750d9c700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff7340a0ba0 0x7ff7340a3060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55aa6ae95940 tx=0x7ff74400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.111+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae9f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.111+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae96500 0x55aa6ae9fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.111+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 0x55aa6aea6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.113+0000 7ff76603fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ad83510 con 0x55aa6ae96500 2023-10-17T01:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.113+0000 7ff76603fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ad831b0 con 0x55aa6ae972b0 2023-10-17T01:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.113+0000 7ff750d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 0x55aa6aea6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.113+0000 7ff750d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 0x55aa6aea6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40320/0 (socket says 172.21.15.45:40320) 2023-10-17T01:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.113+0000 7ff750d9c700 1 -- 172.21.15.45:0/445365287 learned_addr learned my addr 172.21.15.45:0/445365287 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ac6a920 con 0x55aa6ae95ba0 2023-10-17T01:19:31.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff76603fb80 10 client.0 _ll_register_callbacks cb 0x55aa6acd0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:31.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff74bfff700 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae9f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff750d9c700 1 -- 172.21.15.45:0/445365287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae95ba0 msgr2=0x55aa6ae9f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff750d9c700 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae9f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff750d9c700 1 -- 172.21.15.45:0/445365287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae96500 msgr2=0x55aa6ae9fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff750d9c700 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae96500 0x55aa6ae9fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.114+0000 7ff750d9c700 1 -- 172.21.15.45:0/445365287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c10bcb0 con 0x55aa6ae972b0 2023-10-17T01:19:31.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.115+0000 7ff750d9c700 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 0x55aa6aea6e40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff744015dc0 tx=0x7ff744004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd765ab394b3adf server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.115+0000 7ff72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:19:31.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.116+0000 7ff72b7fe700 1 -- 172.21.15.45:0/445365287 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744015ea0 con 0x55aa6ae972b0 2023-10-17T01:19:31.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.116+0000 7ff72b7fe700 1 -- 172.21.15.45:0/445365287 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff744013be0 con 0x55aa6ae972b0 2023-10-17T01:19:31.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.116+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa6ae11d10 con 0x55aa6ae972b0 2023-10-17T01:19:31.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.116+0000 7ff72b7fe700 1 -- 172.21.15.45:0/445365287 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74400eeb0 con 0x55aa6ae972b0 2023-10-17T01:19:31.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.116+0000 7ff76603fb80 10 client.25982 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:31.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.116+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55aa6abfdc60 con 0x55aa6ae972b0 2023-10-17T01:19:31.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff72b7fe700 1 -- 172.21.15.45:0/445365287 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa6ae11d10 con 0x55aa6ae972b0 2023-10-17T01:19:31.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff72b7fe700 1 -- 172.21.15.45:0/445365287 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff74401f3f0 con 0x55aa6ae972b0 2023-10-17T01:19:31.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff76603fb80 10 client.25982 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:31.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff76603fb80 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:31.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff76603fb80 20 client.25982 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:31.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff76603fb80 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aeab7a0 0x55aa6aeadbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff76603fb80 4 client.25982 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:31.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.117+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa6ae64d30 con 0x55aa6aeab7a0 2023-10-17T01:19:31.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.118+0000 7ff74bfff700 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aeab7a0 0x55aa6aeadbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.119+0000 7ff74bfff700 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aeab7a0 0x55aa6aeadbc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2268dd869bac82f9 server_cookie=3ac00c6441dbc21d in_seq=0 out_seq=0 2023-10-17T01:19:31.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.120+0000 7ff72b7fe700 10 client.25982 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.121+0000 7ff72b7fe700 1 -- 172.21.15.45:0/445365287 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55aa6ae64d30 con 0x55aa6aeab7a0 2023-10-17T01:19:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.121+0000 7ff72b7fe700 10 client.25982 handle_command_reply: tid=0 2023-10-17T01:19:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.121+0000 7ff76603fb80 1 client.25982 shutdown 2023-10-17T01:19:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.121+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 msgr2=0x55aa6aea6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.121+0000 7ff76603fb80 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 0x55aa6aea6e40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff744015dc0 tx=0x7ff744004ad0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 shutdown_connections 2023-10-17T01:19:31.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aeab7a0 0x55aa6aeadbc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae972b0 0x55aa6aea6e40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae95ba0 0x55aa6ae9f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 --2- 172.21.15.45:0/445365287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae96500 0x55aa6ae9fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 >> 172.21.15.45:0/445365287 conn(0x55aa6adf50d0 msgr2=0x55aa6ae90020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff72b7fe700 0 client.25982 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.122+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 shutdown_connections 2023-10-17T01:19:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.123+0000 7ff76603fb80 1 -- 172.21.15.45:0/445365287 wait complete. 2023-10-17T01:19:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.123+0000 7ff76603fb80 20 client.25982 trim_cache size 0 max 16384 2023-10-17T01:19:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.160+0000 7ff76603fb80 1 Processor -- start 2023-10-17T01:19:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.160+0000 7ff76603fb80 1 -- start start 2023-10-17T01:19:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae96500 0x55aa6ae9a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae972b0 0x55aa6ae9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff76603fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ae8ae60 con 0x55aa6ae972b0 2023-10-17T01:19:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff76603fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ae9ed90 con 0x55aa6ae95ba0 2023-10-17T01:19:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff76603fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ae8d420 con 0x55aa6ae96500 2023-10-17T01:19:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40332/0 (socket says 172.21.15.45:40332) 2023-10-17T01:19:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff74bfff700 1 -- 172.21.15.45:0/1812559127 learned_addr learned my addr 172.21.15.45:0/1812559127 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.161+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae96500 0x55aa6ae9a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.162+0000 7ff74bfff700 1 -- 172.21.15.45:0/1812559127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae96500 msgr2=0x55aa6ae9a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.162+0000 7ff74bfff700 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae96500 0x55aa6ae9a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.162+0000 7ff74bfff700 1 -- 172.21.15.45:0/1812559127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae972b0 msgr2=0x55aa6ae9a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.162+0000 7ff74bfff700 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae972b0 0x55aa6ae9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.162+0000 7ff74bfff700 1 -- 172.21.15.45:0/1812559127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa6ae9e860 con 0x55aa6ae95ba0 2023-10-17T01:19:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.162+0000 7ff74bfff700 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea7610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0999d0 tx=0x7ff73c010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9483d05061f0448 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.163+0000 7ff732ffd700 1 -- 172.21.15.45:0/1812559127 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0b1070 con 0x55aa6ae95ba0 2023-10-17T01:19:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.163+0000 7ff732ffd700 1 -- 172.21.15.45:0/1812559127 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff73c099a40 con 0x55aa6ae95ba0 2023-10-17T01:19:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.163+0000 7ff732ffd700 1 -- 172.21.15.45:0/1812559127 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0b90c0 con 0x55aa6ae95ba0 2023-10-17T01:19:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 -- 172.21.15.45:0/1812559127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 msgr2=0x55aa6aea7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea7610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0999d0 tx=0x7ff73c010f90 comp rx=0 tx=0).stop 2023-10-17T01:19:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 -- 172.21.15.45:0/1812559127 shutdown_connections 2023-10-17T01:19:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae972b0 0x55aa6ae9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea7610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 --2- 172.21.15.45:0/1812559127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae96500 0x55aa6ae9a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 -- 172.21.15.45:0/1812559127 >> 172.21.15.45:0/1812559127 conn(0x55aa6adf50d0 msgr2=0x55aa6ae9d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 -- 172.21.15.45:0/1812559127 shutdown_connections 2023-10-17T01:19:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.164+0000 7ff76603fb80 1 -- 172.21.15.45:0/1812559127 wait complete. 2023-10-17T01:19:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.165+0000 7ff76603fb80 1 -- start start 2023-10-17T01:19:31.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.165+0000 7ff76603fb80 1 Processor -- start 2023-10-17T01:19:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6aea3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6aefd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6aea4650 con 0x55aa6ae95ba0 2023-10-17T01:19:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6aea47c0 con 0x55aa6ae96500 2023-10-17T01:19:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6aefd6b0 con 0x55aa6ae972b0 2023-10-17T01:19:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff76603fb80 10 client.0 _ll_register_callbacks cb 0x7ff744015790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff750d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6aefd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff750d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6aefd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44622/0 (socket says 172.21.15.45:44622) 2023-10-17T01:19:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff750d9c700 1 -- 172.21.15.45:0/2747193928 learned_addr learned my addr 172.21.15.45:0/2747193928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.166+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6aea3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74bfff700 1 -- 172.21.15.45:0/2747193928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 msgr2=0x55aa6aefd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74bfff700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6aefd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74bfff700 1 -- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 msgr2=0x55aa6aea3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74bfff700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6aea3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74bfff700 1 -- 172.21.15.45:0/2747193928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a56f0 con 0x55aa6ae95ba0 2023-10-17T01:19:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff750d9c700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6aefd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6aea3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff74bfff700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea38f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff73c09da30 tx=0x7ff73c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa71069d1b10fc67 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:19:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff72b7fe700 1 -- 172.21.15.45:0/2747193928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0b1040 con 0x55aa6ae95ba0 2023-10-17T01:19:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff74000bdd0 con 0x55aa6ae95ba0 2023-10-17T01:19:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.167+0000 7ff72b7fe700 1 -- 172.21.15.45:0/2747193928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff73c0a1ab0 con 0x55aa6ae95ba0 2023-10-17T01:19:31.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff72b7fe700 1 -- 172.21.15.45:0/2747193928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73c097de0 con 0x55aa6ae95ba0 2023-10-17T01:19:31.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 10 client.15759 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:31.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55aa6ac3b250 con 0x55aa6ae95ba0 2023-10-17T01:19:31.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff72b7fe700 1 -- 172.21.15.45:0/2747193928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff73c07e080 con 0x55aa6ae95ba0 2023-10-17T01:19:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff72b7fe700 1 -- 172.21.15.45:0/2747193928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff73c0a1620 con 0x55aa6ae95ba0 2023-10-17T01:19:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 10 client.15759 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 20 client.15759 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aefedf0 0x55aa6af01210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 4 client.15759 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:19:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aa6addfd40 con 0x55aa6aefedf0 2023-10-17T01:19:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.168+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aefedf0 0x55aa6af01210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.169+0000 7ff72b7fe700 10 client.15759 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.169+0000 7ff74b7fe700 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aefedf0 0x55aa6af01210 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58b940c271dd7c server_cookie=6e118b39cb52a5ea in_seq=0 out_seq=0 2023-10-17T01:19:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.169+0000 7ff72b7fe700 1 -- 172.21.15.45:0/2747193928 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55aa6addfd40 con 0x55aa6aefedf0 2023-10-17T01:19:31.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.169+0000 7ff72b7fe700 10 client.15759 handle_command_reply: tid=0 2023-10-17T01:19:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.169+0000 7ff76603fb80 1 client.15759 shutdown 2023-10-17T01:19:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 msgr2=0x55aa6aea38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea38f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff73c09da30 tx=0x7ff73c00cf40 comp rx=0 tx=0).stop 2023-10-17T01:19:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 shutdown_connections 2023-10-17T01:19:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa6ae972b0 0x55aa6aefd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa6ae95ba0 0x55aa6aea38f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55aa6aefedf0 0x55aa6af01210 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 --2- 172.21.15.45:0/2747193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aa6ae96500 0x55aa6aea3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff72b7fe700 0 client.15759 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 >> 172.21.15.45:0/2747193928 conn(0x55aa6adf50d0 msgr2=0x55aa6ae9a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 shutdown_connections 2023-10-17T01:19:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 1 -- 172.21.15.45:0/2747193928 wait complete. 2023-10-17T01:19:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.170+0000 7ff76603fb80 20 client.15759 trim_cache size 0 max 16384 2023-10-17T01:19:31.186 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:19:31.186 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:19:31.186 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:19:31.187 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:19:31.187 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:19:31.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:19:31.187 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:19:31.187 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:19:31.188 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:19:31.188 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:19:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff7340a0ba0 msgr2=0x7ff7340a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff7340a0ba0 0x7ff7340a3060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55aa6ae95940 tx=0x7ff74400b040 comp rx=0 tx=0).stop 2023-10-17T01:19:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 msgr2=0x7ff74c19d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 0x7ff74c19d0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e9a0 tx=0x7ff73c00ed60 comp rx=0 tx=0).stop 2023-10-17T01:19:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 shutdown_connections 2023-10-17T01:19:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff7340a0ba0 0x7ff7340a3060 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff74c108c00 0x7ff74c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74c0737b0 0x7ff74c19d0b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 --2- 172.21.15.45:0/2735011354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74c074d90 0x7ff74c19d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.171+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 >> 172.21.15.45:0/2735011354 conn(0x7ff74c0fc290 msgr2=0x7ff74c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.172+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 shutdown_connections 2023-10-17T01:19:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:19:31.172+0000 7ff7527ff700 1 -- 172.21.15.45:0/2735011354 wait complete. 2023-10-17T01:19:31.204 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:19:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.716+0000 7f21231ff700 1 -- 172.21.15.45:0/4245073606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c0ffc50 msgr2=0x7f211c100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.716+0000 7f21231ff700 1 --2- 172.21.15.45:0/4245073606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c0ffc50 0x7f211c100070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f210c009990 tx=0x7f210c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.717+0000 7f21231ff700 1 -- 172.21.15.45:0/4245073606 shutdown_connections 2023-10-17T01:19:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.717+0000 7f21231ff700 1 --2- 172.21.15.45:0/4245073606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c1017e0 0x7f211c108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.717+0000 7f21231ff700 1 --2- 172.21.15.45:0/4245073606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c0ffc50 0x7f211c100070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.717+0000 7f21231ff700 1 --2- 172.21.15.45:0/4245073606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c100e40 0x7f211c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.717+0000 7f21231ff700 1 -- 172.21.15.45:0/4245073606 >> 172.21.15.45:0/4245073606 conn(0x7f211c0fb1a0 msgr2=0x7f211c0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.717+0000 7f21231ff700 1 -- 172.21.15.45:0/4245073606 shutdown_connections 2023-10-17T01:19:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.718+0000 7f21231ff700 1 -- 172.21.15.45:0/4245073606 wait complete. 2023-10-17T01:19:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.719+0000 7f21231ff700 1 Processor -- start 2023-10-17T01:19:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.719+0000 7f21231ff700 1 -- start start 2023-10-17T01:19:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f21231ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0ffc50 0x7f211c1949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f21231ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 0x7f211c194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f21231ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c1017e0 0x7f211c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f21231ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1963f0 con 0x7f211c0ffc50 2023-10-17T01:19:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f21231ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c196560 con 0x7f211c1017e0 2023-10-17T01:19:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f21231ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c196720 con 0x7f211c100e40 2023-10-17T01:19:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.720+0000 7f211bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 0x7f211c194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 0x7f211c194f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44632/0 (socket says 172.21.15.45:44632) 2023-10-17T01:19:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 -- 172.21.15.45:0/4188805674 learned_addr learned my addr 172.21.15.45:0/4188805674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f2120f9b700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0ffc50 0x7f211c1949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 -- 172.21.15.45:0/4188805674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c1017e0 msgr2=0x7f211c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c1017e0 0x7f211c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 -- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0ffc50 msgr2=0x7f211c1949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0ffc50 0x7f211c1949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.721+0000 7f211bfff700 1 -- 172.21.15.45:0/4188805674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0095f0 con 0x7f211c100e40 2023-10-17T01:19:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.722+0000 7f211bfff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 0x7f211c194f10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f211000b530 tx=0x7f211000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.725+0000 7f2119ffb700 1 -- 172.21.15.45:0/4188805674 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110011830 con 0x7f211c100e40 2023-10-17T01:19:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.725+0000 7f2119ffb700 1 -- 172.21.15.45:0/4188805674 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2110011e50 con 0x7f211c100e40 2023-10-17T01:19:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.725+0000 7f2119ffb700 1 -- 172.21.15.45:0/4188805674 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110010e70 con 0x7f211c100e40 2023-10-17T01:19:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.725+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c196a00 con 0x7f211c100e40 2023-10-17T01:19:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.726+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c19d240 con 0x7f211c100e40 2023-10-17T01:19:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.736+0000 7f2119ffb700 1 -- 172.21.15.45:0/4188805674 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2110010510 con 0x7f211c100e40 2023-10-17T01:19:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.736+0000 7f2136a72b80 1 Processor -- start 2023-10-17T01:19:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.736+0000 7f2136a72b80 1 -- start start 2023-10-17T01:19:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.737+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.737+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1190 0x55844d5c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1d90 0x55844d5c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f2136a72b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5189b0 con 0x55844d5c1d90 2023-10-17T01:19:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f2136a72b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d516370 con 0x55844d5c1190 2023-10-17T01:19:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f2136a72b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d511c60 con 0x55844d5c0830 2023-10-17T01:19:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f212179c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f212179c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44648/0 (socket says 172.21.15.45:44648) 2023-10-17T01:19:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f212179c700 1 -- 172.21.15.45:0/716448958 learned_addr learned my addr 172.21.15.45:0/716448958 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.738+0000 7f2120f9b700 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1190 0x55844d5c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.739+0000 7f212179c700 1 -- 172.21.15.45:0/716448958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1190 msgr2=0x55844d5c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.739+0000 7f212179c700 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1190 0x55844d5c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.739+0000 7f212179c700 1 -- 172.21.15.45:0/716448958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1d90 msgr2=0x55844d5c2210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:19:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.739+0000 7f211bfff700 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1d90 0x55844d5c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.739+0000 7f212179c700 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1d90 0x55844d5c2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.739+0000 7f212179c700 1 -- 172.21.15.45:0/716448958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55844d33fd80 con 0x55844d5c0830 2023-10-17T01:19:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.740+0000 7f2119ffb700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f21040a4ff0 0x7f21040a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.740+0000 7f2119ffb700 1 -- 172.21.15.45:0/4188805674 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f21100cbb80 con 0x7f211c100e40 2023-10-17T01:19:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.740+0000 7f212179c700 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f211400e400 tx=0x7f211400e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375eb26576b753 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.740+0000 7f2120f9b700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f21040a4ff0 0x7f21040a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.741+0000 7f2102ffd700 1 -- 172.21.15.45:0/716448958 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114018070 con 0x55844d5c0830 2023-10-17T01:19:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.741+0000 7f2102ffd700 1 -- 172.21.15.45:0/716448958 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211400f040 con 0x55844d5c0830 2023-10-17T01:19:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.742+0000 7f2136a72b80 1 -- 172.21.15.45:0/716448958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 msgr2=0x55844d5c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.742+0000 7f2136a72b80 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f211400e400 tx=0x7f211400e710 comp rx=0 tx=0).stop 2023-10-17T01:19:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.742+0000 7f2120f9b700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f21040a4ff0 0x7f21040a74b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f210c006010 tx=0x7f210c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.742+0000 7f2136a72b80 1 -- 172.21.15.45:0/716448958 shutdown_connections 2023-10-17T01:19:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.742+0000 7f2136a72b80 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1d90 0x55844d5c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.743+0000 7f2136a72b80 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.743+0000 7f2136a72b80 1 --2- 172.21.15.45:0/716448958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1190 0x55844d5c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.743+0000 7f2136a72b80 1 -- 172.21.15.45:0/716448958 >> 172.21.15.45:0/716448958 conn(0x55844d520070 msgr2=0x55844d5bb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.743+0000 7f2136a72b80 1 -- 172.21.15.45:0/716448958 shutdown_connections 2023-10-17T01:19:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.743+0000 7f2136a72b80 1 -- 172.21.15.45:0/716448958 wait complete. 2023-10-17T01:19:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.744+0000 7f2136a72b80 1 -- start start 2023-10-17T01:19:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.744+0000 7f2136a72b80 1 Processor -- start 2023-10-17T01:19:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.745+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c0830 0x55844d5ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 0x55844d5ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f2136a72b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d4ae4b0 con 0x55844d5c0830 2023-10-17T01:19:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f211bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 0x55844d5ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f211bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 0x55844d5ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44652/0 (socket says 172.21.15.45:44652) 2023-10-17T01:19:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f211bfff700 1 -- 172.21.15.45:0/2555146067 learned_addr learned my addr 172.21.15.45:0/2555146067 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d4ae150 con 0x55844d5c1d90 2023-10-17T01:19:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d3958c0 con 0x55844d5c1190 2023-10-17T01:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.746+0000 7f2136a72b80 10 client.0 _ll_register_callbacks cb 0x55844d3fba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.747+0000 7f211bfff700 1 -- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 msgr2=0x55844d5d1810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.747+0000 7f211bfff700 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5d1810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.747+0000 7f211bfff700 1 -- 172.21.15.45:0/2555146067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c0830 msgr2=0x55844d5ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.747+0000 7f211bfff700 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c0830 0x55844d5ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.747+0000 7f211bfff700 1 -- 172.21.15.45:0/2555146067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211400e140 con 0x55844d5c1190 2023-10-17T01:19:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.749+0000 7f211bfff700 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 0x55844d5ca7d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f211000b500 tx=0x7f2110098b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdcf4b837193f4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.749+0000 7f20fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.750+0000 7f20fbfff700 1 -- 172.21.15.45:0/2555146067 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100d0040 con 0x55844d5c1190 2023-10-17T01:19:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.750+0000 7f20fbfff700 1 -- 172.21.15.45:0/2555146067 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2110099900 con 0x55844d5c1190 2023-10-17T01:19:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.750+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55844d53ccb0 con 0x55844d5c1190 2023-10-17T01:19:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.750+0000 7f20fbfff700 1 -- 172.21.15.45:0/2555146067 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2110099200 con 0x55844d5c1190 2023-10-17T01:19:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.750+0000 7f2136a72b80 10 client.25960 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.750+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55844d328c00 con 0x55844d5c1190 2023-10-17T01:19:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.751+0000 7f20fbfff700 1 -- 172.21.15.45:0/2555146067 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55844d53ccb0 con 0x55844d5c1190 2023-10-17T01:19:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f20fbfff700 1 -- 172.21.15.45:0/2555146067 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f21100a0de0 con 0x55844d5c1190 2023-10-17T01:19:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f2136a72b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f2136a72b80 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f2136a72b80 20 client.25960 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d5d6020 0x55844d5d8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f2136a72b80 4 client.25960 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.752+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55844d58fcd0 con 0x55844d5d6020 2023-10-17T01:19:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.753+0000 7f2120f9b700 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d5d6020 0x55844d5d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.754+0000 7f20fbfff700 10 client.25960 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.754+0000 7f2120f9b700 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d5d6020 0x55844d5d8440 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9f87127c3be3e9 server_cookie=61d6466aedfa86e2 in_seq=0 out_seq=0 2023-10-17T01:19:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f20fbfff700 1 -- 172.21.15.45:0/2555146067 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55844d58fcd0 con 0x55844d5d6020 2023-10-17T01:19:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f20fbfff700 10 client.25960 handle_command_reply: tid=0 2023-10-17T01:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 client.25960 shutdown 2023-10-17T01:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 msgr2=0x55844d5ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 0x55844d5ca7d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f211000b500 tx=0x7f2110098b60 comp rx=0 tx=0).stop 2023-10-17T01:19:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 shutdown_connections 2023-10-17T01:19:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d5d6020 0x55844d5d8440 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c0830 0x55844d5ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2555146067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c1190 0x55844d5ca7d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.755+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 >> 172.21.15.45:0/2555146067 conn(0x55844d520070 msgr2=0x55844d5baa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.756+0000 7f20fbfff700 0 client.25960 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.756+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 shutdown_connections 2023-10-17T01:19:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.756+0000 7f2136a72b80 1 -- 172.21.15.45:0/2555146067 wait complete. 2023-10-17T01:19:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.756+0000 7f2136a72b80 20 client.25960 trim_cache size 0 max 16384 2023-10-17T01:19:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 Processor -- start 2023-10-17T01:19:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 -- start start 2023-10-17T01:19:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5cb1d0 con 0x55844d5c1190 2023-10-17T01:19:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5c9770 con 0x55844d5c1d90 2023-10-17T01:19:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.785+0000 7f2136a72b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5c8c60 con 0x55844d5c0830 2023-10-17T01:19:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.786+0000 7f2120f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.786+0000 7f2120f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44654/0 (socket says 172.21.15.45:44654) 2023-10-17T01:19:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.786+0000 7f2120f9b700 1 -- 172.21.15.45:0/1502586401 learned_addr learned my addr 172.21.15.45:0/1502586401 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.786+0000 7f212179c700 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f2120f9b700 1 -- 172.21.15.45:0/1502586401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 msgr2=0x55844d5baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f2120f9b700 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5baa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f2120f9b700 1 -- 172.21.15.45:0/1502586401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 msgr2=0x55844d5c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f2120f9b700 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f2120f9b700 1 -- 172.21.15.45:0/1502586401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55844d3661f0 con 0x55844d5c0830 2023-10-17T01:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f2120f9b700 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f210c009990 tx=0x7f210c01a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ace3e33a973a8b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.787+0000 7f21037fe700 1 -- 172.21.15.45:0/1502586401 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c005040 con 0x55844d5c0830 2023-10-17T01:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.788+0000 7f21037fe700 1 -- 172.21.15.45:0/1502586401 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f210c01e040 con 0x55844d5c0830 2023-10-17T01:19:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.788+0000 7f21037fe700 1 -- 172.21.15.45:0/1502586401 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c005040 con 0x55844d5c0830 2023-10-17T01:19:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.788+0000 7f2136a72b80 1 -- 172.21.15.45:0/1502586401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 msgr2=0x55844d5c0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.788+0000 7f2136a72b80 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f210c009990 tx=0x7f210c01a050 comp rx=0 tx=0).stop 2023-10-17T01:19:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 -- 172.21.15.45:0/1502586401 shutdown_connections 2023-10-17T01:19:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5c0c10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 --2- 172.21.15.45:0/1502586401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 -- 172.21.15.45:0/1502586401 >> 172.21.15.45:0/1502586401 conn(0x55844d520070 msgr2=0x55844d5bb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 -- 172.21.15.45:0/1502586401 shutdown_connections 2023-10-17T01:19:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 -- 172.21.15.45:0/1502586401 wait complete. 2023-10-17T01:19:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.789+0000 7f2136a72b80 1 -- start start 2023-10-17T01:19:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 Processor -- start 2023-10-17T01:19:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5cdd70 con 0x55844d5c1190 2023-10-17T01:19:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5cdee0 con 0x55844d5c1d90 2023-10-17T01:19:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844d5ce0a0 con 0x55844d5c0830 2023-10-17T01:19:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.790+0000 7f2136a72b80 10 client.0 _ll_register_callbacks cb 0x7f2110095630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f2120f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f2120f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44664/0 (socket says 172.21.15.45:44664) 2023-10-17T01:19:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40392/0 (socket says 172.21.15.45:40392) 2023-10-17T01:19:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f2120f9b700 1 -- 172.21.15.45:0/2881540501 learned_addr learned my addr 172.21.15.45:0/2881540501 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f211bfff700 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 -- 172.21.15.45:0/2881540501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 msgr2=0x55844d5cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 -- 172.21.15.45:0/2881540501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 msgr2=0x55844d5cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 -- 172.21.15.45:0/2881540501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1089e0 con 0x55844d5c1d90 2023-10-17T01:19:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f212179c700 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5cd6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2114009ba0 tx=0x7f2114009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6165247ce920ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.791+0000 7f20fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:19:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.793+0000 7f20fbfff700 1 -- 172.21.15.45:0/2881540501 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114018040 con 0x55844d5c1d90 2023-10-17T01:19:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.793+0000 7f20fbfff700 1 -- 172.21.15.45:0/2881540501 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f21140095e0 con 0x55844d5c1d90 2023-10-17T01:19:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.793+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f211400f040 con 0x55844d5c1d90 2023-10-17T01:19:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.794+0000 7f20fbfff700 1 -- 172.21.15.45:0/2881540501 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2114003e10 con 0x55844d5c1d90 2023-10-17T01:19:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.794+0000 7f2136a72b80 10 client.25997 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.794+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55844d33e250 con 0x55844d5c1d90 2023-10-17T01:19:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.795+0000 7f20fbfff700 1 -- 172.21.15.45:0/2881540501 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211400f040 con 0x55844d5c1d90 2023-10-17T01:19:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.795+0000 7f20fbfff700 1 -- 172.21.15.45:0/2881540501 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f2114014a10 con 0x55844d5c1d90 2023-10-17T01:19:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2136a72b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2136a72b80 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2136a72b80 20 client.25997 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d628d60 0x55844d62b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2136a72b80 4 client.25997 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:19:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55844d50ace0 con 0x55844d628d60 2023-10-17T01:19:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.796+0000 7f2120f9b700 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d628d60 0x55844d62b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.797+0000 7f20fbfff700 10 client.25997 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.797+0000 7f2120f9b700 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d628d60 0x55844d62b180 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1bc47e7393c0984 server_cookie=a7cb07a8659e2983 in_seq=0 out_seq=0 2023-10-17T01:19:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.810+0000 7f20fbfff700 1 -- 172.21.15.45:0/2881540501 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55844d50ace0 con 0x55844d628d60 2023-10-17T01:19:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.810+0000 7f20fbfff700 10 client.25997 handle_command_reply: tid=0 2023-10-17T01:19:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.810+0000 7f2136a72b80 1 client.25997 shutdown 2023-10-17T01:19:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.810+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 msgr2=0x55844d5cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.810+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5cd6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2114009ba0 tx=0x7f2114009300 comp rx=0 tx=0).stop 2023-10-17T01:19:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 shutdown_connections 2023-10-17T01:19:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55844d628d60 0x55844d62b180 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55844d5c1d90 0x55844d5cd6d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f20fbfff700 0 client.25997 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55844d5c0830 0x55844d5cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 --2- 172.21.15.45:0/2881540501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55844d5c1190 0x55844d5cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 >> 172.21.15.45:0/2881540501 conn(0x55844d520070 msgr2=0x55844d5c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 shutdown_connections 2023-10-17T01:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 1 -- 172.21.15.45:0/2881540501 wait complete. 2023-10-17T01:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.811+0000 7f2136a72b80 20 client.25997 trim_cache size 0 max 16384 2023-10-17T01:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:19:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:19:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:19:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.812+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f21040a4ff0 msgr2=0x7f21040a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.812+0000 7f21231ff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f21040a4ff0 0x7f21040a74b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f210c006010 tx=0x7f210c00b260 comp rx=0 tx=0).stop 2023-10-17T01:19:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.812+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 msgr2=0x7f211c194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.812+0000 7f21231ff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 0x7f211c194f10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f211000b530 tx=0x7f211000b8f0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.812+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 shutdown_connections 2023-10-17T01:19:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.813+0000 7f21231ff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f21040a4ff0 0x7f21040a74b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.813+0000 7f21231ff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c1017e0 0x7f211c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.813+0000 7f21231ff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0ffc50 0x7f211c1949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.813+0000 7f21231ff700 1 --2- 172.21.15.45:0/4188805674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f211c100e40 0x7f211c194f10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.813+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 >> 172.21.15.45:0/4188805674 conn(0x7f211c0fb1a0 msgr2=0x7f211c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.814+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 shutdown_connections 2023-10-17T01:19:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:31.814+0000 7f21231ff700 1 -- 172.21.15.45:0/4188805674 wait complete. 2023-10-17T01:19:31.911 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:31 smithi181 ceph-mon[134520]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 860 op/s 2023-10-17T01:19:32.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:31 smithi045 ceph-mon[98474]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 860 op/s 2023-10-17T01:19:32.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:31 smithi060 ceph-mon[134136]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 860 op/s 2023-10-17T01:19:32.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:32 smithi181 ceph-mon[134520]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 802 op/s 2023-10-17T01:19:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:32 smithi045 ceph-mon[98474]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 802 op/s 2023-10-17T01:19:33.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:32 smithi060 ceph-mon[134136]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 802 op/s 2023-10-17T01:19:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 265.0 s, 1661.0 tps, lat 60.055 ms stddev 68.203 2023-10-17T01:19:35.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:35 smithi181 ceph-mon[134520]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 886 op/s 2023-10-17T01:19:36.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:35 smithi045 ceph-mon[98474]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 886 op/s 2023-10-17T01:19:36.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:35 smithi060 ceph-mon[134136]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 886 op/s 2023-10-17T01:19:37.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:37 smithi181 ceph-mon[134520]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 842 op/s 2023-10-17T01:19:38.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:37 smithi045 ceph-mon[98474]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 842 op/s 2023-10-17T01:19:38.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:37 smithi060 ceph-mon[134136]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 842 op/s 2023-10-17T01:19:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:39 smithi181 ceph-mon[134520]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 834 op/s 2023-10-17T01:19:40.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:39 smithi045 ceph-mon[98474]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 834 op/s 2023-10-17T01:19:40.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:39 smithi060 ceph-mon[134136]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 834 op/s 2023-10-17T01:19:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 270.0 s, 1764.4 tps, lat 56.998 ms stddev 70.598 2023-10-17T01:19:41.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:40 smithi045 ceph-mon[98474]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:19:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:40 smithi060 ceph-mon[134136]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:19:41.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:40 smithi181 ceph-mon[134520]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-10-17T01:19:43.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:42 smithi045 ceph-mon[98474]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 1.15k op/s 2023-10-17T01:19:43.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:42 smithi060 ceph-mon[134136]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 1.15k op/s 2023-10-17T01:19:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:42 smithi181 ceph-mon[134520]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 1.15k op/s 2023-10-17T01:19:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 -- 172.21.15.45:0/3357425983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 msgr2=0x7f36d41011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 --2- 172.21.15.45:0/3357425983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d41011b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009990 tx=0x7f36c8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 -- 172.21.15.45:0/3357425983 shutdown_connections 2023-10-17T01:19:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 --2- 172.21.15.45:0/3357425983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4102920 0x7f36d41091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 --2- 172.21.15.45:0/3357425983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d41011b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 --2- 172.21.15.45:0/3357425983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4101f80 0x7f36d41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.056+0000 7f36dafff700 1 -- 172.21.15.45:0/3357425983 >> 172.21.15.45:0/3357425983 conn(0x7f36d40fc300 msgr2=0x7f36d40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.057+0000 7f36dafff700 1 -- 172.21.15.45:0/3357425983 shutdown_connections 2023-10-17T01:19:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.057+0000 7f36dafff700 1 -- 172.21.15.45:0/3357425983 wait complete. 2023-10-17T01:19:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 Processor -- start 2023-10-17T01:19:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 -- start start 2023-10-17T01:19:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d4073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4101f80 0x7f36d4073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 0x7f36d4073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4074ce0 con 0x7f36d4101f80 2023-10-17T01:19:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4074e50 con 0x7f36d4102920 2023-10-17T01:19:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.059+0000 7f36dafff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4075010 con 0x7f36d4100d90 2023-10-17T01:19:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 0x7f36d4073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 0x7f36d4073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42094/0 (socket says 172.21.15.45:42094) 2023-10-17T01:19:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d959c700 1 -- 172.21.15.45:0/2928899660 learned_addr learned my addr 172.21.15.45:0/2928899660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d4073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d3fff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4101f80 0x7f36d4073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d8d9b700 1 -- 172.21.15.45:0/2928899660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 msgr2=0x7f36d4073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 0x7f36d4073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d8d9b700 1 -- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4101f80 msgr2=0x7f36d4073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4101f80 0x7f36d4073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d8d9b700 1 -- 172.21.15.45:0/2928899660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80095f0 con 0x7f36d4100d90 2023-10-17T01:19:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.060+0000 7f36d959c700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 0x7f36d4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:19:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.061+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d4073140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f36c80098e0 tx=0x7f36c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.061+0000 7f36d1ffb700 1 -- 172.21.15.45:0/2928899660 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c801d070 con 0x7f36d4100d90 2023-10-17T01:19:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.061+0000 7f36d1ffb700 1 -- 172.21.15.45:0/2928899660 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36c80054e0 con 0x7f36d4100d90 2023-10-17T01:19:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.061+0000 7f36d1ffb700 1 -- 172.21.15.45:0/2928899660 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8004bd0 con 0x7f36d4100d90 2023-10-17T01:19:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.061+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d419ac00 con 0x7f36d4100d90 2023-10-17T01:19:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.062+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d419b060 con 0x7f36d4100d90 2023-10-17T01:19:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.065+0000 7f36d1ffb700 1 -- 172.21.15.45:0/2928899660 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f36c8004d50 con 0x7f36d4100d90 2023-10-17T01:19:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.067+0000 7f36d1ffb700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36c40a0c70 0x7f36c40a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.068+0000 7f36d1ffb700 1 -- 172.21.15.45:0/2928899660 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f36c80cd2d0 con 0x7f36d4100d90 2023-10-17T01:19:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.068+0000 7f36d3fff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36c40a0c70 0x7f36c40a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.069+0000 7f36ee8c6b80 1 Processor -- start 2023-10-17T01:19:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 -- start start 2023-10-17T01:19:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea734dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734e190 0x560ea734e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 0x560ea734f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea72a59b0 con 0x560ea734d830 2023-10-17T01:19:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea72a3370 con 0x560ea734e190 2023-10-17T01:19:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea729ec60 con 0x560ea734ed90 2023-10-17T01:19:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36d959c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 0x560ea734f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.070+0000 7f36d8d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea734dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 0x560ea734f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40478/0 (socket says 172.21.15.45:40478) 2023-10-17T01:19:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 -- 172.21.15.45:0/2975277913 learned_addr learned my addr 172.21.15.45:0/2975277913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 -- 172.21.15.45:0/2975277913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734e190 msgr2=0x560ea734e790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:19:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734e190 0x560ea734e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 -- 172.21.15.45:0/2975277913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 msgr2=0x560ea734dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea734dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d959c700 1 -- 172.21.15.45:0/2975277913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ea70ccd80 con 0x560ea734ed90 2023-10-17T01:19:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.071+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea734dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:19:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.072+0000 7f36d959c700 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 0x560ea734f210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007850 tx=0x7f36cc009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b97f25b3af13ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.072+0000 7f36d3fff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36c40a0c70 0x7f36c40a3130 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36d4074440 tx=0x7f36c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.072+0000 7f36beffd700 1 -- 172.21.15.45:0/2975277913 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc003a00 con 0x560ea734ed90 2023-10-17T01:19:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.072+0000 7f36beffd700 1 -- 172.21.15.45:0/2975277913 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc00f040 con 0x560ea734ed90 2023-10-17T01:19:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.072+0000 7f36beffd700 1 -- 172.21.15.45:0/2975277913 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004280 con 0x560ea734ed90 2023-10-17T01:19:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/2975277913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 msgr2=0x560ea734f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 0x560ea734f210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007850 tx=0x7f36cc009230 comp rx=0 tx=0).stop 2023-10-17T01:19:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/2975277913 shutdown_connections 2023-10-17T01:19:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734ed90 0x560ea734f210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea734dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/2975277913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734e190 0x560ea734e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/2975277913 >> 172.21.15.45:0/2975277913 conn(0x560ea72ad070 msgr2=0x560ea73482c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/2975277913 shutdown_connections 2023-10-17T01:19:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.073+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/2975277913 wait complete. 2023-10-17T01:19:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.074+0000 7f36ee8c6b80 1 -- start start 2023-10-17T01:19:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.074+0000 7f36ee8c6b80 1 Processor -- start 2023-10-17T01:19:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.075+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734d830 0x560ea7357390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.075+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea73578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.075+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734ed90 0x560ea735e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.075+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea723b4b0 con 0x560ea734ed90 2023-10-17T01:19:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.075+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea723b150 con 0x560ea734d830 2023-10-17T01:19:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.075+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea71228c0 con 0x560ea734e190 2023-10-17T01:19:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.076+0000 7f36ee8c6b80 10 client.0 _ll_register_callbacks cb 0x560ea7188a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.076+0000 7f36d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea73578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.076+0000 7f36d8d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734d830 0x560ea7357390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.076+0000 7f36d8d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734d830 0x560ea7357390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42108/0 (socket says 172.21.15.45:42108) 2023-10-17T01:19:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.076+0000 7f36d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea73578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40488/0 (socket says 172.21.15.45:40488) 2023-10-17T01:19:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.076+0000 7f36d8d9b700 1 -- 172.21.15.45:0/3828735252 learned_addr learned my addr 172.21.15.45:0/3828735252 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d959c700 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734ed90 0x560ea735e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d3fff700 1 -- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734d830 msgr2=0x560ea7357390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d3fff700 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734d830 0x560ea7357390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d3fff700 1 -- 172.21.15.45:0/3828735252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734ed90 msgr2=0x560ea735e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d3fff700 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734ed90 0x560ea735e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d3fff700 1 -- 172.21.15.45:0/3828735252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc007500 con 0x560ea734e190 2023-10-17T01:19:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.077+0000 7f36d3fff700 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea73578d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x560ea734d5d0 tx=0x7f36c00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e563bb22579a3b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36b3fff700 1 -- 172.21.15.45:0/3828735252 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004830 con 0x560ea734e190 2023-10-17T01:19:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36b3fff700 1 -- 172.21.15.45:0/3828735252 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f36c0004990 con 0x560ea734e190 2023-10-17T01:19:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ea72c9cb0 con 0x560ea734e190 2023-10-17T01:19:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36b3fff700 1 -- 172.21.15.45:0/3828735252 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c0004b50 con 0x560ea734e190 2023-10-17T01:19:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36ee8c6b80 10 client.25978 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.078+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560ea70b5c00 con 0x560ea734e190 2023-10-17T01:19:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.079+0000 7f36b3fff700 1 -- 172.21.15.45:0/3828735252 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ea72c9cb0 con 0x560ea734e190 2023-10-17T01:19:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.079+0000 7f36b3fff700 1 -- 172.21.15.45:0/3828735252 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f36c0024030 con 0x560ea734e190 2023-10-17T01:19:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.079+0000 7f36ee8c6b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.079+0000 7f36ee8c6b80 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36ee8c6b80 20 client.25978 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73631c0 0x560ea73655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36ee8c6b80 4 client.25978 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ea731ccd0 con 0x560ea73631c0 2023-10-17T01:19:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73631c0 0x560ea73655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73631c0 0x560ea73655e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d257e93bcdd63 server_cookie=baffbd6f6bdf9dbc in_seq=0 out_seq=0 2023-10-17T01:19:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.080+0000 7f36b3fff700 10 client.25978 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.083+0000 7f36b3fff700 1 -- 172.21.15.45:0/3828735252 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x560ea731ccd0 con 0x560ea73631c0 2023-10-17T01:19:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.083+0000 7f36b3fff700 10 client.25978 handle_command_reply: tid=0 2023-10-17T01:19:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.084+0000 7f36ee8c6b80 1 client.25978 shutdown 2023-10-17T01:19:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.086+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 msgr2=0x560ea73578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea73578d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x560ea734d5d0 tx=0x7f36c00047c0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 shutdown_connections 2023-10-17T01:19:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734ed90 0x560ea735e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73631c0 0x560ea73655e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734d830 0x560ea7357390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/3828735252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea73578d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36b3fff700 0 client.25978 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.087+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 >> 172.21.15.45:0/3828735252 conn(0x560ea72ad070 msgr2=0x560ea7347a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.088+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 shutdown_connections 2023-10-17T01:19:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.088+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/3828735252 wait complete. 2023-10-17T01:19:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.088+0000 7f36ee8c6b80 20 client.25978 trim_cache size 0 max 16384 2023-10-17T01:19:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.115+0000 7f36ee8c6b80 1 Processor -- start 2023-10-17T01:19:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 -- start start 2023-10-17T01:19:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7356a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7355260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea7357390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea731ccd0 con 0x560ea734d830 2023-10-17T01:19:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea72c9cb0 con 0x560ea734ed90 2023-10-17T01:19:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea73429e0 con 0x560ea734e190 2023-10-17T01:19:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36d959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea7357390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36d959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea7357390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42124/0 (socket says 172.21.15.45:42124) 2023-10-17T01:19:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36d959c700 1 -- 172.21.15.45:0/854063874 learned_addr learned my addr 172.21.15.45:0/854063874 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.116+0000 7f36d3fff700 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7355260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36d959c700 1 -- 172.21.15.45:0/854063874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 msgr2=0x560ea7355260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36d959c700 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7355260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36d959c700 1 -- 172.21.15.45:0/854063874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 msgr2=0x560ea7356a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36d959c700 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7356a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36d959c700 1 -- 172.21.15.45:0/854063874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ea70f31f0 con 0x560ea734ed90 2023-10-17T01:19:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36d959c700 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea7357390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007850 tx=0x7f36cc004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374a672c0e4e12bb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.117+0000 7f36bf7fe700 1 -- 172.21.15.45:0/854063874 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004c30 con 0x560ea734ed90 2023-10-17T01:19:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.118+0000 7f36bf7fe700 1 -- 172.21.15.45:0/854063874 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc004d90 con 0x560ea734ed90 2023-10-17T01:19:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.118+0000 7f36bf7fe700 1 -- 172.21.15.45:0/854063874 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc005530 con 0x560ea734ed90 2023-10-17T01:19:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.118+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/854063874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 msgr2=0x560ea7357390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.118+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea7357390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007850 tx=0x7f36cc004a90 comp rx=0 tx=0).stop 2023-10-17T01:19:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.119+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/854063874 shutdown_connections 2023-10-17T01:19:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.119+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea7357390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.119+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7356a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.119+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/854063874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7355260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.119+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/854063874 >> 172.21.15.45:0/854063874 conn(0x560ea72ad070 msgr2=0x560ea7356060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.120+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/854063874 shutdown_connections 2023-10-17T01:19:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.120+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/854063874 wait complete. 2023-10-17T01:19:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.120+0000 7f36ee8c6b80 1 -- start start 2023-10-17T01:19:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 Processor -- start 2023-10-17T01:19:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7358a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7358fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea735bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea735c210 con 0x560ea734d830 2023-10-17T01:19:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea735c380 con 0x560ea734ed90 2023-10-17T01:19:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ea735c540 con 0x560ea734e190 2023-10-17T01:19:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.121+0000 7f36ee8c6b80 10 client.0 _ll_register_callbacks cb 0x7f36c00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.122+0000 7f36d959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea735bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.122+0000 7f36d959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea735bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42134/0 (socket says 172.21.15.45:42134) 2023-10-17T01:19:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.122+0000 7f36d959c700 1 -- 172.21.15.45:0/4103587112 learned_addr learned my addr 172.21.15.45:0/4103587112 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.123+0000 7f36d3fff700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7358fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.123+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7358a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.124+0000 7f36d3fff700 1 -- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 msgr2=0x560ea735bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.124+0000 7f36d3fff700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea735bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.124+0000 7f36d3fff700 1 -- 172.21.15.45:0/4103587112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 msgr2=0x560ea7358a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.124+0000 7f36d3fff700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7358a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.124+0000 7f36d3fff700 1 -- 172.21.15.45:0/4103587112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ea70ccd80 con 0x560ea734e190 2023-10-17T01:19:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.124+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7358a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:19:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.125+0000 7f36d3fff700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7358fb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x560ea7345160 tx=0x7f36c0011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a70f38bd1481bc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.125+0000 7f36b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.125+0000 7f36b3fff700 1 -- 172.21.15.45:0/4103587112 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c001e9a0 con 0x560ea734e190 2023-10-17T01:19:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.125+0000 7f36b3fff700 1 -- 172.21.15.45:0/4103587112 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f36c001eb00 con 0x560ea734e190 2023-10-17T01:19:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.125+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36cc00f040 con 0x560ea734e190 2023-10-17T01:19:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36b3fff700 1 -- 172.21.15.45:0/4103587112 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36c001ecc0 con 0x560ea734e190 2023-10-17T01:19:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36ee8c6b80 10 client.25984 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560ea70cb250 con 0x560ea734e190 2023-10-17T01:19:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36b3fff700 1 -- 172.21.15.45:0/4103587112 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc00f040 con 0x560ea734e190 2023-10-17T01:19:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36b3fff700 1 -- 172.21.15.45:0/4103587112 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f36c0024070 con 0x560ea734e190 2023-10-17T01:19:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36ee8c6b80 10 client.25984 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36ee8c6b80 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36ee8c6b80 20 client.25984 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.126+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73c6c70 0x560ea73c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.127+0000 7f36ee8c6b80 4 client.25984 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:19:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.127+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ea7297ce0 con 0x560ea73c6c70 2023-10-17T01:19:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.127+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73c6c70 0x560ea73c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.128+0000 7f36b3fff700 10 client.25984 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.128+0000 7f36d8d9b700 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73c6c70 0x560ea73c9090 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879e7f884f15cbf7 server_cookie=e0edd51db8d16be2 in_seq=0 out_seq=0 2023-10-17T01:19:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.142+0000 7f36b3fff700 1 -- 172.21.15.45:0/4103587112 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ea7297ce0 con 0x560ea73c6c70 2023-10-17T01:19:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.142+0000 7f36b3fff700 10 client.25984 handle_command_reply: tid=0 2023-10-17T01:19:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.142+0000 7f36ee8c6b80 1 client.25984 shutdown 2023-10-17T01:19:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 msgr2=0x560ea7358fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7358fb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x560ea7345160 tx=0x7f36c0011b60 comp rx=0 tx=0).stop 2023-10-17T01:19:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 shutdown_connections 2023-10-17T01:19:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ea73c6c70 0x560ea73c9090 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ea734ed90 0x560ea735bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36b3fff700 0 client.25984 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ea734d830 0x560ea7358a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 --2- 172.21.15.45:0/4103587112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ea734e190 0x560ea7358fb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 >> 172.21.15.45:0/4103587112 conn(0x560ea72ad070 msgr2=0x560ea7355260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 shutdown_connections 2023-10-17T01:19:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 1 -- 172.21.15.45:0/4103587112 wait complete. 2023-10-17T01:19:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.143+0000 7f36ee8c6b80 20 client.25984 trim_cache size 0 max 16384 2023-10-17T01:19:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:19:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:19:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:19:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:19:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36c40a0c70 msgr2=0x7f36c40a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36c40a0c70 0x7f36c40a3130 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36d4074440 tx=0x7f36c0009260 comp rx=0 tx=0).stop 2023-10-17T01:19:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 msgr2=0x7f36d4073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d4073140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f36c80098e0 tx=0x7f36c8004730 comp rx=0 tx=0).stop 2023-10-17T01:19:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 shutdown_connections 2023-10-17T01:19:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f36c40a0c70 0x7f36c40a3130 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36d4102920 0x7f36d4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d4100d90 0x7f36d4073140 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 --2- 172.21.15.45:0/2928899660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36d4101f80 0x7f36d4073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 >> 172.21.15.45:0/2928899660 conn(0x7f36d40fc300 msgr2=0x7f36d40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 shutdown_connections 2023-10-17T01:19:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:45.144+0000 7f36dafff700 1 -- 172.21.15.45:0/2928899660 wait complete. 2023-10-17T01:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 275.0 s, 1669.4 tps, lat 60.365 ms stddev 105.765 2023-10-17T01:19:45.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:45 smithi181 ceph-mon[134520]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 1.85k op/s 2023-10-17T01:19:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:45 smithi045 ceph-mon[98474]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 1.85k op/s 2023-10-17T01:19:46.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:45 smithi060 ceph-mon[134136]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 1.85k op/s 2023-10-17T01:19:47.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:47 smithi181 ceph-mon[134520]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 2.02k op/s 2023-10-17T01:19:48.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:47 smithi045 ceph-mon[98474]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 2.02k op/s 2023-10-17T01:19:48.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:47 smithi060 ceph-mon[134136]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 2.02k op/s 2023-10-17T01:19:49.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:49 smithi181 ceph-mon[134520]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.96k op/s 2023-10-17T01:19:50.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:49 smithi045 ceph-mon[98474]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.96k op/s 2023-10-17T01:19:50.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:49 smithi060 ceph-mon[134136]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.96k op/s 2023-10-17T01:19:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 280.0 s, 2192.0 tps, lat 45.452 ms stddev 51.707 2023-10-17T01:19:50.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:50 smithi181 ceph-mon[134520]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 MiB/s wr, 2.10k op/s 2023-10-17T01:19:51.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:50 smithi045 ceph-mon[98474]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 MiB/s wr, 2.10k op/s 2023-10-17T01:19:51.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:50 smithi060 ceph-mon[134136]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 MiB/s wr, 2.10k op/s 2023-10-17T01:19:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:52 smithi045 ceph-mon[98474]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-10-17T01:19:53.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:52 smithi060 ceph-mon[134136]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-10-17T01:19:53.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:52 smithi181 ceph-mon[134520]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 1.95k op/s 2023-10-17T01:19:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 285.0 s, 2391.8 tps, lat 41.678 ms stddev 54.595 2023-10-17T01:19:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:55 smithi181 ceph-mon[134520]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.75k op/s 2023-10-17T01:19:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:55 smithi045 ceph-mon[98474]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.75k op/s 2023-10-17T01:19:56.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:55 smithi060 ceph-mon[134136]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.75k op/s 2023-10-17T01:19:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:56 smithi045 ceph-mon[98474]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 974 op/s 2023-10-17T01:19:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:56 smithi060 ceph-mon[134136]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 974 op/s 2023-10-17T01:19:57.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:56 smithi181 ceph-mon[134520]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 974 op/s 2023-10-17T01:19:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.370+0000 7f07af3ff700 1 -- 172.21.15.45:0/2598215719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a81041a0 msgr2=0x7f07a8104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.370+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2598215719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a81041a0 0x7f07a8104600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0798009990 tx=0x7f0798009ca0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.371+0000 7f07af3ff700 1 -- 172.21.15.45:0/2598215719 shutdown_connections 2023-10-17T01:19:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.371+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2598215719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a8104b40 0x7f07a810b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.371+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2598215719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a8102fb0 0x7f07a81033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.371+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2598215719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a81041a0 0x7f07a8104600 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.371+0000 7f07af3ff700 1 -- 172.21.15.45:0/2598215719 >> 172.21.15.45:0/2598215719 conn(0x7f07a80fe540 msgr2=0x7f07a8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.372+0000 7f07af3ff700 1 -- 172.21.15.45:0/2598215719 shutdown_connections 2023-10-17T01:19:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.372+0000 7f07af3ff700 1 -- 172.21.15.45:0/2598215719 wait complete. 2023-10-17T01:19:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.373+0000 7f07af3ff700 1 Processor -- start 2023-10-17T01:19:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.373+0000 7f07af3ff700 1 -- start start 2023-10-17T01:19:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.373+0000 7f07af3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 0x7f07a8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.373+0000 7f07af3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a81041a0 0x7f07a81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07af3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a8104b40 0x7f07a819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07af3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819a790 con 0x7f07a8104b40 2023-10-17T01:19:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07af3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819a900 con 0x7f07a81041a0 2023-10-17T01:19:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07af3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819aac0 con 0x7f07a8102fb0 2023-10-17T01:19:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ac99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a81041a0 0x7f07a81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a8104b40 0x7f07a819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 0x7f07a8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 0x7f07a8198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40658/0 (socket says 172.21.15.45:40658) 2023-10-17T01:19:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ac99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a81041a0 0x7f07a81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41644/0 (socket says 172.21.15.45:41644) 2023-10-17T01:19:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 -- 172.21.15.45:0/2620743842 learned_addr learned my addr 172.21.15.45:0/2620743842 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 -- 172.21.15.45:0/2620743842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a81041a0 msgr2=0x7f07a81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a81041a0 0x7f07a81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 -- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a8104b40 msgr2=0x7f07a819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a8104b40 0x7f07a819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.374+0000 7f07ad19b700 1 -- 172.21.15.45:0/2620743842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980095f0 con 0x7f07a8102fb0 2023-10-17T01:19:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.375+0000 7f07ad19b700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 0x7f07a8198da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07a400efc0 tx=0x7f07a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.376+0000 7f079e7fc700 1 -- 172.21.15.45:0/2620743842 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4018070 con 0x7f07a8102fb0 2023-10-17T01:19:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.376+0000 7f079e7fc700 1 -- 172.21.15.45:0/2620743842 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a400f040 con 0x7f07a8102fb0 2023-10-17T01:19:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.376+0000 7f079e7fc700 1 -- 172.21.15.45:0/2620743842 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4004280 con 0x7f07a8102fb0 2023-10-17T01:19:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.376+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a819ad40 con 0x7f07a8102fb0 2023-10-17T01:19:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.376+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a81a14b0 con 0x7f07a8102fb0 2023-10-17T01:19:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.381+0000 7f079e7fc700 1 -- 172.21.15.45:0/2620743842 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f07a4003a00 con 0x7f07a8102fb0 2023-10-17T01:19:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.383+0000 7f07c2ccab80 1 Processor -- start 2023-10-17T01:19:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.383+0000 7f07c2ccab80 1 -- start start 2023-10-17T01:19:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.383+0000 7f079e7fc700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07940a0cf0 0x7f07940a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f079e7fc700 1 -- 172.21.15.45:0/2620743842 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07a40cbf80 con 0x7f07a8102fb0 2023-10-17T01:19:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6311b0 0x55ff7b6315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b62fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6303e0 0x55ff7b630870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b57f9b0 con 0x55ff7b6311b0 2023-10-17T01:19:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b57d370 con 0x55ff7b6303e0 2023-10-17T01:19:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b578c60 con 0x55ff7b631b10 2023-10-17T01:19:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07940a0cf0 0x7f07940a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6303e0 0x55ff7b630870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6303e0 0x55ff7b630870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41650/0 (socket says 172.21.15.45:41650) 2023-10-17T01:19:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07ad99c700 1 -- 172.21.15.45:0/4267654535 learned_addr learned my addr 172.21.15.45:0/4267654535 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07ac99a700 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b62fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.384+0000 7f07ad19b700 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6311b0 0x55ff7b6315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ac99a700 1 -- 172.21.15.45:0/4267654535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6303e0 msgr2=0x55ff7b630870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ac99a700 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6303e0 0x55ff7b630870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ac99a700 1 -- 172.21.15.45:0/4267654535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6311b0 msgr2=0x55ff7b6315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ac99a700 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6311b0 0x55ff7b6315d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ac99a700 1 -- 172.21.15.45:0/4267654535 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7b3a6d80 con 0x55ff7b631b10 2023-10-17T01:19:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07940a0cf0 0x7f07940a31b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f07a00139e0 tx=0x7f07a0009c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07ac99a700 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b62fe00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0798000c00 tx=0x7f0798005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b079b8ddcccbcc62 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07937fe700 1 -- 172.21.15.45:0/4267654535 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079801d070 con 0x55ff7b631b10 2023-10-17T01:19:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07937fe700 1 -- 172.21.15.45:0/4267654535 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079801e040 con 0x55ff7b631b10 2023-10-17T01:19:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.385+0000 7f07937fe700 1 -- 172.21.15.45:0/4267654535 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798004900 con 0x55ff7b631b10 2023-10-17T01:19:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.386+0000 7f07c2ccab80 1 -- 172.21.15.45:0/4267654535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 msgr2=0x55ff7b62fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.386+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b62fe00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0798000c00 tx=0x7f0798005ea0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 -- 172.21.15.45:0/4267654535 shutdown_connections 2023-10-17T01:19:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6303e0 0x55ff7b630870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6311b0 0x55ff7b6315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/4267654535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b62fe00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 -- 172.21.15.45:0/4267654535 >> 172.21.15.45:0/4267654535 conn(0x55ff7b587070 msgr2=0x55ff7b6222d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 -- 172.21.15.45:0/4267654535 shutdown_connections 2023-10-17T01:19:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.387+0000 7f07c2ccab80 1 -- 172.21.15.45:0/4267654535 wait complete. 2023-10-17T01:19:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.388+0000 7f07c2ccab80 1 -- start start 2023-10-17T01:19:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.388+0000 7f07c2ccab80 1 Processor -- start 2023-10-17T01:19:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b639a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b6311b0 0x55ff7b639fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 0x55ff7b640e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b5154b0 con 0x55ff7b6303e0 2023-10-17T01:19:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b515150 con 0x55ff7b631b10 2023-10-17T01:19:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b3fc8c0 con 0x55ff7b6311b0 2023-10-17T01:19:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07c2ccab80 10 client.0 _ll_register_callbacks cb 0x55ff7b462a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.389+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 0x55ff7b640e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 0x55ff7b640e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41666/0 (socket says 172.21.15.45:41666) 2023-10-17T01:19:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 -- 172.21.15.45:0/3964436392 learned_addr learned my addr 172.21.15.45:0/3964436392 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ac99a700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b6311b0 0x55ff7b639fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad19b700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b639a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 -- 172.21.15.45:0/3964436392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b6311b0 msgr2=0x55ff7b639fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b6311b0 0x55ff7b639fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 -- 172.21.15.45:0/3964436392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 msgr2=0x55ff7b639a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b639a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 -- 172.21.15.45:0/3964436392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a810bd40 con 0x55ff7b631b10 2023-10-17T01:19:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad19b700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b639a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:19:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f07ad99c700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 0x55ff7b640e00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07a0004290 tx=0x7f07a001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6328681bd6f5d867 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f0790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:19:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.390+0000 7f0790ff9700 1 -- 172.21.15.45:0/3964436392 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0013170 con 0x55ff7b631b10 2023-10-17T01:19:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.391+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff7b5a3cb0 con 0x55ff7b631b10 2023-10-17T01:19:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.391+0000 7f0790ff9700 1 -- 172.21.15.45:0/3964436392 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07a0025030 con 0x55ff7b631b10 2023-10-17T01:19:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.391+0000 7f0790ff9700 1 -- 172.21.15.45:0/3964436392 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a00132d0 con 0x55ff7b631b10 2023-10-17T01:19:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.391+0000 7f07c2ccab80 10 client.26024 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.391+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ff7b38fc00 con 0x55ff7b631b10 2023-10-17T01:19:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.391+0000 7f0790ff9700 1 -- 172.21.15.45:0/3964436392 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff7b5a3cb0 con 0x55ff7b631b10 2023-10-17T01:19:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.392+0000 7f0790ff9700 1 -- 172.21.15.45:0/3964436392 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f07a0010720 con 0x55ff7b631b10 2023-10-17T01:19:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.392+0000 7f07c2ccab80 10 client.26024 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.392+0000 7f07c2ccab80 10 client.26024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f07c2ccab80 20 client.26024 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b645860 0x55ff7b647c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f07c2ccab80 4 client.26024 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:19:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff7b5f6cd0 con 0x55ff7b645860 2023-10-17T01:19:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f07ad19b700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b645860 0x55ff7b647c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f0790ff9700 10 client.26024 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.393+0000 7f07ad19b700 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b645860 0x55ff7b647c80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15f99b8714f4016 server_cookie=9ac752453cb9c2e6 in_seq=0 out_seq=0 2023-10-17T01:19:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.394+0000 7f0790ff9700 1 -- 172.21.15.45:0/3964436392 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55ff7b5f6cd0 con 0x55ff7b645860 2023-10-17T01:19:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.394+0000 7f0790ff9700 10 client.26024 handle_command_reply: tid=0 2023-10-17T01:19:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.394+0000 7f07c2ccab80 1 client.26024 shutdown 2023-10-17T01:19:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.394+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 msgr2=0x55ff7b640e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 0x55ff7b640e00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07a0004290 tx=0x7f07a001d000 comp rx=0 tx=0).stop 2023-10-17T01:19:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 shutdown_connections 2023-10-17T01:19:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b645860 0x55ff7b647c80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b631b10 0x55ff7b640e00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b639a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/3964436392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b6311b0 0x55ff7b639fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 >> 172.21.15.45:0/3964436392 conn(0x55ff7b587070 msgr2=0x55ff7b621a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f0790ff9700 0 client.26024 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 shutdown_connections 2023-10-17T01:19:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 1 -- 172.21.15.45:0/3964436392 wait complete. 2023-10-17T01:19:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.395+0000 7f07c2ccab80 20 client.26024 trim_cache size 0 max 16384 2023-10-17T01:19:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.419+0000 7f07c2ccab80 1 Processor -- start 2023-10-17T01:19:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 -- start start 2023-10-17T01:19:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b633600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b621a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b634170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b638640 con 0x55ff7b6303e0 2023-10-17T01:19:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b623d50 con 0x55ff7b6311b0 2023-10-17T01:19:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.420+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b634d00 con 0x55ff7b631b10 2023-10-17T01:19:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad19b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b633600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b634170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b634170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40676/0 (socket says 172.21.15.45:40676) 2023-10-17T01:19:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ac99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b621a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 -- 172.21.15.45:0/2168180650 learned_addr learned my addr 172.21.15.45:0/2168180650 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 -- 172.21.15.45:0/2168180650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 msgr2=0x55ff7b621a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b621a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 -- 172.21.15.45:0/2168180650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 msgr2=0x55ff7b633600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b633600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 -- 172.21.15.45:0/2168180650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7b3cd1f0 con 0x55ff7b631b10 2023-10-17T01:19:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.422+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b634170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07a0004260 tx=0x7f07a00137d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e6efbdf570d746 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.423+0000 7f0793fff700 1 -- 172.21.15.45:0/2168180650 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000fbc0 con 0x55ff7b631b10 2023-10-17T01:19:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.423+0000 7f0793fff700 1 -- 172.21.15.45:0/2168180650 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a0010c70 con 0x55ff7b631b10 2023-10-17T01:19:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.423+0000 7f0793fff700 1 -- 172.21.15.45:0/2168180650 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00038e0 con 0x55ff7b631b10 2023-10-17T01:19:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2168180650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 msgr2=0x55ff7b634170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b634170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07a0004260 tx=0x7f07a00137d0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2168180650 shutdown_connections 2023-10-17T01:19:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b634170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b633600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2168180650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b621a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2168180650 >> 172.21.15.45:0/2168180650 conn(0x55ff7b587070 msgr2=0x55ff7b6307c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2168180650 shutdown_connections 2023-10-17T01:19:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2168180650 wait complete. 2023-10-17T01:19:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.424+0000 7f07c2ccab80 1 -- start start 2023-10-17T01:19:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.425+0000 7f07c2ccab80 1 Processor -- start 2023-10-17T01:19:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b63ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b63d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b63d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b63de70 con 0x55ff7b6303e0 2023-10-17T01:19:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b63dfe0 con 0x55ff7b6311b0 2023-10-17T01:19:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7b6a8200 con 0x55ff7b631b10 2023-10-17T01:19:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.426+0000 7f07c2ccab80 10 client.0 _ll_register_callbacks cb 0x55ff7b462a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:19:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.427+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b63d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.427+0000 7f07ad99c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b63d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40678/0 (socket says 172.21.15.45:40678) 2023-10-17T01:19:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.427+0000 7f07ad99c700 1 -- 172.21.15.45:0/2381722426 learned_addr learned my addr 172.21.15.45:0/2381722426 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:19:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.427+0000 7f07ad19b700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b63ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.427+0000 7f07ac99a700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b63d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f07ad99c700 1 -- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 msgr2=0x55ff7b63d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b63d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f07ad99c700 1 -- 172.21.15.45:0/2381722426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 msgr2=0x55ff7b63ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:19:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b63ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f07ad99c700 1 -- 172.21.15.45:0/2381722426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff7b38fc00 con 0x55ff7b631b10 2023-10-17T01:19:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f0790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:19:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.428+0000 7f07ad99c700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b63d780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f07a0013710 tx=0x7f07a0013740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63777fb2e6e6795 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:19:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f0790ff9700 1 -- 172.21.15.45:0/2381722426 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0010640 con 0x55ff7b631b10 2023-10-17T01:19:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079801e040 con 0x55ff7b631b10 2023-10-17T01:19:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f0790ff9700 1 -- 172.21.15.45:0/2381722426 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07a0025040 con 0x55ff7b631b10 2023-10-17T01:19:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f0790ff9700 1 -- 172.21.15.45:0/2381722426 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a00107a0 con 0x55ff7b631b10 2023-10-17T01:19:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 10 client.26002 fetch_fsmap learned FSMap version 40 2023-10-17T01:19:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ff7b3a5250 con 0x55ff7b631b10 2023-10-17T01:19:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f0790ff9700 1 -- 172.21.15.45:0/2381722426 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079801e040 con 0x55ff7b631b10 2023-10-17T01:19:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f0790ff9700 1 -- 172.21.15.45:0/2381722426 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f07a0003ce0 con 0x55ff7b631b10 2023-10-17T01:19:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 10 client.26002 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:19:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:19:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 20 client.26002 populate_metadata read hostname 'smithi045' 2023-10-17T01:19:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b6a97f0 0x55ff7b6abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:19:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 4 client.26002 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:19:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.429+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff7b571ce0 con 0x55ff7b6a97f0 2023-10-17T01:19:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.430+0000 7f07ad19b700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b6a97f0 0x55ff7b6abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:19:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.432+0000 7f0790ff9700 10 client.26002 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.431+0000 7f07ad19b700 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b6a97f0 0x55ff7b6abc10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60205b64d8208365 server_cookie=560639260d0c55e in_seq=0 out_seq=0 2023-10-17T01:19:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.445+0000 7f0790ff9700 1 -- 172.21.15.45:0/2381722426 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff7b571ce0 con 0x55ff7b6a97f0 2023-10-17T01:19:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.445+0000 7f0790ff9700 10 client.26002 handle_command_reply: tid=0 2023-10-17T01:19:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.446+0000 7f07c2ccab80 1 client.26002 shutdown 2023-10-17T01:19:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 msgr2=0x55ff7b63d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b63d780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f07a0013710 tx=0x7f07a0013740 comp rx=0 tx=0).stop 2023-10-17T01:19:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 shutdown_connections 2023-10-17T01:19:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff7b631b10 0x55ff7b63d780 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff7b6303e0 0x55ff7b63ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55ff7b6a97f0 0x55ff7b6abc10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 --2- 172.21.15.45:0/2381722426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ff7b6311b0 0x55ff7b63d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f0790ff9700 0 client.26002 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:19:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 >> 172.21.15.45:0/2381722426 conn(0x55ff7b587070 msgr2=0x55ff7b633600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 shutdown_connections 2023-10-17T01:19:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 1 -- 172.21.15.45:0/2381722426 wait complete. 2023-10-17T01:19:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.447+0000 7f07c2ccab80 20 client.26002 trim_cache size 0 max 16384 2023-10-17T01:19:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:19:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:19:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:19:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:19:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07940a0cf0 msgr2=0x7f07940a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07940a0cf0 0x7f07940a31b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f07a00139e0 tx=0x7f07a0009c30 comp rx=0 tx=0).stop 2023-10-17T01:19:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 msgr2=0x7f07a8198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:19:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 0x7f07a8198da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07a400efc0 tx=0x7f07a400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 shutdown_connections 2023-10-17T01:19:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f07940a0cf0 0x7f07940a31b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a8104b40 0x7f07a819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a8102fb0 0x7f07a8198da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 --2- 172.21.15.45:0/2620743842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a81041a0 0x7f07a81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:19:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 >> 172.21.15.45:0/2620743842 conn(0x7f07a80fe540 msgr2=0x7f07a8100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:19:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 shutdown_connections 2023-10-17T01:19:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:19:58.448+0000 7f07af3ff700 1 -- 172.21.15.45:0/2620743842 wait complete. 2023-10-17T01:19:59.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:19:58 smithi045 ceph-mon[98474]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 678 op/s 2023-10-17T01:19:59.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:19:58 smithi060 ceph-mon[134136]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 678 op/s 2023-10-17T01:19:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:19:58 smithi181 ceph-mon[134520]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 678 op/s 2023-10-17T01:20:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 290.0 s, 2343.0 tps, lat 42.860 ms stddev 50.176 2023-10-17T01:20:00.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:00 smithi181 ceph-mon[134520]: overall HEALTH_OK 2023-10-17T01:20:01.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:00 smithi045 ceph-mon[98474]: overall HEALTH_OK 2023-10-17T01:20:01.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:00 smithi060 ceph-mon[134136]: overall HEALTH_OK 2023-10-17T01:20:01.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:20:01.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 -- 172.21.15.45:0/3096731528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 msgr2=0x7f32b0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 --2- 172.21.15.45:0/3096731528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b0101180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009990 tx=0x7f32a0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 -- 172.21.15.45:0/3096731528 shutdown_connections 2023-10-17T01:20:01.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 --2- 172.21.15.45:0/3096731528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 0x7f32b01091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 --2- 172.21.15.45:0/3096731528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b0101180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 --2- 172.21.15.45:0/3096731528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 0x7f32b01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 -- 172.21.15.45:0/3096731528 >> 172.21.15.45:0/3096731528 conn(0x7f32b00fc2d0 msgr2=0x7f32b00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:01.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.582+0000 7f32b5bff700 1 -- 172.21.15.45:0/3096731528 shutdown_connections 2023-10-17T01:20:01.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.583+0000 7f32b5bff700 1 -- 172.21.15.45:0/3096731528 wait complete. 2023-10-17T01:20:01.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.584+0000 7f32b5bff700 1 Processor -- start 2023-10-17T01:20:01.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.584+0000 7f32b5bff700 1 -- start start 2023-10-17T01:20:01.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32b5bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32b5bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 0x7f32b006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32b5bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 0x7f32b01a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32b5bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0072c00 con 0x7f32b0101f50 2023-10-17T01:20:01.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32b5bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0072d70 con 0x7f32b01028f0 2023-10-17T01:20:01.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32b5bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01a7180 con 0x7f32b0100d60 2023-10-17T01:20:01.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32affff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 0x7f32b01a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40686/0 (socket says 172.21.15.45:40686) 2023-10-17T01:20:01.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32af7fe700 1 -- 172.21.15.45:0/104489276 learned_addr learned my addr 172.21.15.45:0/104489276 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:01.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.585+0000 7f32affff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 0x7f32b01a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41698/0 (socket says 172.21.15.45:41698) 2023-10-17T01:20:01.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32aeffd700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 0x7f32b006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32af7fe700 1 -- 172.21.15.45:0/104489276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 msgr2=0x7f32b01a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32af7fe700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 0x7f32b01a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32af7fe700 1 -- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 msgr2=0x7f32b006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32af7fe700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 0x7f32b006fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32af7fe700 1 -- 172.21.15.45:0/104489276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00095f0 con 0x7f32b0100d60 2023-10-17T01:20:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32aeffd700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 0x7f32b006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32af7fe700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b006f5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004710 tx=0x7f32a0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32acff9700 1 -- 172.21.15.45:0/104489276 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a001d070 con 0x7f32b0100d60 2023-10-17T01:20:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32acff9700 1 -- 172.21.15.45:0/104489276 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a0021410 con 0x7f32b0100d60 2023-10-17T01:20:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32acff9700 1 -- 172.21.15.45:0/104489276 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0017650 con 0x7f32b0100d60 2023-10-17T01:20:01.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.586+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b01a7380 con 0x7f32b0100d60 2023-10-17T01:20:01.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.587+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b01a7750 con 0x7f32b0100d60 2023-10-17T01:20:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.593+0000 7f32c9520b80 1 Processor -- start 2023-10-17T01:20:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.593+0000 7f32acff9700 1 -- 172.21.15.45:0/104489276 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f32a0021580 con 0x7f32b0100d60 2023-10-17T01:20:01.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 -- start start 2023-10-17T01:20:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d375ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3760d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 0x55f0d3761730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d36b7a10 con 0x55f0d3760500 2023-10-17T01:20:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d36b53d0 con 0x55f0d375fba0 2023-10-17T01:20:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.594+0000 7f32c9520b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d36b0cc0 con 0x55f0d37612b0 2023-10-17T01:20:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 0x55f0d3761730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 0x55f0d3761730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40700/0 (socket says 172.21.15.45:40700) 2023-10-17T01:20:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 -- 172.21.15.45:0/2351996360 learned_addr learned my addr 172.21.15.45:0/2351996360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32affff700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d375ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 -- 172.21.15.45:0/2351996360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 msgr2=0x55f0d375ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d375ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 -- 172.21.15.45:0/2351996360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 msgr2=0x55f0d3760d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32af7fe700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3760d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3760d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 -- 172.21.15.45:0/2351996360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d34dede0 con 0x55f0d37612b0 2023-10-17T01:20:01.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32affff700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d375ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:20:01.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32af7fe700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3760d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:01.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.595+0000 7f32aeffd700 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 0x55f0d3761730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009cc0 tx=0x7f32a4010370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98fe6a0969cfebb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:01.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.596+0000 7f3295ffb700 1 -- 172.21.15.45:0/2351996360 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4010c70 con 0x55f0d37612b0 2023-10-17T01:20:01.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.596+0000 7f3295ffb700 1 -- 172.21.15.45:0/2351996360 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a40044f0 con 0x55f0d37612b0 2023-10-17T01:20:01.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.596+0000 7f3295ffb700 1 -- 172.21.15.45:0/2351996360 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4012450 con 0x55f0d37612b0 2023-10-17T01:20:01.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.597+0000 7f32c9520b80 1 -- 172.21.15.45:0/2351996360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 msgr2=0x55f0d3761730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.597+0000 7f32c9520b80 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 0x55f0d3761730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009cc0 tx=0x7f32a4010370 comp rx=0 tx=0).stop 2023-10-17T01:20:01.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32acff9700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32980a0b20 0x7f32980a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32acff9700 1 -- 172.21.15.45:0/104489276 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32a00cd610 con 0x7f32b0100d60 2023-10-17T01:20:01.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32c9520b80 1 -- 172.21.15.45:0/2351996360 shutdown_connections 2023-10-17T01:20:01.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32c9520b80 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d37612b0 0x55f0d3761730 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32c9520b80 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d375ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32c9520b80 1 --2- 172.21.15.45:0/2351996360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3760d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32c9520b80 1 -- 172.21.15.45:0/2351996360 >> 172.21.15.45:0/2351996360 conn(0x55f0d36bf0d0 msgr2=0x55f0d375a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:01.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.598+0000 7f32affff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32980a0b20 0x7f32980a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.599+0000 7f32c9520b80 1 -- 172.21.15.45:0/2351996360 shutdown_connections 2023-10-17T01:20:01.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.599+0000 7f32c9520b80 1 -- 172.21.15.45:0/2351996360 wait complete. 2023-10-17T01:20:01.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.600+0000 7f32affff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32980a0b20 0x7f32980a2fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55f0d375f940 tx=0x7f32a800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:01.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.600+0000 7f32c9520b80 1 -- start start 2023-10-17T01:20:01.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 Processor -- start 2023-10-17T01:20:01.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d375fba0 0x55f0d37720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3772620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3779660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d364d510 con 0x55f0d375fba0 2023-10-17T01:20:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d364d1b0 con 0x55f0d37612b0 2023-10-17T01:20:01.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.601+0000 7f32c9520b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d3534920 con 0x55f0d3760500 2023-10-17T01:20:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32c9520b80 10 client.0 _ll_register_callbacks cb 0x55f0d359aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3772620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32affff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3779660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3772620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40712/0 (socket says 172.21.15.45:40712) 2023-10-17T01:20:01.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 -- 172.21.15.45:0/1907254790 learned_addr learned my addr 172.21.15.45:0/1907254790 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 -- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 msgr2=0x55f0d3779660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3779660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 -- 172.21.15.45:0/1907254790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d375fba0 msgr2=0x55f0d37720e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:20:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d375fba0 0x55f0d37720e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.602+0000 7f32aeffd700 1 -- 172.21.15.45:0/1907254790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a4009920 con 0x55f0d3760500 2023-10-17T01:20:01.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3772620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32a4000c00 tx=0x7f32a4003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bcba9a300af3958 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:01.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f328effd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:20:01.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f328effd700 1 -- 172.21.15.45:0/1907254790 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4005230 con 0x55f0d3760500 2023-10-17T01:20:01.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0d36dbd10 con 0x55f0d3760500 2023-10-17T01:20:01.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f328effd700 1 -- 172.21.15.45:0/1907254790 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32a4016070 con 0x55f0d3760500 2023-10-17T01:20:01.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f328effd700 1 -- 172.21.15.45:0/1907254790 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32a40038a0 con 0x55f0d3760500 2023-10-17T01:20:01.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f32c9520b80 10 client.26014 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:01.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.603+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f0d34c7c60 con 0x55f0d3760500 2023-10-17T01:20:01.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f328effd700 1 -- 172.21.15.45:0/1907254790 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0d36dbd10 con 0x55f0d3760500 2023-10-17T01:20:01.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f328effd700 1 -- 172.21.15.45:0/1907254790 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f32a4005390 con 0x55f0d3760500 2023-10-17T01:20:01.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f32c9520b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f32c9520b80 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f32c9520b80 20 client.26014 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d377dd40 0x55f0d3780160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f32c9520b80 4 client.26014 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:20:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.604+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0d372ed30 con 0x55f0d377dd40 2023-10-17T01:20:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.605+0000 7f32af7fe700 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d377dd40 0x55f0d3780160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.605+0000 7f32af7fe700 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d377dd40 0x55f0d3780160 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4dbf3b61567210 server_cookie=336456ebb37c9c3c in_seq=0 out_seq=0 2023-10-17T01:20:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.605+0000 7f328effd700 10 client.26014 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.606+0000 7f328effd700 1 -- 172.21.15.45:0/1907254790 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55f0d372ed30 con 0x55f0d377dd40 2023-10-17T01:20:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.606+0000 7f328effd700 10 client.26014 handle_command_reply: tid=0 2023-10-17T01:20:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 client.26014 shutdown 2023-10-17T01:20:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 msgr2=0x55f0d3772620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3772620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32a4000c00 tx=0x7f32a4003eb0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 shutdown_connections 2023-10-17T01:20:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d377dd40 0x55f0d3780160 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3779660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d375fba0 0x55f0d37720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1907254790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3772620 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 >> 172.21.15.45:0/1907254790 conn(0x55f0d36bf0d0 msgr2=0x55f0d375a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:01.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.607+0000 7f328effd700 0 client.26014 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.608+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 shutdown_connections 2023-10-17T01:20:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.608+0000 7f32c9520b80 1 -- 172.21.15.45:0/1907254790 wait complete. 2023-10-17T01:20:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.608+0000 7f32c9520b80 20 client.26014 trim_cache size 0 max 16384 2023-10-17T01:20:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.638+0000 7f32c9520b80 1 Processor -- start 2023-10-17T01:20:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.638+0000 7f32c9520b80 1 -- start start 2023-10-17T01:20:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d3761690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3771790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d37612b0 0x55f0d3767710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32c9520b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d3754f00 con 0x55f0d37612b0 2023-10-17T01:20:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32c9520b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d377dae0 con 0x55f0d375fba0 2023-10-17T01:20:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32c9520b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d3757420 con 0x55f0d3760500 2023-10-17T01:20:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3771790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3771790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40714/0 (socket says 172.21.15.45:40714) 2023-10-17T01:20:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32aeffd700 1 -- 172.21.15.45:0/3541173359 learned_addr learned my addr 172.21.15.45:0/3541173359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32af7fe700 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d3761690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32affff700 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d37612b0 0x55f0d3767710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.639+0000 7f32aeffd700 1 -- 172.21.15.45:0/3541173359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 msgr2=0x55f0d3761690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.640+0000 7f32aeffd700 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d3761690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.640+0000 7f32aeffd700 1 -- 172.21.15.45:0/3541173359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d37612b0 msgr2=0x55f0d3767710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.640+0000 7f32aeffd700 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d37612b0 0x55f0d3767710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.640+0000 7f32aeffd700 1 -- 172.21.15.45:0/3541173359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d3505250 con 0x55f0d3760500 2023-10-17T01:20:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.640+0000 7f32af7fe700 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d3761690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:01.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.640+0000 7f32aeffd700 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3771790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f32a400a010 tx=0x7f32a4009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948fa687f3c01f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:01.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.641+0000 7f32967fc700 1 -- 172.21.15.45:0/3541173359 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a40058e0 con 0x55f0d3760500 2023-10-17T01:20:01.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.641+0000 7f32967fc700 1 -- 172.21.15.45:0/3541173359 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a401f070 con 0x55f0d3760500 2023-10-17T01:20:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.641+0000 7f32967fc700 1 -- 172.21.15.45:0/3541173359 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4012a90 con 0x55f0d3760500 2023-10-17T01:20:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 -- 172.21.15.45:0/3541173359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 msgr2=0x55f0d3771790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3771790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f32a400a010 tx=0x7f32a4009400 comp rx=0 tx=0).stop 2023-10-17T01:20:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 -- 172.21.15.45:0/3541173359 shutdown_connections 2023-10-17T01:20:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d37612b0 0x55f0d3767710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d375fba0 0x55f0d3761690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 --2- 172.21.15.45:0/3541173359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d3760500 0x55f0d3771790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 -- 172.21.15.45:0/3541173359 >> 172.21.15.45:0/3541173359 conn(0x55f0d36bf0d0 msgr2=0x55f0d3770e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 -- 172.21.15.45:0/3541173359 shutdown_connections 2023-10-17T01:20:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.642+0000 7f32c9520b80 1 -- 172.21.15.45:0/3541173359 wait complete. 2023-10-17T01:20:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.643+0000 7f32c9520b80 1 -- start start 2023-10-17T01:20:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.643+0000 7f32c9520b80 1 Processor -- start 2023-10-17T01:20:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 0x55f0d3775590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3775ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3776ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d3777020 con 0x55f0d3760500 2023-10-17T01:20:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d3777190 con 0x55f0d37612b0 2023-10-17T01:20:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d3777300 con 0x55f0d375fba0 2023-10-17T01:20:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32c9520b80 10 client.0 _ll_register_callbacks cb 0x7f32a4009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 0x55f0d3775590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 0x55f0d3775590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40718/0 (socket says 172.21.15.45:40718) 2023-10-17T01:20:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32af7fe700 1 -- 172.21.15.45:0/1461347997 learned_addr learned my addr 172.21.15.45:0/1461347997 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3775ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.644+0000 7f32affff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3776ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.645+0000 7f32af7fe700 1 -- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 msgr2=0x55f0d3776ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.645+0000 7f32af7fe700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3776ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.645+0000 7f32af7fe700 1 -- 172.21.15.45:0/1461347997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 msgr2=0x55f0d3775ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.645+0000 7f32af7fe700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3775ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.645+0000 7f32af7fe700 1 -- 172.21.15.45:0/1461347997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0d34c7c60 con 0x55f0d375fba0 2023-10-17T01:20:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.646+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3775ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:20:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.646+0000 7f32af7fe700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 0x55f0d3775590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f32a009ac50 tx=0x7f32a009abb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fb10b84a2405aa server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.646+0000 7f328effd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:20:01.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.646+0000 7f328effd700 1 -- 172.21.15.45:0/1461347997 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0021a80 con 0x55f0d375fba0 2023-10-17T01:20:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f328effd700 1 -- 172.21.15.45:0/1461347997 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32a009af20 con 0x55f0d375fba0 2023-10-17T01:20:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32a40044f0 con 0x55f0d375fba0 2023-10-17T01:20:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f328effd700 1 -- 172.21.15.45:0/1461347997 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32a009b0e0 con 0x55f0d375fba0 2023-10-17T01:20:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f32c9520b80 10 client.26020 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f0d34dd2b0 con 0x55f0d375fba0 2023-10-17T01:20:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f328effd700 1 -- 172.21.15.45:0/1461347997 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a00b3020 con 0x55f0d375fba0 2023-10-17T01:20:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.647+0000 7f328effd700 1 -- 172.21.15.45:0/1461347997 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f32a00a2810 con 0x55f0d375fba0 2023-10-17T01:20:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32c9520b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32c9520b80 10 client.26020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32c9520b80 20 client.26020 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d37d0490 0x55f0d37d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32c9520b80 4 client.26020 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:20:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f0d36a9d40 con 0x55f0d37d0490 2023-10-17T01:20:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d37d0490 0x55f0d37d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f32aeffd700 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d37d0490 0x55f0d37d28b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa3511e77bbe6ac server_cookie=a4b4eaec3085625b in_seq=0 out_seq=0 2023-10-17T01:20:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.648+0000 7f328effd700 10 client.26020 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f328effd700 1 -- 172.21.15.45:0/1461347997 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f0d36a9d40 con 0x55f0d37d0490 2023-10-17T01:20:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f328effd700 10 client.26020 handle_command_reply: tid=0 2023-10-17T01:20:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 client.26020 shutdown 2023-10-17T01:20:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 msgr2=0x55f0d3775590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 0x55f0d3775590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f32a009ac50 tx=0x7f32a009abb0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 shutdown_connections 2023-10-17T01:20:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f0d37d0490 0x55f0d37d28b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0d37612b0 0x55f0d3776ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f0d375fba0 0x55f0d3775590 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 --2- 172.21.15.45:0/1461347997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0d3760500 0x55f0d3775ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 >> 172.21.15.45:0/1461347997 conn(0x55f0d36bf0d0 msgr2=0x55f0d3767710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.649+0000 7f328effd700 0 client.26020 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.650+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 shutdown_connections 2023-10-17T01:20:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.650+0000 7f32c9520b80 1 -- 172.21.15.45:0/1461347997 wait complete. 2023-10-17T01:20:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.651+0000 7f32c9520b80 20 client.26020 trim_cache size 0 max 16384 2023-10-17T01:20:01.662 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:20:01.662 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:20:01.662 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:20:01.662 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:20:01.663 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:20:01.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:20:01.663 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:20:01.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:20:01.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:20:01.664 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:20:01.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32980a0b20 msgr2=0x7f32980a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32980a0b20 0x7f32980a2fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55f0d375f940 tx=0x7f32a800b040 comp rx=0 tx=0).stop 2023-10-17T01:20:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 msgr2=0x7f32b006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b006f5e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004710 tx=0x7f32a0009580 comp rx=0 tx=0).stop 2023-10-17T01:20:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 shutdown_connections 2023-10-17T01:20:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32980a0b20 0x7f32980a2fe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b01028f0 0x7f32b01a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32b0100d60 0x7f32b006f5e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 --2- 172.21.15.45:0/104489276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32b0101f50 0x7f32b006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.652+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 >> 172.21.15.45:0/104489276 conn(0x7f32b00fc2d0 msgr2=0x7f32b0105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:01.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.653+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 shutdown_connections 2023-10-17T01:20:01.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:01.653+0000 7f32b5bff700 1 -- 172.21.15.45:0/104489276 wait complete. 2023-10-17T01:20:01.693 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:20:01.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:01 smithi181 ceph-mon[134520]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 780 op/s 2023-10-17T01:20:02.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:01 smithi045 ceph-mon[98474]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 780 op/s 2023-10-17T01:20:02.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:01 smithi060 ceph-mon[134136]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 780 op/s 2023-10-17T01:20:03.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:03 smithi181 ceph-mon[134520]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 671 op/s 2023-10-17T01:20:04.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:03 smithi045 ceph-mon[98474]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 671 op/s 2023-10-17T01:20:04.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:03 smithi060 ceph-mon[134136]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 671 op/s 2023-10-17T01:20:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 295.0 s, 2155.8 tps, lat 46.318 ms stddev 48.831 2023-10-17T01:20:05.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:05 smithi181 ceph-mon[134520]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 727 op/s 2023-10-17T01:20:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:05 smithi045 ceph-mon[98474]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 727 op/s 2023-10-17T01:20:06.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:05 smithi060 ceph-mon[134136]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 727 op/s 2023-10-17T01:20:07.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:07 smithi181 ceph-mon[134520]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 625 op/s 2023-10-17T01:20:08.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:07 smithi045 ceph-mon[98474]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 625 op/s 2023-10-17T01:20:08.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:07 smithi060 ceph-mon[134136]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 625 op/s 2023-10-17T01:20:09.910 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:09 smithi181 ceph-mon[134520]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 621 op/s 2023-10-17T01:20:10.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:09 smithi045 ceph-mon[98474]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 621 op/s 2023-10-17T01:20:10.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:09 smithi060 ceph-mon[134136]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 621 op/s 2023-10-17T01:20:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 300.0 s, 1699.4 tps, lat 58.981 ms stddev 68.219 2023-10-17T01:20:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 -- 172.21.15.45:0/1757343209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8101fa0 msgr2=0x7f6dd8102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 --2- 172.21.15.45:0/1757343209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8101fa0 0x7f6dd8102400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009990 tx=0x7f6dc0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 -- 172.21.15.45:0/1757343209 shutdown_connections 2023-10-17T01:20:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 --2- 172.21.15.45:0/1757343209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8102940 0x7f6dd81091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 --2- 172.21.15.45:0/1757343209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6dd8100db0 0x7f6dd81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 --2- 172.21.15.45:0/1757343209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8101fa0 0x7f6dd8102400 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 -- 172.21.15.45:0/1757343209 >> 172.21.15.45:0/1757343209 conn(0x7f6dd80fc300 msgr2=0x7f6dd80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 -- 172.21.15.45:0/1757343209 shutdown_connections 2023-10-17T01:20:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.677+0000 7f6dddfff700 1 -- 172.21.15.45:0/1757343209 wait complete. 2023-10-17T01:20:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.678+0000 7f6dddfff700 1 Processor -- start 2023-10-17T01:20:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 -- start start 2023-10-17T01:20:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8100db0 0x7f6dd8198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6dd8101fa0 0x7f6dd8199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 0x7f6dd819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819a830 con 0x7f6dd8100db0 2023-10-17T01:20:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819a9a0 con 0x7f6dd8101fa0 2023-10-17T01:20:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.679+0000 7f6dddfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819ab60 con 0x7f6dd8102940 2023-10-17T01:20:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.680+0000 7f6dd7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 0x7f6dd819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.680+0000 7f6dd7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 0x7f6dd819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50988/0 (socket says 172.21.15.45:50988) 2023-10-17T01:20:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.680+0000 7f6dd7fff700 1 -- 172.21.15.45:0/713050637 learned_addr learned my addr 172.21.15.45:0/713050637 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd7fff700 1 -- 172.21.15.45:0/713050637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6dd8101fa0 msgr2=0x7f6dd8199380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:20:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6dd8101fa0 0x7f6dd8199380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd7fff700 1 -- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8100db0 msgr2=0x7f6dd8198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8100db0 0x7f6dd8198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8100db0 0x7f6dd8198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd7fff700 1 -- 172.21.15.45:0/713050637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00095f0 con 0x7f6dd8102940 2023-10-17T01:20:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8100db0 0x7f6dd8198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.681+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 0x7f6dd819a120 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b550 tx=0x7f6dd000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.682+0000 7f6dd57fa700 1 -- 172.21.15.45:0/713050637 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0011830 con 0x7f6dd8102940 2023-10-17T01:20:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.682+0000 7f6dd57fa700 1 -- 172.21.15.45:0/713050637 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd0011e50 con 0x7f6dd8102940 2023-10-17T01:20:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.682+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd819ae40 con 0x7f6dd8102940 2023-10-17T01:20:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.682+0000 7f6dd57fa700 1 -- 172.21.15.45:0/713050637 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000f2e0 con 0x7f6dd8102940 2023-10-17T01:20:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.682+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd806aa20 con 0x7f6dd8102940 2023-10-17T01:20:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.687+0000 7f6dd57fa700 1 -- 172.21.15.45:0/713050637 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f6dd0011990 con 0x7f6dd8102940 2023-10-17T01:20:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 Processor -- start 2023-10-17T01:20:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 -- start start 2023-10-17T01:20:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc877ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6dd57fa700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6dc40a0b20 0x7f6dc40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc877b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc877c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc86d29b0 con 0x55dfc877bd90 2023-10-17T01:20:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc86d0370 con 0x55dfc877b190 2023-10-17T01:20:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.690+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc86cbc60 con 0x55dfc877a830 2023-10-17T01:20:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc877ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc877ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:50998/0 (socket says 172.21.15.45:50998) 2023-10-17T01:20:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2470172810 learned_addr learned my addr 172.21.15.45:0/2470172810 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dcffff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6dc40a0b20 0x7f6dc40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc877c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dcffff700 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc877b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2470172810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 msgr2=0x55dfc877b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc877b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2470172810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 msgr2=0x55dfc877c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc877c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd57fa700 1 -- 172.21.15.45:0/713050637 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6dd00cbc50 con 0x7f6dd8102940 2023-10-17T01:20:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2470172810 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc84f9d80 con 0x55dfc877a830 2023-10-17T01:20:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dcffff700 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc877b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.691+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc877ac50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80093e0 tx=0x7f6dc8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=437e98c14ac4369e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.692+0000 7f6dcdffb700 1 -- 172.21.15.45:0/2470172810 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8019070 con 0x55dfc877a830 2023-10-17T01:20:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.692+0000 7f6dcdffb700 1 -- 172.21.15.45:0/2470172810 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc800f050 con 0x55dfc877a830 2023-10-17T01:20:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.692+0000 7f6dcdffb700 1 -- 172.21.15.45:0/2470172810 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8015ea0 con 0x55dfc877a830 2023-10-17T01:20:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.692+0000 7f6dcffff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6dc40a0b20 0x7f6dc40a2fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b680 tx=0x7f6dc0005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2470172810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 msgr2=0x55dfc877ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc877ac50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80093e0 tx=0x7f6dc8009920 comp rx=0 tx=0).stop 2023-10-17T01:20:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2470172810 shutdown_connections 2023-10-17T01:20:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc877c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc877ac50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2470172810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc877b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.693+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2470172810 >> 172.21.15.45:0/2470172810 conn(0x55dfc86da070 msgr2=0x55dfc87752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.694+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2470172810 shutdown_connections 2023-10-17T01:20:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.694+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2470172810 wait complete. 2023-10-17T01:20:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.695+0000 7f6df19b3b80 1 -- start start 2023-10-17T01:20:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.695+0000 7f6df19b3b80 1 Processor -- start 2023-10-17T01:20:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877a830 0x55dfc8784370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc87848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877bd90 0x55dfc878b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc86684b0 con 0x55dfc877a830 2023-10-17T01:20:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc8668150 con 0x55dfc877b190 2023-10-17T01:20:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc854f8c0 con 0x55dfc877bd90 2023-10-17T01:20:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6df19b3b80 10 client.0 _ll_register_callbacks cb 0x55dfc85b5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc87848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc87848b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39392/0 (socket says 172.21.15.45:39392) 2023-10-17T01:20:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6dcffff700 1 -- 172.21.15.45:0/3148747797 learned_addr learned my addr 172.21.15.45:0/3148747797 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.696+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877a830 0x55dfc8784370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877bd90 0x55dfc878b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 -- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 msgr2=0x55dfc87848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc87848b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 -- 172.21.15.45:0/3148747797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877a830 msgr2=0x55dfc8784370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877a830 0x55dfc8784370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 -- 172.21.15.45:0/3148747797 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc8009090 con 0x55dfc877bd90 2023-10-17T01:20:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877bd90 0x55dfc878b8f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00c9af0 tx=0x7f6dd0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f3c5f538c63bbf server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.697+0000 7f6db6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.698+0000 7f6db6ffd700 1 -- 172.21.15.45:0/3148747797 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0010eb0 con 0x55dfc877bd90 2023-10-17T01:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.698+0000 7f6db6ffd700 1 -- 172.21.15.45:0/3148747797 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6dd00999d0 con 0x55dfc877bd90 2023-10-17T01:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.698+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfc86f6cb0 con 0x55dfc877bd90 2023-10-17T01:20:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.698+0000 7f6db6ffd700 1 -- 172.21.15.45:0/3148747797 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd0010380 con 0x55dfc877bd90 2023-10-17T01:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.699+0000 7f6df19b3b80 10 client.26032 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.699+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55dfc84e2c00 con 0x55dfc877bd90 2023-10-17T01:20:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.700+0000 7f6db6ffd700 1 -- 172.21.15.45:0/3148747797 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfc86f6cb0 con 0x55dfc877bd90 2023-10-17T01:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.700+0000 7f6db6ffd700 1 -- 172.21.15.45:0/3148747797 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f6dd0099530 con 0x55dfc877bd90 2023-10-17T01:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.701+0000 7f6df19b3b80 10 client.26032 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.701+0000 7f6df19b3b80 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.701+0000 7f6df19b3b80 20 client.26032 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.701+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc8798940 0x55dfc879ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.701+0000 7f6df19b3b80 4 client.26032 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.701+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfc8749cd0 con 0x55dfc8798940 2023-10-17T01:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.702+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc8798940 0x55dfc879ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.703+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc8798940 0x55dfc879ad60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c056b508f7b703bc server_cookie=19386eb62af7bfc7 in_seq=0 out_seq=0 2023-10-17T01:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.703+0000 7f6db6ffd700 10 client.26032 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.703+0000 7f6db6ffd700 1 -- 172.21.15.45:0/3148747797 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55dfc8749cd0 con 0x55dfc8798940 2023-10-17T01:20:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6db6ffd700 10 client.26032 handle_command_reply: tid=0 2023-10-17T01:20:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 client.26032 shutdown 2023-10-17T01:20:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877bd90 msgr2=0x55dfc878b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877bd90 0x55dfc878b8f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00c9af0 tx=0x7f6dd0004860 comp rx=0 tx=0).stop 2023-10-17T01:20:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 shutdown_connections 2023-10-17T01:20:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877bd90 0x55dfc878b8f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877a830 0x55dfc8784370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc8798940 0x55dfc879ad60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/3148747797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877b190 0x55dfc87848b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 >> 172.21.15.45:0/3148747797 conn(0x55dfc86da070 msgr2=0x55dfc8774a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6db6ffd700 0 client.26032 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.704+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 shutdown_connections 2023-10-17T01:20:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.705+0000 7f6df19b3b80 1 -- 172.21.15.45:0/3148747797 wait complete. 2023-10-17T01:20:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.705+0000 7f6df19b3b80 20 client.26032 trim_cache size 0 max 16384 2023-10-17T01:20:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.732+0000 7f6df19b3b80 1 Processor -- start 2023-10-17T01:20:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.732+0000 7f6df19b3b80 1 -- start start 2023-10-17T01:20:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.732+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc879ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877b190 0x55dfc8783060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877bd90 0x55dfc8774a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc86f6cb0 con 0x55dfc877b190 2023-10-17T01:20:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc8795d60 con 0x55dfc877bd90 2023-10-17T01:20:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc8785270 con 0x55dfc877a830 2023-10-17T01:20:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6dd77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc879ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6dd77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc879ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51028/0 (socket says 172.21.15.45:51028) 2023-10-17T01:20:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2425484318 learned_addr learned my addr 172.21.15.45:0/2425484318 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.733+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877bd90 0x55dfc8774a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.734+0000 7f6dcffff700 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877b190 0x55dfc8783060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.734+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2425484318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877bd90 msgr2=0x55dfc8774a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.734+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877bd90 0x55dfc8774a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.734+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2425484318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877b190 msgr2=0x55dfc8783060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.734+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877b190 0x55dfc8783060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.734+0000 7f6dd77fe700 1 -- 172.21.15.45:0/2425484318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc85201f0 con 0x55dfc877a830 2023-10-17T01:20:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.735+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc879ad30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8000f80 tx=0x7f6dc8015ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c5935f618677df5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.736+0000 7f6dce7fc700 1 -- 172.21.15.45:0/2425484318 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8019040 con 0x55dfc877a830 2023-10-17T01:20:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.736+0000 7f6dce7fc700 1 -- 172.21.15.45:0/2425484318 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc802a070 con 0x55dfc877a830 2023-10-17T01:20:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.736+0000 7f6dce7fc700 1 -- 172.21.15.45:0/2425484318 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8026b10 con 0x55dfc877a830 2023-10-17T01:20:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.737+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2425484318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 msgr2=0x55dfc879ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.737+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc879ad30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8000f80 tx=0x7f6dc8015ab0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2425484318 shutdown_connections 2023-10-17T01:20:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877bd90 0x55dfc8774a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877a830 0x55dfc879ad30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/2425484318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877b190 0x55dfc8783060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2425484318 >> 172.21.15.45:0/2425484318 conn(0x55dfc86da070 msgr2=0x55dfc8771460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2425484318 shutdown_connections 2023-10-17T01:20:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.738+0000 7f6df19b3b80 1 -- 172.21.15.45:0/2425484318 wait complete. 2023-10-17T01:20:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.739+0000 7f6df19b3b80 1 -- start start 2023-10-17T01:20:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.739+0000 7f6df19b3b80 1 Processor -- start 2023-10-17T01:20:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 0x55dfc87897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877b190 0x55dfc87e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc87e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6dd77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 0x55dfc87897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6dd77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 0x55dfc87897d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:39410/0 (socket says 172.21.15.45:39410) 2023-10-17T01:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6dd77fe700 1 -- 172.21.15.45:0/579244827 learned_addr learned my addr 172.21.15.45:0/579244827 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc87e9000 con 0x55dfc877bd90 2023-10-17T01:20:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc87e9170 con 0x55dfc877a830 2023-10-17T01:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc87e9330 con 0x55dfc877b190 2023-10-17T01:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6df19b3b80 10 client.0 _ll_register_callbacks cb 0x7f6dd0098e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6dcffff700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877b190 0x55dfc87e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.740+0000 7f6dd7fff700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc87e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6dd77fe700 1 -- 172.21.15.45:0/579244827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877b190 msgr2=0x55dfc87e62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877b190 0x55dfc87e62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6dd77fe700 1 -- 172.21.15.45:0/579244827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 msgr2=0x55dfc87e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc87e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6dd77fe700 1 -- 172.21.15.45:0/579244827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfc84f9d80 con 0x55dfc877a830 2023-10-17T01:20:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6db6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.741+0000 7f6dd77fe700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 0x55dfc87897d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009550 tx=0x7f6dc8015940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80aaab7a8717906f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.742+0000 7f6db6ffd700 1 -- 172.21.15.45:0/579244827 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8019050 con 0x55dfc877a830 2023-10-17T01:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.742+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dc800f050 con 0x55dfc877a830 2023-10-17T01:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.742+0000 7f6db6ffd700 1 -- 172.21.15.45:0/579244827 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6dc8026320 con 0x55dfc877a830 2023-10-17T01:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.742+0000 7f6db6ffd700 1 -- 172.21.15.45:0/579244827 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc8015d20 con 0x55dfc877a830 2023-10-17T01:20:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.742+0000 7f6df19b3b80 10 client.26057 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.742+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55dfc84f8250 con 0x55dfc877a830 2023-10-17T01:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6db6ffd700 1 -- 172.21.15.45:0/579244827 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc800f050 con 0x55dfc877a830 2023-10-17T01:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6db6ffd700 1 -- 172.21.15.45:0/579244827 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f6dc802fa10 con 0x55dfc877a830 2023-10-17T01:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6df19b3b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6df19b3b80 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6df19b3b80 20 client.26057 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc87ea910 0x55dfc87ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6df19b3b80 4 client.26057 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:20:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.744+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dfc86c4ce0 con 0x55dfc87ea910 2023-10-17T01:20:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.745+0000 7f6dcffff700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc87ea910 0x55dfc87ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.746+0000 7f6db6ffd700 10 client.26057 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.746+0000 7f6dcffff700 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc87ea910 0x55dfc87ecd30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f545095a39c1d162 server_cookie=231850e4166cb534 in_seq=0 out_seq=0 2023-10-17T01:20:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.766+0000 7f6db6ffd700 1 -- 172.21.15.45:0/579244827 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dfc86c4ce0 con 0x55dfc87ea910 2023-10-17T01:20:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.766+0000 7f6db6ffd700 10 client.26057 handle_command_reply: tid=0 2023-10-17T01:20:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.767+0000 7f6df19b3b80 1 client.26057 shutdown 2023-10-17T01:20:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 msgr2=0x55dfc87897d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 0x55dfc87897d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009550 tx=0x7f6dc8015940 comp rx=0 tx=0).stop 2023-10-17T01:20:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 shutdown_connections 2023-10-17T01:20:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfc877bd90 0x55dfc87e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55dfc87ea910 0x55dfc87ecd30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dfc877a830 0x55dfc87897d0 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009550 tx=0x7f6dc8015940 comp rx=0 tx=0).stop 2023-10-17T01:20:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 --2- 172.21.15.45:0/579244827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dfc877b190 0x55dfc87e62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 >> 172.21.15.45:0/579244827 conn(0x55dfc86da070 msgr2=0x55dfc8783060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.768+0000 7f6db6ffd700 0 client.26057 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.769+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 shutdown_connections 2023-10-17T01:20:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.769+0000 7f6df19b3b80 1 -- 172.21.15.45:0/579244827 wait complete. 2023-10-17T01:20:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.769+0000 7f6df19b3b80 20 client.26057 trim_cache size 0 max 16384 2023-10-17T01:20:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:20:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:20:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:20:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:20:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6dc40a0b20 msgr2=0x7f6dc40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6dc40a0b20 0x7f6dc40a2fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b680 tx=0x7f6dc0005ad0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 msgr2=0x7f6dd819a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 0x7f6dd819a120 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b550 tx=0x7f6dd000b910 comp rx=0 tx=0).stop 2023-10-17T01:20:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 shutdown_connections 2023-10-17T01:20:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f6dc40a0b20 0x7f6dc40a2fe0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dd8102940 0x7f6dd819a120 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd8100db0 0x7f6dd8198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 --2- 172.21.15.45:0/713050637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6dd8101fa0 0x7f6dd8199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 >> 172.21.15.45:0/713050637 conn(0x7f6dd80fc300 msgr2=0x7f6dd8107880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 shutdown_connections 2023-10-17T01:20:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:11.770+0000 7f6dddfff700 1 -- 172.21.15.45:0/713050637 wait complete. 2023-10-17T01:20:11.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:11 smithi181 ceph-mon[134520]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 767 op/s 2023-10-17T01:20:12.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:11 smithi045 ceph-mon[98474]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 767 op/s 2023-10-17T01:20:12.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:11 smithi060 ceph-mon[134136]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 767 op/s 2023-10-17T01:20:13.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:12 smithi045 ceph-mon[98474]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 737 op/s 2023-10-17T01:20:13.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:12 smithi060 ceph-mon[134136]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 737 op/s 2023-10-17T01:20:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:12 smithi181 ceph-mon[134520]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 737 op/s 2023-10-17T01:20:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 305.0 s, 1801.0 tps, lat 55.224 ms stddev 63.582 2023-10-17T01:20:15.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:15 smithi181 ceph-mon[134520]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 892 op/s 2023-10-17T01:20:16.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:15 smithi045 ceph-mon[98474]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 892 op/s 2023-10-17T01:20:16.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:15 smithi060 ceph-mon[134136]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 892 op/s 2023-10-17T01:20:17.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:17 smithi181 ceph-mon[134520]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 831 op/s 2023-10-17T01:20:18.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:17 smithi045 ceph-mon[98474]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 831 op/s 2023-10-17T01:20:18.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:17 smithi060 ceph-mon[134136]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 831 op/s 2023-10-17T01:20:19.913 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:19 smithi181 ceph-mon[134520]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 817 op/s 2023-10-17T01:20:20.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:19 smithi045 ceph-mon[98474]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 817 op/s 2023-10-17T01:20:20.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:19 smithi060 ceph-mon[134136]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 817 op/s 2023-10-17T01:20:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 310.0 s, 2115.6 tps, lat 47.571 ms stddev 57.416 2023-10-17T01:20:21.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:21 smithi181 ceph-mon[134520]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 942 op/s 2023-10-17T01:20:22.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:21 smithi045 ceph-mon[98474]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 942 op/s 2023-10-17T01:20:22.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:21 smithi060 ceph-mon[134136]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 942 op/s 2023-10-17T01:20:23.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:22 smithi045 ceph-mon[98474]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 869 op/s 2023-10-17T01:20:23.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:22 smithi060 ceph-mon[134136]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 869 op/s 2023-10-17T01:20:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:22 smithi181 ceph-mon[134520]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 869 op/s 2023-10-17T01:20:24.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:23 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:20:24.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:23 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:20:24.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:23 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:20:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.034+0000 7f938f7ff700 1 -- 172.21.15.45:0/833045854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 msgr2=0x7f938810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.034+0000 7f938f7ff700 1 --2- 172.21.15.45:0/833045854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 0x7f938810cb50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9378009990 tx=0x7f9378009ca0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.035+0000 7f938f7ff700 1 -- 172.21.15.45:0/833045854 shutdown_connections 2023-10-17T01:20:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.035+0000 7f938f7ff700 1 --2- 172.21.15.45:0/833045854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f938810d090 0x7f938810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.035+0000 7f938f7ff700 1 --2- 172.21.15.45:0/833045854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388107d50 0x7f938810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.035+0000 7f938f7ff700 1 --2- 172.21.15.45:0/833045854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 0x7f938810cb50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.035+0000 7f938f7ff700 1 -- 172.21.15.45:0/833045854 >> 172.21.15.45:0/833045854 conn(0x7f938806db80 msgr2=0x7f938806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.036+0000 7f938f7ff700 1 -- 172.21.15.45:0/833045854 shutdown_connections 2023-10-17T01:20:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.036+0000 7f938f7ff700 1 -- 172.21.15.45:0/833045854 wait complete. 2023-10-17T01:20:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.042+0000 7f938f7ff700 1 Processor -- start 2023-10-17T01:20:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.042+0000 7f938f7ff700 1 -- start start 2023-10-17T01:20:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938f7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 0x7f93881a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938f7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 0x7f93881a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938f7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810d090 0x7f93881a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938f7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a7360 con 0x7f938810d090 2023-10-17T01:20:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938f7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a74d0 con 0x7f9388107d50 2023-10-17T01:20:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938f7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a7690 con 0x7f938810a6c0 2023-10-17T01:20:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 0x7f93881a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 0x7f93881a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45858/0 (socket says 172.21.15.45:45858) 2023-10-17T01:20:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938d59b700 1 -- 172.21.15.45:0/446046927 learned_addr learned my addr 172.21.15.45:0/446046927 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.043+0000 7f938cd9a700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 0x7f93881a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.044+0000 7f938dd9c700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810d090 0x7f93881a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.044+0000 7f938d59b700 1 -- 172.21.15.45:0/446046927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 msgr2=0x7f93881a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.044+0000 7f938d59b700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 0x7f93881a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.044+0000 7f938d59b700 1 -- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810d090 msgr2=0x7f93881a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.044+0000 7f938d59b700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810d090 0x7f93881a6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.044+0000 7f938d59b700 1 -- 172.21.15.45:0/446046927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780095f0 con 0x7f9388107d50 2023-10-17T01:20:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.047+0000 7f938d59b700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 0x7f93881a5940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f938400c9b0 tx=0x7f938400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.047+0000 7f937e7fc700 1 -- 172.21.15.45:0/446046927 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384007b00 con 0x7f9388107d50 2023-10-17T01:20:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.048+0000 7f937e7fc700 1 -- 172.21.15.45:0/446046927 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9384007c60 con 0x7f9388107d50 2023-10-17T01:20:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.048+0000 7f937e7fc700 1 -- 172.21.15.45:0/446046927 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384018660 con 0x7f9388107d50 2023-10-17T01:20:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.048+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93881a7970 con 0x7f9388107d50 2023-10-17T01:20:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.048+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881ae130 con 0x7f9388107d50 2023-10-17T01:20:25.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:20:25.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:20:25.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:24 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:20:25.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:24 smithi045 ceph-mon[98474]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 942 op/s 2023-10-17T01:20:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.055+0000 7f937e7fc700 1 -- 172.21.15.45:0/446046927 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9384018880 con 0x7f9388107d50 2023-10-17T01:20:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.058+0000 7f937e7fc700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93740a0d20 0x7f93740a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.058+0000 7f937e7fc700 1 -- 172.21.15.45:0/446046927 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93840cc130 con 0x7f9388107d50 2023-10-17T01:20:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.060+0000 7f938cd9a700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93740a0d20 0x7f93740a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.060+0000 7f93a3134b80 1 Processor -- start 2023-10-17T01:20:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 -- start start 2023-10-17T01:20:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e08b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e080a9b0 con 0x55b6e08b3190 2023-10-17T01:20:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e0808370 con 0x55b6e08b2830 2023-10-17T01:20:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f93a3134b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e0803c60 con 0x55b6e08b3d90 2023-10-17T01:20:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.061+0000 7f938d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.062+0000 7f938d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45862/0 (socket says 172.21.15.45:45862) 2023-10-17T01:20:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.062+0000 7f938d59b700 1 -- 172.21.15.45:0/3758294850 learned_addr learned my addr 172.21.15.45:0/3758294850 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.062+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e08b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.062+0000 7f938dd9c700 1 -- 172.21.15.45:0/3758294850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 msgr2=0x55b6e08b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.062+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.062+0000 7f938dd9c700 1 -- 172.21.15.45:0/3758294850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 msgr2=0x55b6e08b3790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:20:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.063+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.063+0000 7f938dd9c700 1 -- 172.21.15.45:0/3758294850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e0631d80 con 0x55b6e08b3d90 2023-10-17T01:20:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.063+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e08b4210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9380009ea0 tx=0x7f9380009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9729ad620600574 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.064+0000 7f938cd9a700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93740a0d20 0x7f93740a31e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f937800b5c0 tx=0x7f9378005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.064+0000 7f93737fe700 1 -- 172.21.15.45:0/3758294850 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380011070 con 0x55b6e08b3d90 2023-10-17T01:20:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.065+0000 7f93737fe700 1 -- 172.21.15.45:0/3758294850 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938000c0d0 con 0x55b6e08b3d90 2023-10-17T01:20:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.065+0000 7f93737fe700 1 -- 172.21.15.45:0/3758294850 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380015cb0 con 0x55b6e08b3d90 2023-10-17T01:20:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.066+0000 7f93a3134b80 1 -- 172.21.15.45:0/3758294850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 msgr2=0x55b6e08b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.066+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e08b4210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9380009ea0 tx=0x7f9380009ed0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.067+0000 7f93a3134b80 1 -- 172.21.15.45:0/3758294850 shutdown_connections 2023-10-17T01:20:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.067+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e08b4210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.067+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.067+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3758294850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.067+0000 7f93a3134b80 1 -- 172.21.15.45:0/3758294850 >> 172.21.15.45:0/3758294850 conn(0x55b6e0812070 msgr2=0x55b6e08ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.068+0000 7f93a3134b80 1 -- 172.21.15.45:0/3758294850 shutdown_connections 2023-10-17T01:20:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.068+0000 7f93a3134b80 1 -- 172.21.15.45:0/3758294850 wait complete. 2023-10-17T01:20:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.069+0000 7f93a3134b80 1 -- start start 2023-10-17T01:20:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.070+0000 7f93a3134b80 1 Processor -- start 2023-10-17T01:20:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3190 0x55b6e08bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 0x55b6e08c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e07a04b0 con 0x55b6e08b3d90 2023-10-17T01:20:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e07a0150 con 0x55b6e08b3190 2023-10-17T01:20:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e06878c0 con 0x55b6e08b2830 2023-10-17T01:20:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.076+0000 7f93a3134b80 10 client.0 _ll_register_callbacks cb 0x55b6e06eda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:25.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:20:25.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:20:25.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:24 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:20:25.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:24 smithi060 ceph-mon[134136]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 942 op/s 2023-10-17T01:20:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.078+0000 7f938cd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3190 0x55b6e08bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.078+0000 7f938dd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 0x55b6e08c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.078+0000 7f938d59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.078+0000 7f938dd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 0x55b6e08c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48264/0 (socket says 172.21.15.45:48264) 2023-10-17T01:20:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.078+0000 7f938dd9c700 1 -- 172.21.15.45:0/3707685356 learned_addr learned my addr 172.21.15.45:0/3707685356 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f938dd9c700 1 -- 172.21.15.45:0/3707685356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 msgr2=0x55b6e08bc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08bc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f938dd9c700 1 -- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3190 msgr2=0x55b6e08bc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3190 0x55b6e08bc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f938dd9c700 1 -- 172.21.15.45:0/3707685356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9380009b50 con 0x55b6e08b3d90 2023-10-17T01:20:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f938dd9c700 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 0x55b6e08c3940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93800095b0 tx=0x7f9380003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaa78b8c310f5fb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.079+0000 7f9370ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:20:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.080+0000 7f9370ff9700 1 -- 172.21.15.45:0/3707685356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380011070 con 0x55b6e08b3d90 2023-10-17T01:20:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.080+0000 7f9370ff9700 1 -- 172.21.15.45:0/3707685356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93800159b0 con 0x55b6e08b3d90 2023-10-17T01:20:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.080+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6e082ecb0 con 0x55b6e08b3d90 2023-10-17T01:20:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f9370ff9700 1 -- 172.21.15.45:0/3707685356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938000ce40 con 0x55b6e08b3d90 2023-10-17T01:20:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f93a3134b80 10 client.15819 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b6e061ac00 con 0x55b6e08b3d90 2023-10-17T01:20:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f9370ff9700 1 -- 172.21.15.45:0/3707685356 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6e082ecb0 con 0x55b6e08b3d90 2023-10-17T01:20:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f9370ff9700 1 -- 172.21.15.45:0/3707685356 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f938001e420 con 0x55b6e08b3d90 2023-10-17T01:20:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f93a3134b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.082+0000 7f93a3134b80 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.083+0000 7f93a3134b80 20 client.15819 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.083+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e08c8140 0x55b6e08ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.083+0000 7f93a3134b80 4 client.15819 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:20:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.083+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6e0881cd0 con 0x55b6e08c8140 2023-10-17T01:20:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.084+0000 7f938d59b700 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e08c8140 0x55b6e08ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.085+0000 7f938d59b700 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e08c8140 0x55b6e08ca560 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e320c5e0ecbecf81 server_cookie=242e06ef82db3824 in_seq=0 out_seq=0 2023-10-17T01:20:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.085+0000 7f9370ff9700 10 client.15819 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.086+0000 7f9370ff9700 1 -- 172.21.15.45:0/3707685356 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55b6e0881cd0 con 0x55b6e08c8140 2023-10-17T01:20:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.086+0000 7f9370ff9700 10 client.15819 handle_command_reply: tid=0 2023-10-17T01:20:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.086+0000 7f93a3134b80 1 client.15819 shutdown 2023-10-17T01:20:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 msgr2=0x55b6e08c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 0x55b6e08c3940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93800095b0 tx=0x7f9380003d40 comp rx=0 tx=0).stop 2023-10-17T01:20:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 shutdown_connections 2023-10-17T01:20:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3d90 0x55b6e08c3940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08bc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e08c8140 0x55b6e08ca560 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 --2- 172.21.15.45:0/3707685356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3190 0x55b6e08bc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 >> 172.21.15.45:0/3707685356 conn(0x55b6e0812070 msgr2=0x55b6e08aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f9370ff9700 0 client.15819 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 shutdown_connections 2023-10-17T01:20:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.088+0000 7f93a3134b80 1 -- 172.21.15.45:0/3707685356 wait complete. 2023-10-17T01:20:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.089+0000 7f93a3134b80 20 client.15819 trim_cache size 0 max 16384 2023-10-17T01:20:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.119+0000 7f93a3134b80 1 Processor -- start 2023-10-17T01:20:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 -- start start 2023-10-17T01:20:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08ca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 0x55b6e08b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e08baec0 con 0x55b6e08b3190 2023-10-17T01:20:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e08bb8a0 con 0x55b6e08b3d90 2023-10-17T01:20:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.120+0000 7f93a3134b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e08ae5b0 con 0x55b6e08b2830 2023-10-17T01:20:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.121+0000 7f938dd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 0x55b6e08b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.121+0000 7f938cd9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.121+0000 7f938dd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 0x55b6e08b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45874/0 (socket says 172.21.15.45:45874) 2023-10-17T01:20:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.121+0000 7f938dd9c700 1 -- 172.21.15.45:0/1993941409 learned_addr learned my addr 172.21.15.45:0/1993941409 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.123+0000 7f938dd9c700 1 -- 172.21.15.45:0/1993941409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 msgr2=0x55b6e08ca530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:20:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.123+0000 7f938dd9c700 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08ca530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.124+0000 7f938dd9c700 1 -- 172.21.15.45:0/1993941409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 msgr2=0x55b6e08ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.124+0000 7f938dd9c700 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.124+0000 7f938dd9c700 1 -- 172.21.15.45:0/1993941409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e06581f0 con 0x55b6e08b3d90 2023-10-17T01:20:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.129+0000 7f938dd9c700 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 0x55b6e08b6d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9380009870 tx=0x7f9380009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ec89c6f8e265a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.129+0000 7f9373fff700 1 -- 172.21.15.45:0/1993941409 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380011040 con 0x55b6e08b3d90 2023-10-17T01:20:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.130+0000 7f9373fff700 1 -- 172.21.15.45:0/1993941409 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9380012040 con 0x55b6e08b3d90 2023-10-17T01:20:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.130+0000 7f9373fff700 1 -- 172.21.15.45:0/1993941409 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380026770 con 0x55b6e08b3d90 2023-10-17T01:20:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 -- 172.21.15.45:0/1993941409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 msgr2=0x55b6e08b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 0x55b6e08b6d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9380009870 tx=0x7f9380009e20 comp rx=0 tx=0).stop 2023-10-17T01:20:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 -- 172.21.15.45:0/1993941409 shutdown_connections 2023-10-17T01:20:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b3d90 0x55b6e08b6d20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b2830 0x55b6e08ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 --2- 172.21.15.45:0/1993941409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e08ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.131+0000 7f93a3134b80 1 -- 172.21.15.45:0/1993941409 >> 172.21.15.45:0/1993941409 conn(0x55b6e0812070 msgr2=0x55b6e08bb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.132+0000 7f93a3134b80 1 -- 172.21.15.45:0/1993941409 shutdown_connections 2023-10-17T01:20:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.132+0000 7f93a3134b80 1 -- 172.21.15.45:0/1993941409 wait complete. 2023-10-17T01:20:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.132+0000 7f93a3134b80 1 -- start start 2023-10-17T01:20:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 Processor -- start 2023-10-17T01:20:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e0915a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e0918040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e0918750 con 0x55b6e08b3190 2023-10-17T01:20:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e09188c0 con 0x55b6e08b2830 2023-10-17T01:20:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e0918a80 con 0x55b6e08b3d90 2023-10-17T01:20:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.133+0000 7f93a3134b80 10 client.0 _ll_register_callbacks cb 0x7f9380018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.134+0000 7f938d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.134+0000 7f938d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45890/0 (socket says 172.21.15.45:45890) 2023-10-17T01:20:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.134+0000 7f938d59b700 1 -- 172.21.15.45:0/444505535 learned_addr learned my addr 172.21.15.45:0/444505535 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.135+0000 7f938cd9a700 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e0915a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.136+0000 7f938d59b700 1 -- 172.21.15.45:0/444505535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 msgr2=0x55b6e0918040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.136+0000 7f938d59b700 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e0918040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.136+0000 7f938d59b700 1 -- 172.21.15.45:0/444505535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 msgr2=0x55b6e0915a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.136+0000 7f938d59b700 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e0915a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.136+0000 7f938d59b700 1 -- 172.21.15.45:0/444505535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6e061ac00 con 0x55b6e08b2830 2023-10-17T01:20:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.137+0000 7f938d59b700 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08c1810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9384007fd0 tx=0x7f9384099640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a667df83c1699e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.137+0000 7f9370ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:20:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.138+0000 7f9370ff9700 1 -- 172.21.15.45:0/444505535 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938409d580 con 0x55b6e08b2830 2023-10-17T01:20:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.138+0000 7f9370ff9700 1 -- 172.21.15.45:0/444505535 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93840a17c0 con 0x55b6e08b2830 2023-10-17T01:20:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.139+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f938000c0d0 con 0x55b6e08b2830 2023-10-17T01:20:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.139+0000 7f9370ff9700 1 -- 172.21.15.45:0/444505535 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938409d120 con 0x55b6e08b2830 2023-10-17T01:20:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.139+0000 7f93a3134b80 10 client.26072 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.139+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b6e0630250 con 0x55b6e08b2830 2023-10-17T01:20:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.139+0000 7f9370ff9700 1 -- 172.21.15.45:0/444505535 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938000c0d0 con 0x55b6e08b2830 2023-10-17T01:20:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.139+0000 7f9370ff9700 1 -- 172.21.15.45:0/444505535 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f93840b7020 con 0x55b6e08b2830 2023-10-17T01:20:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.140+0000 7f93a3134b80 10 client.26072 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.140+0000 7f93a3134b80 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.140+0000 7f93a3134b80 20 client.26072 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.140+0000 7f93a3134b80 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e091a3d0 0x55b6e091c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.140+0000 7f93a3134b80 4 client.26072 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:20:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.140+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6e07fcce0 con 0x55b6e091a3d0 2023-10-17T01:20:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.141+0000 7f938cd9a700 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e091a3d0 0x55b6e091c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.143+0000 7f9370ff9700 10 client.26072 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.143+0000 7f938cd9a700 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e091a3d0 0x55b6e091c7f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dad3b2659587ec8 server_cookie=58ef67b73748435e in_seq=0 out_seq=0 2023-10-17T01:20:25.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:20:25.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:20:25.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:24 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:20:25.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:24 smithi181 ceph-mon[134520]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 942 op/s 2023-10-17T01:20:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f9370ff9700 1 -- 172.21.15.45:0/444505535 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6e07fcce0 con 0x55b6e091a3d0 2023-10-17T01:20:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f9370ff9700 10 client.26072 handle_command_reply: tid=0 2023-10-17T01:20:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 client.26072 shutdown 2023-10-17T01:20:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 msgr2=0x55b6e08c1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08c1810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9384007fd0 tx=0x7f9384099640 comp rx=0 tx=0).stop 2023-10-17T01:20:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 shutdown_connections 2023-10-17T01:20:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e08b3d90 0x55b6e0918040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e091a3d0 0x55b6e091c7f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e08b2830 0x55b6e08c1810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 --2- 172.21.15.45:0/444505535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e08b3190 0x55b6e0915a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 >> 172.21.15.45:0/444505535 conn(0x55b6e0812070 msgr2=0x55b6e08ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f9370ff9700 0 client.26072 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 shutdown_connections 2023-10-17T01:20:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.194+0000 7f93a3134b80 1 -- 172.21.15.45:0/444505535 wait complete. 2023-10-17T01:20:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.195+0000 7f93a3134b80 20 client.26072 trim_cache size 0 max 16384 2023-10-17T01:20:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:20:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:20:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:20:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:20:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.195+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93740a0d20 msgr2=0x7f93740a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.195+0000 7f938f7ff700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93740a0d20 0x7f93740a31e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f937800b5c0 tx=0x7f9378005fb0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.195+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 msgr2=0x7f93881a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.195+0000 7f938f7ff700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 0x7f93881a5940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f938400c9b0 tx=0x7f938400cd70 comp rx=0 tx=0).stop 2023-10-17T01:20:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 shutdown_connections 2023-10-17T01:20:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93740a0d20 0x7f93740a31e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810d090 0x7f93881a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9388107d50 0x7f93881a5940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 --2- 172.21.15.45:0/446046927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f938810a6c0 0x7f93881a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 >> 172.21.15.45:0/446046927 conn(0x7f938806db80 msgr2=0x7f9388109520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 shutdown_connections 2023-10-17T01:20:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:25.196+0000 7f938f7ff700 1 -- 172.21.15.45:0/446046927 wait complete. 2023-10-17T01:20:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 315.0 s, 1555.1 tps, lat 59.565 ms stddev 78.217 2023-10-17T01:20:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:26 smithi045 ceph-mon[98474]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 917 op/s 2023-10-17T01:20:27.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:26 smithi060 ceph-mon[134136]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 917 op/s 2023-10-17T01:20:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:26 smithi181 ceph-mon[134520]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 917 op/s 2023-10-17T01:20:29.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:28 smithi045 ceph-mon[98474]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2023-10-17T01:20:29.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:28 smithi060 ceph-mon[134136]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2023-10-17T01:20:29.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:28 smithi181 ceph-mon[134520]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2023-10-17T01:20:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 320.0 s, 1754.3 tps, lat 60.821 ms stddev 103.464 2023-10-17T01:20:31.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:20:31.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:31 smithi181 ceph-mon[134520]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 1.92k op/s 2023-10-17T01:20:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.035+0000 7f5322fff700 1 -- 172.21.15.45:0/3445217132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 msgr2=0x7f531c106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.035+0000 7f5322fff700 1 --2- 172.21.15.45:0/3445217132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c106550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f530c009990 tx=0x7f530c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.036+0000 7f5322fff700 1 -- 172.21.15.45:0/3445217132 shutdown_connections 2023-10-17T01:20:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.036+0000 7f5322fff700 1 --2- 172.21.15.45:0/3445217132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c106a90 0x7f531c108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.036+0000 7f5322fff700 1 --2- 172.21.15.45:0/3445217132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 0x7f531c103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.036+0000 7f5322fff700 1 --2- 172.21.15.45:0/3445217132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c106550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.036+0000 7f5322fff700 1 -- 172.21.15.45:0/3445217132 >> 172.21.15.45:0/3445217132 conn(0x7f531c0fb220 msgr2=0x7f531c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:32.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.036+0000 7f5322fff700 1 -- 172.21.15.45:0/3445217132 shutdown_connections 2023-10-17T01:20:32.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.039+0000 7f5322fff700 1 -- 172.21.15.45:0/3445217132 wait complete. 2023-10-17T01:20:32.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.040+0000 7f5322fff700 1 Processor -- start 2023-10-17T01:20:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.046+0000 7f5322fff700 1 -- start start 2023-10-17T01:20:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5322fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 0x7f531c194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5322fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5322fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c106a90 0x7f531c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5322fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19c3b0 con 0x7f531c101830 2023-10-17T01:20:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5322fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19c4f0 con 0x7f531c104160 2023-10-17T01:20:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5322fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c19c680 con 0x7f531c106a90 2023-10-17T01:20:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5313fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5313fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c194d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58814/0 (socket says 172.21.15.45:58814) 2023-10-17T01:20:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.047+0000 7f5313fff700 1 -- 172.21.15.45:0/3843764570 learned_addr learned my addr 172.21.15.45:0/3843764570 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.049+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 0x7f531c194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.050+0000 7f5320d9b700 1 -- 172.21.15.45:0/3843764570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c106a90 msgr2=0x7f531c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.050+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c106a90 0x7f531c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.050+0000 7f5320d9b700 1 -- 172.21.15.45:0/3843764570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 msgr2=0x7f531c194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.050+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.050+0000 7f5320d9b700 1 -- 172.21.15.45:0/3843764570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009520 con 0x7f531c101830 2023-10-17T01:20:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.051+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 0x7f531c194830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f531800e9f0 tx=0x7f531800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:32.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:31 smithi045 ceph-mon[98474]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 1.92k op/s 2023-10-17T01:20:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.052+0000 7f5311ffb700 1 -- 172.21.15.45:0/3843764570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531800cb70 con 0x7f531c101830 2023-10-17T01:20:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.052+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5314009c40 con 0x7f531c101830 2023-10-17T01:20:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.052+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c0095f0 con 0x7f531c101830 2023-10-17T01:20:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.053+0000 7f5311ffb700 1 -- 172.21.15.45:0/3843764570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53180044e0 con 0x7f531c101830 2023-10-17T01:20:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.053+0000 7f5311ffb700 1 -- 172.21.15.45:0/3843764570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318003e90 con 0x7f531c101830 2023-10-17T01:20:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.060+0000 7f5311ffb700 1 -- 172.21.15.45:0/3843764570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f5318004020 con 0x7f531c101830 2023-10-17T01:20:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.062+0000 7f5336899b80 1 Processor -- start 2023-10-17T01:20:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.064+0000 7f5311ffb700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f53040a4fb0 0x7f53040a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.064+0000 7f5336899b80 1 -- start start 2023-10-17T01:20:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.064+0000 7f5311ffb700 1 -- 172.21.15.45:0/3843764570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5318014070 con 0x7f531c101830 2023-10-17T01:20:32.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.065+0000 7f5336899b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f6039a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.065+0000 7f532159c700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f53040a4fb0 0x7f53040a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.065+0000 7f5336899b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 0x560f6039b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.065+0000 7f5336899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f6039bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.065+0000 7f5336899b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f602e9a10 con 0x560f6039ad10 2023-10-17T01:20:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.065+0000 7f5336899b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f602e73d0 con 0x560f6039a3b0 2023-10-17T01:20:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5336899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f602e2cc0 con 0x560f6039bac0 2023-10-17T01:20:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f6039a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f6039a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58826/0 (socket says 172.21.15.45:58826) 2023-10-17T01:20:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 -- 172.21.15.45:0/3750933157 learned_addr learned my addr 172.21.15.45:0/3750933157 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f532159c700 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f6039bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 -- 172.21.15.45:0/3750933157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 msgr2=0x560f6039bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f6039bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 -- 172.21.15.45:0/3750933157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 msgr2=0x560f6039b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 0x560f6039b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.066+0000 7f5320d9b700 1 -- 172.21.15.45:0/3750933157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f60110de0 con 0x560f6039a3b0 2023-10-17T01:20:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.067+0000 7f5320d9b700 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f6039a7d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53180a1780 tx=0x7f5318099620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30aacc1ff7d41678 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.068+0000 7f5302ffd700 1 -- 172.21.15.45:0/3750933157 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180b2220 con 0x560f6039a3b0 2023-10-17T01:20:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.068+0000 7f5302ffd700 1 -- 172.21.15.45:0/3750933157 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531809da70 con 0x560f6039a3b0 2023-10-17T01:20:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.069+0000 7f5302ffd700 1 -- 172.21.15.45:0/3750933157 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531809a100 con 0x560f6039a3b0 2023-10-17T01:20:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.070+0000 7f532159c700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f53040a4fb0 0x7f53040a7470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5314009fd0 tx=0x7f5314009f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.070+0000 7f5336899b80 1 -- 172.21.15.45:0/3750933157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 msgr2=0x560f6039a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.070+0000 7f5336899b80 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f6039a7d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53180a1780 tx=0x7f5318099620 comp rx=0 tx=0).stop 2023-10-17T01:20:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.071+0000 7f5336899b80 1 -- 172.21.15.45:0/3750933157 shutdown_connections 2023-10-17T01:20:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.071+0000 7f5336899b80 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f6039bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.071+0000 7f5336899b80 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f6039a7d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.071+0000 7f5336899b80 1 --2- 172.21.15.45:0/3750933157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 0x560f6039b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.071+0000 7f5336899b80 1 -- 172.21.15.45:0/3750933157 >> 172.21.15.45:0/3750933157 conn(0x560f602f10d0 msgr2=0x560f6038c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:32.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.071+0000 7f5336899b80 1 -- 172.21.15.45:0/3750933157 shutdown_connections 2023-10-17T01:20:32.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.072+0000 7f5336899b80 1 -- 172.21.15.45:0/3750933157 wait complete. 2023-10-17T01:20:32.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.073+0000 7f5336899b80 1 -- start start 2023-10-17T01:20:32.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.074+0000 7f5336899b80 1 Processor -- start 2023-10-17T01:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f603952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 0x560f60395820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60398840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6027f510 con 0x560f6039ad10 2023-10-17T01:20:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6027f1b0 con 0x560f6039a3b0 2023-10-17T01:20:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f60166920 con 0x560f6039bac0 2023-10-17T01:20:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5336899b80 10 client.0 _ll_register_callbacks cb 0x560f601cca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f532159c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60398840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5320d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f603952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f532159c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60398840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:52628/0 (socket says 172.21.15.45:52628) 2023-10-17T01:20:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f532159c700 1 -- 172.21.15.45:0/773528306 learned_addr learned my addr 172.21.15.45:0/773528306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.076+0000 7f5320d9b700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f603952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58832/0 (socket says 172.21.15.45:58832) 2023-10-17T01:20:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.077+0000 7f532159c700 1 -- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 msgr2=0x560f603952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.077+0000 7f532159c700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f603952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.077+0000 7f532159c700 1 -- 172.21.15.45:0/773528306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 msgr2=0x560f60395820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:20:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.077+0000 7f532159c700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 0x560f60395820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.077+0000 7f532159c700 1 -- 172.21.15.45:0/773528306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c0095f0 con 0x560f6039bac0 2023-10-17T01:20:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.077+0000 7f5320d9b700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f603952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:20:32.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:31 smithi060 ceph-mon[134136]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 37 MiB/s wr, 1.92k op/s 2023-10-17T01:20:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.079+0000 7f532159c700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60398840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5314015b80 tx=0x7f531400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4b7a4f6d69c0cf server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.079+0000 7f52fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:20:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.080+0000 7f52fbfff700 1 -- 172.21.15.45:0/773528306 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400e030 con 0x560f6039bac0 2023-10-17T01:20:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.081+0000 7f52fbfff700 1 -- 172.21.15.45:0/773528306 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f531400f070 con 0x560f6039bac0 2023-10-17T01:20:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.081+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f6030dd10 con 0x560f6039bac0 2023-10-17T01:20:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.081+0000 7f52fbfff700 1 -- 172.21.15.45:0/773528306 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5314018a20 con 0x560f6039bac0 2023-10-17T01:20:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.081+0000 7f5336899b80 10 client.26059 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.081+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560f600f9c60 con 0x560f6039bac0 2023-10-17T01:20:32.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.082+0000 7f52fbfff700 1 -- 172.21.15.45:0/773528306 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f6030dd10 con 0x560f6039bac0 2023-10-17T01:20:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.082+0000 7f52fbfff700 1 -- 172.21.15.45:0/773528306 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5314021070 con 0x560f6039bac0 2023-10-17T01:20:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.083+0000 7f5336899b80 10 client.26059 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.083+0000 7f5336899b80 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.083+0000 7f5336899b80 20 client.26059 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.083+0000 7f5336899b80 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f603b8800 0x560f603bac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.083+0000 7f5336899b80 4 client.26059 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:20:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.083+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f60360d30 con 0x560f603b8800 2023-10-17T01:20:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.084+0000 7f5320d9b700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f603b8800 0x560f603bac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.085+0000 7f5320d9b700 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f603b8800 0x560f603bac20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b32883f38554500 server_cookie=756d3a6456760f3f in_seq=0 out_seq=0 2023-10-17T01:20:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.085+0000 7f52fbfff700 10 client.26059 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.086+0000 7f52fbfff700 1 -- 172.21.15.45:0/773528306 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x560f60360d30 con 0x560f603b8800 2023-10-17T01:20:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.086+0000 7f52fbfff700 10 client.26059 handle_command_reply: tid=0 2023-10-17T01:20:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.086+0000 7f5336899b80 1 client.26059 shutdown 2023-10-17T01:20:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.086+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 msgr2=0x560f60398840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.086+0000 7f5336899b80 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60398840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5314015b80 tx=0x7f531400b040 comp rx=0 tx=0).stop 2023-10-17T01:20:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 shutdown_connections 2023-10-17T01:20:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60398840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f603b8800 0x560f603bac20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f603952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 --2- 172.21.15.45:0/773528306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039ad10 0x560f60395820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f52fbfff700 0 client.26059 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 >> 172.21.15.45:0/773528306 conn(0x560f602f10d0 msgr2=0x560f6038c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 shutdown_connections 2023-10-17T01:20:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 1 -- 172.21.15.45:0/773528306 wait complete. 2023-10-17T01:20:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.087+0000 7f5336899b80 20 client.26059 trim_cache size 0 max 16384 2023-10-17T01:20:32.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 Processor -- start 2023-10-17T01:20:32.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 -- start start 2023-10-17T01:20:32.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039a3b0 0x560f60393f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 0x560f603952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60395820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f60393af0 con 0x560f6039a3b0 2023-10-17T01:20:32.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f60389420 con 0x560f6039ad10 2023-10-17T01:20:32.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.114+0000 7f5336899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6038d6f0 con 0x560f6039bac0 2023-10-17T01:20:32.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.115+0000 7f5313fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 0x560f603952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.115+0000 7f532159c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60395820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.115+0000 7f5313fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 0x560f603952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58840/0 (socket says 172.21.15.45:58840) 2023-10-17T01:20:32.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.115+0000 7f5313fff700 1 -- 172.21.15.45:0/2943980200 learned_addr learned my addr 172.21.15.45:0/2943980200 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:32.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.115+0000 7f5320d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039a3b0 0x560f60393f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f532159c700 1 -- 172.21.15.45:0/2943980200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 msgr2=0x560f603952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f532159c700 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 0x560f603952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f532159c700 1 -- 172.21.15.45:0/2943980200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039a3b0 msgr2=0x560f60393f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f532159c700 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039a3b0 0x560f60393f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f532159c700 1 -- 172.21.15.45:0/2943980200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f60137250 con 0x560f6039bac0 2023-10-17T01:20:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f5313fff700 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 0x560f603952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:20:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f532159c700 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60395820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5314022d50 tx=0x7f53140096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0ebd05dcfe19bc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.116+0000 7f53037fe700 1 -- 172.21.15.45:0/2943980200 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314017360 con 0x560f6039bac0 2023-10-17T01:20:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.117+0000 7f53037fe700 1 -- 172.21.15.45:0/2943980200 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531401e040 con 0x560f6039bac0 2023-10-17T01:20:32.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.117+0000 7f53037fe700 1 -- 172.21.15.45:0/2943980200 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314022a40 con 0x560f6039bac0 2023-10-17T01:20:32.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 -- 172.21.15.45:0/2943980200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 msgr2=0x560f60395820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60395820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5314022d50 tx=0x7f53140096a0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 -- 172.21.15.45:0/2943980200 shutdown_connections 2023-10-17T01:20:32.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039bac0 0x560f60395820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039a3b0 0x560f60393f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 --2- 172.21.15.45:0/2943980200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039ad10 0x560f603952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.118+0000 7f5336899b80 1 -- 172.21.15.45:0/2943980200 >> 172.21.15.45:0/2943980200 conn(0x560f602f10d0 msgr2=0x560f603930f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:32.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.119+0000 7f5336899b80 1 -- 172.21.15.45:0/2943980200 shutdown_connections 2023-10-17T01:20:32.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.119+0000 7f5336899b80 1 -- 172.21.15.45:0/2943980200 wait complete. 2023-10-17T01:20:32.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.119+0000 7f5336899b80 1 -- start start 2023-10-17T01:20:32.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 Processor -- start 2023-10-17T01:20:32.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f604028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039ad10 0x560f60402de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039bac0 0x560f60409960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f60409ea0 con 0x560f6039bac0 2023-10-17T01:20:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f60409fe0 con 0x560f6039a3b0 2023-10-17T01:20:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6040a1a0 con 0x560f6039ad10 2023-10-17T01:20:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.120+0000 7f5336899b80 10 client.0 _ll_register_callbacks cb 0x7f5314010740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5320d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f604028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039ad10 0x560f60402de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5320d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f604028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58856/0 (socket says 172.21.15.45:58856) 2023-10-17T01:20:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5320d9b700 1 -- 172.21.15.45:0/97898882 learned_addr learned my addr 172.21.15.45:0/97898882 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f532159c700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039bac0 0x560f60409960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 -- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 msgr2=0x560f604028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f604028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 -- 172.21.15.45:0/97898882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039bac0 msgr2=0x560f60409960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039bac0 0x560f60409960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 -- 172.21.15.45:0/97898882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c109af0 con 0x560f6039ad10 2023-10-17T01:20:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5320d9b700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f604028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:20:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f532159c700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039bac0 0x560f60409960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:20:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f5313fff700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039ad10 0x560f60402de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b2e0 tx=0x7f530c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f100dfb21cbc88cc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.121+0000 7f52fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:20:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.122+0000 7f52fbfff700 1 -- 172.21.15.45:0/97898882 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c004500 con 0x560f6039ad10 2023-10-17T01:20:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.122+0000 7f52fbfff700 1 -- 172.21.15.45:0/97898882 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f530c004660 con 0x560f6039ad10 2023-10-17T01:20:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.122+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f531809da70 con 0x560f6039ad10 2023-10-17T01:20:32.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.122+0000 7f52fbfff700 1 -- 172.21.15.45:0/97898882 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f530c004820 con 0x560f6039ad10 2023-10-17T01:20:32.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.122+0000 7f5336899b80 10 client.26065 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:32.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.122+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f5314022e70 con 0x560f6039ad10 2023-10-17T01:20:32.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.123+0000 7f52fbfff700 1 -- 172.21.15.45:0/97898882 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f530c034020 con 0x560f6039ad10 2023-10-17T01:20:32.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f52fbfff700 1 -- 172.21.15.45:0/97898882 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f530c024480 con 0x560f6039ad10 2023-10-17T01:20:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5336899b80 10 client.26065 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5336899b80 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5336899b80 20 client.26065 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5336899b80 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f6040bbf0 0x560f6040e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5336899b80 4 client.26065 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:20:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f602dbd40 con 0x560f6040bbf0 2023-10-17T01:20:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.124+0000 7f5320d9b700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f6040bbf0 0x560f6040e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5320d9b700 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f6040bbf0 0x560f6040e010 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5d66c62a9a153c server_cookie=a52f21d35df1769e in_seq=0 out_seq=0 2023-10-17T01:20:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f52fbfff700 10 client.26065 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f52fbfff700 1 -- 172.21.15.45:0/97898882 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560f602dbd40 con 0x560f6040bbf0 2023-10-17T01:20:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f52fbfff700 10 client.26065 handle_command_reply: tid=0 2023-10-17T01:20:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 client.26065 shutdown 2023-10-17T01:20:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039ad10 msgr2=0x560f60402de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039ad10 0x560f60402de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b2e0 tx=0x7f530c00bf60 comp rx=0 tx=0).stop 2023-10-17T01:20:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 shutdown_connections 2023-10-17T01:20:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560f6039bac0 0x560f60409960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560f6040bbf0 0x560f6040e010 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560f6039a3b0 0x560f604028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 --2- 172.21.15.45:0/97898882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f6039ad10 0x560f60402de0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 >> 172.21.15.45:0/97898882 conn(0x560f602f10d0 msgr2=0x560f60393f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.125+0000 7f52fbfff700 0 client.26065 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 shutdown_connections 2023-10-17T01:20:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5336899b80 1 -- 172.21.15.45:0/97898882 wait complete. 2023-10-17T01:20:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5336899b80 20 client.26065 trim_cache size 0 max 16384 2023-10-17T01:20:32.138 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:20:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:20:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:20:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:20:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:20:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:20:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:20:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:20:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:20:32.139 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:20:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f53040a4fb0 msgr2=0x7f53040a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5322fff700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f53040a4fb0 0x7f53040a7470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5314009fd0 tx=0x7f5314009f00 comp rx=0 tx=0).stop 2023-10-17T01:20:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 msgr2=0x7f531c194830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.126+0000 7f5322fff700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 0x7f531c194830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f531800e9f0 tx=0x7f531800edb0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 shutdown_connections 2023-10-17T01:20:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f53040a4fb0 0x7f53040a7470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c106a90 0x7f531c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f531c101830 0x7f531c194830 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 --2- 172.21.15.45:0/3843764570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f531c104160 0x7f531c194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 >> 172.21.15.45:0/3843764570 conn(0x7f531c0fb220 msgr2=0x7f531c1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 shutdown_connections 2023-10-17T01:20:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:20:32.127+0000 7f5322fff700 1 -- 172.21.15.45:0/3843764570 wait complete. 2023-10-17T01:20:32.156 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:20:33.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:33 smithi181 ceph-mon[134520]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.89k op/s 2023-10-17T01:20:34.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:33 smithi045 ceph-mon[98474]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.89k op/s 2023-10-17T01:20:34.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:33 smithi060 ceph-mon[134136]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.89k op/s 2023-10-17T01:20:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 325.0 s, 2393.4 tps, lat 41.609 ms stddev 48.111 2023-10-17T01:20:36.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:35 smithi045 ceph-mon[98474]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.90k op/s 2023-10-17T01:20:36.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:35 smithi060 ceph-mon[134136]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.90k op/s 2023-10-17T01:20:36.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:35 smithi181 ceph-mon[134520]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 1.90k op/s 2023-10-17T01:20:38.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:37 smithi045 ceph-mon[98474]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 1.77k op/s 2023-10-17T01:20:38.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:37 smithi060 ceph-mon[134136]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 1.77k op/s 2023-10-17T01:20:38.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:37 smithi181 ceph-mon[134520]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 1.77k op/s 2023-10-17T01:20:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.440+0000 7f454a9ff700 1 -- 172.21.15.45:0/119355842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544104140 msgr2=0x7f4544106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.440+0000 7f454a9ff700 1 --2- 172.21.15.45:0/119355842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544104140 0x7f4544106530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f452c009990 tx=0x7f452c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 -- 172.21.15.45:0/119355842 shutdown_connections 2023-10-17T01:20:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 --2- 172.21.15.45:0/119355842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544106a70 0x7f4544108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 --2- 172.21.15.45:0/119355842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4544101810 0x7f4544103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 --2- 172.21.15.45:0/119355842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544104140 0x7f4544106530 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 -- 172.21.15.45:0/119355842 >> 172.21.15.45:0/119355842 conn(0x7f45440fb1a0 msgr2=0x7f45440fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 -- 172.21.15.45:0/119355842 shutdown_connections 2023-10-17T01:20:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.441+0000 7f454a9ff700 1 -- 172.21.15.45:0/119355842 wait complete. 2023-10-17T01:20:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.443+0000 7f454a9ff700 1 Processor -- start 2023-10-17T01:20:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 -- start start 2023-10-17T01:20:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544101810 0x7f4544198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 0x7f45441992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4544106a70 0x7f454419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419a7a0 con 0x7f4544106a70 2023-10-17T01:20:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419a910 con 0x7f4544101810 2023-10-17T01:20:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.444+0000 7f454a9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454419aad0 con 0x7f4544104140 2023-10-17T01:20:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 0x7f45441992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 0x7f45441992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38220/0 (socket says 172.21.15.45:38220) 2023-10-17T01:20:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 -- 172.21.15.45:0/1395481229 learned_addr learned my addr 172.21.15.45:0/1395481229 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f4543fff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544101810 0x7f4544198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f4548f9c700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4544106a70 0x7f454419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 -- 172.21.15.45:0/1395481229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544101810 msgr2=0x7f4544198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544101810 0x7f4544198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 -- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4544106a70 msgr2=0x7f454419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4544106a70 0x7f454419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 -- 172.21.15.45:0/1395481229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0095f0 con 0x7f4544104140 2023-10-17T01:20:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.445+0000 7f45437fe700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 0x7f45441992f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b9e0 tx=0x7f452c00ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.446+0000 7f45417fa700 1 -- 172.21.15.45:0/1395481229 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c01d070 con 0x7f4544104140 2023-10-17T01:20:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.446+0000 7f45417fa700 1 -- 172.21.15.45:0/1395481229 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f452c00bed0 con 0x7f4544104140 2023-10-17T01:20:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.446+0000 7f45417fa700 1 -- 172.21.15.45:0/1395481229 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c022df0 con 0x7f4544104140 2023-10-17T01:20:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.446+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454419ad50 con 0x7f4544104140 2023-10-17T01:20:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.446+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45441a14f0 con 0x7f4544104140 2023-10-17T01:20:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.451+0000 7f45417fa700 1 -- 172.21.15.45:0/1395481229 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f452c022470 con 0x7f4544104140 2023-10-17T01:20:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.453+0000 7f455e36eb80 1 Processor -- start 2023-10-17T01:20:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.453+0000 7f45417fa700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45300a0ba0 0x7f45300a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.453+0000 7f45417fa700 1 -- 172.21.15.45:0/1395481229 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f452c0cd570 con 0x7f4544104140 2023-10-17T01:20:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.454+0000 7f455e36eb80 1 -- start start 2023-10-17T01:20:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.454+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1605c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.454+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e1606790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.454+0000 7f4548f9c700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45300a0ba0 0x7f45300a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e1607210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f455e36eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e155d9b0 con 0x55e7e1605830 2023-10-17T01:20:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f455e36eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e155b370 con 0x55e7e1606d90 2023-10-17T01:20:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f455e36eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1556c60 con 0x55e7e1606190 2023-10-17T01:20:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1605c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1605c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47102/0 (socket says 172.21.15.45:47102) 2023-10-17T01:20:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 -- 172.21.15.45:0/800865184 learned_addr learned my addr 172.21.15.45:0/800865184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f45437fe700 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e1606790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4548f9c700 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e1607210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 -- 172.21.15.45:0/800865184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 msgr2=0x55e7e1606790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e1606790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4548f9c700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45300a0ba0 0x7f45300a3060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f453c00f540 tx=0x7f453c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 -- 172.21.15.45:0/800865184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 msgr2=0x55e7e1607210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e1607210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 -- 172.21.15.45:0/800865184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7e1384d80 con 0x55e7e1605830 2023-10-17T01:20:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.455+0000 7f4543fff700 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1605c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4534009d20 tx=0x7f4534011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfb5aded62b1f10 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.456+0000 7f453a7fc700 1 -- 172.21.15.45:0/800865184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534004a30 con 0x55e7e1605830 2023-10-17T01:20:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.456+0000 7f453a7fc700 1 -- 172.21.15.45:0/800865184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45340054e0 con 0x55e7e1605830 2023-10-17T01:20:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.456+0000 7f453a7fc700 1 -- 172.21.15.45:0/800865184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534019830 con 0x55e7e1605830 2023-10-17T01:20:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 -- 172.21.15.45:0/800865184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 msgr2=0x55e7e1605c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1605c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4534009d20 tx=0x7f4534011720 comp rx=0 tx=0).stop 2023-10-17T01:20:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 -- 172.21.15.45:0/800865184 shutdown_connections 2023-10-17T01:20:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e1607210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1605c50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 --2- 172.21.15.45:0/800865184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e1606790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 -- 172.21.15.45:0/800865184 >> 172.21.15.45:0/800865184 conn(0x55e7e1565070 msgr2=0x55e7e16002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 -- 172.21.15.45:0/800865184 shutdown_connections 2023-10-17T01:20:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.457+0000 7f455e36eb80 1 -- 172.21.15.45:0/800865184 wait complete. 2023-10-17T01:20:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.458+0000 7f455e36eb80 1 -- start start 2023-10-17T01:20:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.458+0000 7f455e36eb80 1 Processor -- start 2023-10-17T01:20:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.459+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e160f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.459+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606190 0x55e7e160f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.459+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e16168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.459+0000 7f455e36eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e14f34b0 con 0x55e7e1606190 2023-10-17T01:20:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.459+0000 7f455e36eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e14f3150 con 0x55e7e1606d90 2023-10-17T01:20:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.459+0000 7f455e36eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e13da8c0 con 0x55e7e1605830 2023-10-17T01:20:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e16168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e16168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53738/0 (socket says 172.21.15.45:53738) 2023-10-17T01:20:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 -- 172.21.15.45:0/3988892126 learned_addr learned my addr 172.21.15.45:0/3988892126 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f455e36eb80 10 client.0 _ll_register_callbacks cb 0x55e7e1440a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4543fff700 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e160f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 -- 172.21.15.45:0/3988892126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 msgr2=0x55e7e160f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e160f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 -- 172.21.15.45:0/3988892126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606190 msgr2=0x55e7e160f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606190 0x55e7e160f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 -- 172.21.15.45:0/3988892126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4534009980 con 0x55e7e1606d90 2023-10-17T01:20:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f4548f9c700 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e16168b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f453c0089e0 tx=0x7f453c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d860a24bff154a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.460+0000 7f45237fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:20:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.461+0000 7f45237fe700 1 -- 172.21.15.45:0/3988892126 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c01c040 con 0x55e7e1606d90 2023-10-17T01:20:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.461+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7e1581cb0 con 0x55e7e1606d90 2023-10-17T01:20:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.461+0000 7f45237fe700 1 -- 172.21.15.45:0/3988892126 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f453c017ce0 con 0x55e7e1606d90 2023-10-17T01:20:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.461+0000 7f45237fe700 1 -- 172.21.15.45:0/3988892126 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f453c00e040 con 0x55e7e1606d90 2023-10-17T01:20:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 10 client.26102 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e7e136dc00 con 0x55e7e1606d90 2023-10-17T01:20:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f45237fe700 1 -- 172.21.15.45:0/3988892126 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f453c054020 con 0x55e7e1606d90 2023-10-17T01:20:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f45237fe700 1 -- 172.21.15.45:0/3988892126 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f453c043040 con 0x55e7e1606d90 2023-10-17T01:20:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 10 client.26102 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 20 client.26102 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1619940 0x55e7e161bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 4 client.26102 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:20:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.462+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7e15d4cd0 con 0x55e7e1619940 2023-10-17T01:20:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.463+0000 7f4543fff700 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1619940 0x55e7e161bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.463+0000 7f45237fe700 10 client.26102 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.463+0000 7f4543fff700 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1619940 0x55e7e161bd60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8358dc2b705bf7b3 server_cookie=bf823126facbfbe7 in_seq=0 out_seq=0 2023-10-17T01:20:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.464+0000 7f45237fe700 1 -- 172.21.15.45:0/3988892126 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55e7e15d4cd0 con 0x55e7e1619940 2023-10-17T01:20:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.464+0000 7f45237fe700 10 client.26102 handle_command_reply: tid=0 2023-10-17T01:20:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.464+0000 7f455e36eb80 1 client.26102 shutdown 2023-10-17T01:20:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 msgr2=0x55e7e16168b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e16168b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f453c0089e0 tx=0x7f453c004ad0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 shutdown_connections 2023-10-17T01:20:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1619940 0x55e7e161bd60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e16168b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e160f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3988892126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606190 0x55e7e160f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 >> 172.21.15.45:0/3988892126 conn(0x55e7e1565070 msgr2=0x55e7e15ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f45237fe700 0 client.26102 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 shutdown_connections 2023-10-17T01:20:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 1 -- 172.21.15.45:0/3988892126 wait complete. 2023-10-17T01:20:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.465+0000 7f455e36eb80 20 client.26102 trim_cache size 0 max 16384 2023-10-17T01:20:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.490+0000 7f455e36eb80 1 Processor -- start 2023-10-17T01:20:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.490+0000 7f455e36eb80 1 -- start start 2023-10-17T01:20:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1600150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e161b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e160d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f455e36eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1600db0 con 0x55e7e1605830 2023-10-17T01:20:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f455e36eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e160e7e0 con 0x55e7e1606d90 2023-10-17T01:20:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f455e36eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1601df0 con 0x55e7e1606190 2023-10-17T01:20:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f4548f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e160d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f4548f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e160d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53746/0 (socket says 172.21.15.45:53746) 2023-10-17T01:20:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f4548f9c700 1 -- 172.21.15.45:0/671444821 learned_addr learned my addr 172.21.15.45:0/671444821 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.491+0000 7f4543fff700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1600150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.492+0000 7f4548f9c700 1 -- 172.21.15.45:0/671444821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 msgr2=0x55e7e161b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.492+0000 7f45437fe700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e161b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.492+0000 7f4548f9c700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e161b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.492+0000 7f4548f9c700 1 -- 172.21.15.45:0/671444821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 msgr2=0x55e7e1600150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.492+0000 7f4548f9c700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1600150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.492+0000 7f4548f9c700 1 -- 172.21.15.45:0/671444821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7e13ab1f0 con 0x55e7e1606d90 2023-10-17T01:20:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.493+0000 7f45437fe700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e161b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.493+0000 7f4543fff700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1600150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.493+0000 7f4548f9c700 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e160d1b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f453c0061e0 tx=0x7f453c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d49e94749d2ef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.494+0000 7f453affd700 1 -- 172.21.15.45:0/671444821 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c00e070 con 0x55e7e1606d90 2023-10-17T01:20:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.494+0000 7f453affd700 1 -- 172.21.15.45:0/671444821 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f453c054040 con 0x55e7e1606d90 2023-10-17T01:20:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.494+0000 7f453affd700 1 -- 172.21.15.45:0/671444821 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c0051c0 con 0x55e7e1606d90 2023-10-17T01:20:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 -- 172.21.15.45:0/671444821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 msgr2=0x55e7e160d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e160d1b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f453c0061e0 tx=0x7f453c000f40 comp rx=0 tx=0).stop 2023-10-17T01:20:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 -- 172.21.15.45:0/671444821 shutdown_connections 2023-10-17T01:20:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606d90 0x55e7e160d1b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1605830 0x55e7e1600150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 --2- 172.21.15.45:0/671444821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1606190 0x55e7e161b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 -- 172.21.15.45:0/671444821 >> 172.21.15.45:0/671444821 conn(0x55e7e1565070 msgr2=0x55e7e1606570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 -- 172.21.15.45:0/671444821 shutdown_connections 2023-10-17T01:20:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 -- 172.21.15.45:0/671444821 wait complete. 2023-10-17T01:20:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.495+0000 7f455e36eb80 1 -- start start 2023-10-17T01:20:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 Processor -- start 2023-10-17T01:20:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e166ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 0x55e7e1672500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606d90 0x55e7e1674a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1674f90 con 0x55e7e1606d90 2023-10-17T01:20:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e16750d0 con 0x55e7e1606190 2023-10-17T01:20:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1675240 con 0x55e7e1605830 2023-10-17T01:20:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.496+0000 7f455e36eb80 10 client.0 _ll_register_callbacks cb 0x7f453c009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 0x55e7e1672500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f4543fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e166ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 0x55e7e1672500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53758/0 (socket says 172.21.15.45:53758) 2023-10-17T01:20:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f4543fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e166ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38270/0 (socket says 172.21.15.45:38270) 2023-10-17T01:20:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 -- 172.21.15.45:0/3333509005 learned_addr learned my addr 172.21.15.45:0/3333509005 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f4548f9c700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606d90 0x55e7e1674a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 -- 172.21.15.45:0/3333509005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 msgr2=0x55e7e166ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e166ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 -- 172.21.15.45:0/3333509005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606d90 msgr2=0x55e7e1674a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606d90 0x55e7e1674a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 -- 172.21.15.45:0/3333509005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7e136dc00 con 0x55e7e1606190 2023-10-17T01:20:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f4543fff700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e166ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:20:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45437fe700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 0x55e7e1672500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f452c029000 tx=0x7f452c09b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a37b1e31420f129 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45237fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:20:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.497+0000 7f45237fe700 1 -- 172.21.15.45:0/3333509005 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c0cd310 con 0x55e7e1606190 2023-10-17T01:20:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.498+0000 7f45237fe700 1 -- 172.21.15.45:0/3333509005 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f452c09eae0 con 0x55e7e1606190 2023-10-17T01:20:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.498+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45340054e0 con 0x55e7e1606190 2023-10-17T01:20:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.498+0000 7f45237fe700 1 -- 172.21.15.45:0/3333509005 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f452c0a2c90 con 0x55e7e1606190 2023-10-17T01:20:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.498+0000 7f455e36eb80 10 client.26108 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.498+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e7e1383250 con 0x55e7e1606190 2023-10-17T01:20:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f45237fe700 1 -- 172.21.15.45:0/3333509005 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45340054e0 con 0x55e7e1606190 2023-10-17T01:20:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f45237fe700 1 -- 172.21.15.45:0/3333509005 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f452c099c10 con 0x55e7e1606190 2023-10-17T01:20:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f455e36eb80 10 client.26108 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f455e36eb80 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f455e36eb80 20 client.26108 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1676880 0x55e7e1678ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f455e36eb80 4 client.26108 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:20:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.499+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e7e154fce0 con 0x55e7e1676880 2023-10-17T01:20:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.502+0000 7f4543fff700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1676880 0x55e7e1678ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.503+0000 7f4543fff700 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1676880 0x55e7e1678ca0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fab53ccd3688265 server_cookie=e4d6050b2a75d30b in_seq=0 out_seq=0 2023-10-17T01:20:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.503+0000 7f45237fe700 10 client.26108 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.519+0000 7f45237fe700 1 -- 172.21.15.45:0/3333509005 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e7e154fce0 con 0x55e7e1676880 2023-10-17T01:20:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f45237fe700 10 client.26108 handle_command_reply: tid=0 2023-10-17T01:20:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 client.26108 shutdown 2023-10-17T01:20:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 msgr2=0x55e7e1672500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 0x55e7e1672500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f452c029000 tx=0x7f452c09b1f0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 shutdown_connections 2023-10-17T01:20:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e7e1606d90 0x55e7e1674a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7e1605830 0x55e7e166ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e7e1676880 0x55e7e1678ca0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 --2- 172.21.15.45:0/3333509005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e7e1606190 0x55e7e1672500 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 >> 172.21.15.45:0/3333509005 conn(0x55e7e1565070 msgr2=0x55e7e1600110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f45237fe700 0 client.26108 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 shutdown_connections 2023-10-17T01:20:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 1 -- 172.21.15.45:0/3333509005 wait complete. 2023-10-17T01:20:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.520+0000 7f455e36eb80 20 client.26108 trim_cache size 0 max 16384 2023-10-17T01:20:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:20:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:20:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:20:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:20:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.521+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45300a0ba0 msgr2=0x7f45300a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.521+0000 7f454a9ff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45300a0ba0 0x7f45300a3060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f453c00f540 tx=0x7f453c00b040 comp rx=0 tx=0).stop 2023-10-17T01:20:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.521+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 msgr2=0x7f45441992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.521+0000 7f454a9ff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 0x7f45441992f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b9e0 tx=0x7f452c00ba10 comp rx=0 tx=0).stop 2023-10-17T01:20:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 shutdown_connections 2023-10-17T01:20:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45300a0ba0 0x7f45300a3060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4544106a70 0x7f454419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4544101810 0x7f4544198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 --2- 172.21.15.45:0/1395481229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4544104140 0x7f45441992f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 >> 172.21.15.45:0/1395481229 conn(0x7f45440fb1a0 msgr2=0x7f45440ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 shutdown_connections 2023-10-17T01:20:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:38.522+0000 7f454a9ff700 1 -- 172.21.15.45:0/1395481229 wait complete. 2023-10-17T01:20:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:38 smithi045 ceph-mon[98474]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2023-10-17T01:20:39.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:38 smithi060 ceph-mon[134136]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2023-10-17T01:20:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:38 smithi181 ceph-mon[134520]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2023-10-17T01:20:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 330.0 s, 2260.6 tps, lat 44.650 ms stddev 54.754 2023-10-17T01:20:41.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:41 smithi181 ceph-mon[134520]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 1.49k op/s 2023-10-17T01:20:42.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:41 smithi045 ceph-mon[98474]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 1.49k op/s 2023-10-17T01:20:42.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:41 smithi060 ceph-mon[134136]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 1.49k op/s 2023-10-17T01:20:44.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:43 smithi045 ceph-mon[98474]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 743 op/s 2023-10-17T01:20:44.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:43 smithi060 ceph-mon[134136]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 743 op/s 2023-10-17T01:20:44.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:43 smithi181 ceph-mon[134520]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 743 op/s 2023-10-17T01:20:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 335.0 s, 2190.4 tps, lat 45.742 ms stddev 55.779 2023-10-17T01:20:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:45 smithi181 ceph-mon[134520]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 751 op/s 2023-10-17T01:20:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:45 smithi045 ceph-mon[98474]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 751 op/s 2023-10-17T01:20:46.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:45 smithi060 ceph-mon[134136]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 751 op/s 2023-10-17T01:20:48.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:47 smithi045 ceph-mon[98474]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 693 op/s 2023-10-17T01:20:48.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:47 smithi060 ceph-mon[134136]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 693 op/s 2023-10-17T01:20:48.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:47 smithi181 ceph-mon[134520]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 693 op/s 2023-10-17T01:20:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:48 smithi045 ceph-mon[98474]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 690 op/s 2023-10-17T01:20:49.087 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:48 smithi060 ceph-mon[134136]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 690 op/s 2023-10-17T01:20:49.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:48 smithi181 ceph-mon[134520]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 690 op/s 2023-10-17T01:20:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 340.0 s, 2235.6 tps, lat 44.336 ms stddev 55.676 2023-10-17T01:20:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 -- 172.21.15.45:0/1555568859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c107d50 msgr2=0x7f1d4c10a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 --2- 172.21.15.45:0/1555568859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10a180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b0a0 tx=0x7f1d4400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 -- 172.21.15.45:0/1555568859 shutdown_connections 2023-10-17T01:20:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 --2- 172.21.15.45:0/1555568859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10ab40 0x7f1d4c1139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 --2- 172.21.15.45:0/1555568859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10a180 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 --2- 172.21.15.45:0/1555568859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c10a6c0 0x7f1d4c111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 -- 172.21.15.45:0/1555568859 >> 172.21.15.45:0/1555568859 conn(0x7f1d4c06df20 msgr2=0x7f1d4c070380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 -- 172.21.15.45:0/1555568859 shutdown_connections 2023-10-17T01:20:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.785+0000 7f1d53bff700 1 -- 172.21.15.45:0/1555568859 wait complete. 2023-10-17T01:20:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.787+0000 7f1d53bff700 1 Processor -- start 2023-10-17T01:20:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.788+0000 7f1d53bff700 1 -- start start 2023-10-17T01:20:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.788+0000 7f1d53bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.788+0000 7f1d53bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c10a6c0 0x7f1d4c10ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.788+0000 7f1d53bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10ab40 0x7f1d4c10c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.788+0000 7f1d53bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c10cf50 con 0x7f1d4c10ab40 2023-10-17T01:20:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d53bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c10d0c0 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d53bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c10d280 con 0x7f1d4c107d50 2023-10-17T01:20:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60304/0 (socket says 172.21.15.45:60304) 2023-10-17T01:20:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5199b700 1 -- 172.21.15.45:0/3509931564 learned_addr learned my addr 172.21.15.45:0/3509931564 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c10a6c0 0x7f1d4c10ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5119a700 1 -- 172.21.15.45:0/3509931564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c107d50 msgr2=0x7f1d4c10b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5119a700 1 -- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10ab40 msgr2=0x7f1d4c10c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10ab40 0x7f1d4c10c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.789+0000 7f1d5119a700 1 -- 172.21.15.45:0/3509931564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009d00 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.790+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c10a6c0 0x7f1d4c10ba20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800d950 tx=0x7f1d4800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.790+0000 7f1d3effd700 1 -- 172.21.15.45:0/3509931564 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48009930 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.791+0000 7f1d3effd700 1 -- 172.21.15.45:0/3509931564 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d48010460 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.791+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c10d560 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.791+0000 7f1d3effd700 1 -- 172.21.15.45:0/3509931564 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800b5d0 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.791+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c10dd10 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.796+0000 7f1d3effd700 1 -- 172.21.15.45:0/3509931564 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f1d48009a90 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.806+0000 7f1d3effd700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1d380a0cf0 0x7f1d380a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.807+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1d380a0cf0 0x7f1d380a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.812+0000 7f1d67427b80 1 Processor -- start 2023-10-17T01:20:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.812+0000 7f1d3effd700 1 -- 172.21.15.45:0/3509931564 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1d480cbc60 con 0x7f1d4c10a6c0 2023-10-17T01:20:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.812+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1d380a0cf0 0x7f1d380a31b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b830 tx=0x7f1d44006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.813+0000 7f1d67427b80 1 -- start start 2023-10-17T01:20:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736bd830 0x5562736bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736be190 0x5562736be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x5562736bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d67427b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562736159b0 con 0x5562736be190 2023-10-17T01:20:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d67427b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273613370 con 0x5562736bd830 2023-10-17T01:20:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d67427b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55627360ec60 con 0x5562736bed90 2023-10-17T01:20:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d5219c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x5562736bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d5219c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x5562736bf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60320/0 (socket says 172.21.15.45:60320) 2023-10-17T01:20:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.814+0000 7f1d5219c700 1 -- 172.21.15.45:0/3994275497 learned_addr learned my addr 172.21.15.45:0/3994275497 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.815+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736be190 0x5562736be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.816+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736bd830 0x5562736bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.817+0000 7f1d5199b700 1 -- 172.21.15.45:0/3994275497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 msgr2=0x5562736bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.817+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x5562736bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.817+0000 7f1d5199b700 1 -- 172.21.15.45:0/3994275497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736be190 msgr2=0x5562736be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.817+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736be190 0x5562736be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.817+0000 7f1d5199b700 1 -- 172.21.15.45:0/3994275497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55627343cd80 con 0x5562736bd830 2023-10-17T01:20:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.818+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736bd830 0x5562736bdc50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009fd0 tx=0x7f1d4401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c9b4b3f9c91eaa server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.826+0000 7f1d37fff700 1 -- 172.21.15.45:0/3994275497 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400e040 con 0x5562736bd830 2023-10-17T01:20:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.826+0000 7f1d37fff700 1 -- 172.21.15.45:0/3994275497 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d44003e60 con 0x5562736bd830 2023-10-17T01:20:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.826+0000 7f1d37fff700 1 -- 172.21.15.45:0/3994275497 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44005280 con 0x5562736bd830 2023-10-17T01:20:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 -- 172.21.15.45:0/3994275497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736bd830 msgr2=0x5562736bdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736bd830 0x5562736bdc50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009fd0 tx=0x7f1d4401c000 comp rx=0 tx=0).stop 2023-10-17T01:20:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 -- 172.21.15.45:0/3994275497 shutdown_connections 2023-10-17T01:20:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x5562736bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736bd830 0x5562736bdc50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3994275497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736be190 0x5562736be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 -- 172.21.15.45:0/3994275497 >> 172.21.15.45:0/3994275497 conn(0x55627361d070 msgr2=0x5562736b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 -- 172.21.15.45:0/3994275497 shutdown_connections 2023-10-17T01:20:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.827+0000 7f1d67427b80 1 -- 172.21.15.45:0/3994275497 wait complete. 2023-10-17T01:20:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 -- start start 2023-10-17T01:20:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 Processor -- start 2023-10-17T01:20:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562735ab4b0 con 0x5562736bed90 2023-10-17T01:20:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562735ab150 con 0x5562736be190 2023-10-17T01:20:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.828+0000 7f1d67427b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562734928c0 con 0x5562736bd830 2023-10-17T01:20:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d67427b80 10 client.0 _ll_register_callbacks cb 0x5562734f8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38826/0 (socket says 172.21.15.45:38826) 2023-10-17T01:20:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 -- 172.21.15.45:0/1721356100 learned_addr learned my addr 172.21.15.45:0/1721356100 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 -- 172.21.15.45:0/1721356100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 msgr2=0x5562736c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 -- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 msgr2=0x5562736c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.829+0000 7f1d5219c700 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.830+0000 7f1d5219c700 1 -- 172.21.15.45:0/1721356100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c1144b0 con 0x5562736bed90 2023-10-17T01:20:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.830+0000 7f1d5219c700 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736ce880 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000beb0 tx=0x7f1d4000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b4ede9140c4a59 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.830+0000 7f1d357fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:20:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.830+0000 7f1d357fa700 1 -- 172.21.15.45:0/1721356100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40004250 con 0x5562736bed90 2023-10-17T01:20:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.830+0000 7f1d357fa700 1 -- 172.21.15.45:0/1721356100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1d400043b0 con 0x5562736bed90 2023-10-17T01:20:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.830+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556273639cb0 con 0x5562736bed90 2023-10-17T01:20:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d357fa700 1 -- 172.21.15.45:0/1721356100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d40004570 con 0x5562736bed90 2023-10-17T01:20:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 10 client.15858 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556273425c00 con 0x5562736bed90 2023-10-17T01:20:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d357fa700 1 -- 172.21.15.45:0/1721356100 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556273639cb0 con 0x5562736bed90 2023-10-17T01:20:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d357fa700 1 -- 172.21.15.45:0/1721356100 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1d4001a920 con 0x5562736bed90 2023-10-17T01:20:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 20 client.15858 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562736d31e0 0x5562736d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 4 client.15858 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:20:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.831+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55627368ccd0 con 0x5562736d31e0 2023-10-17T01:20:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.835+0000 7f1d5199b700 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562736d31e0 0x5562736d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.837+0000 7f1d5199b700 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562736d31e0 0x5562736d5600 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8884015b04b4eb server_cookie=85c3c899da73a5 in_seq=0 out_seq=0 2023-10-17T01:20:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.837+0000 7f1d357fa700 10 client.15858 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d357fa700 1 -- 172.21.15.45:0/1721356100 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55627368ccd0 con 0x5562736d31e0 2023-10-17T01:20:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d357fa700 10 client.15858 handle_command_reply: tid=0 2023-10-17T01:20:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 client.15858 shutdown 2023-10-17T01:20:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 msgr2=0x5562736ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736ce880 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000beb0 tx=0x7f1d4000bf90 comp rx=0 tx=0).stop 2023-10-17T01:20:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 shutdown_connections 2023-10-17T01:20:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736ce880 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562736d31e0 0x5562736d5600 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 --2- 172.21.15.45:0/1721356100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 >> 172.21.15.45:0/1721356100 conn(0x55627361d070 msgr2=0x5562736b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d357fa700 0 client.15858 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 shutdown_connections 2023-10-17T01:20:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.838+0000 7f1d67427b80 1 -- 172.21.15.45:0/1721356100 wait complete. 2023-10-17T01:20:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.839+0000 7f1d67427b80 20 client.15858 trim_cache size 0 max 16384 2023-10-17T01:20:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.863+0000 7f1d67427b80 1 Processor -- start 2023-10-17T01:20:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.863+0000 7f1d67427b80 1 -- start start 2023-10-17T01:20:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736b9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d67427b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562736c67e0 con 0x5562736bed90 2023-10-17T01:20:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d67427b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562736b29c0 con 0x5562736be190 2023-10-17T01:20:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d67427b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562736c5040 con 0x5562736bd830 2023-10-17T01:20:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d5199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d5119a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d5199b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736b9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60336/0 (socket says 172.21.15.45:60336) 2023-10-17T01:20:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d5199b700 1 -- 172.21.15.45:0/2236980441 learned_addr learned my addr 172.21.15.45:0/2236980441 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.864+0000 7f1d5219c700 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d5119a700 1 -- 172.21.15.45:0/2236980441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 msgr2=0x5562736b9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d5119a700 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736b9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d5119a700 1 -- 172.21.15.45:0/2236980441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 msgr2=0x5562736c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d5119a700 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d5119a700 1 -- 172.21.15.45:0/2236980441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562734631f0 con 0x5562736be190 2023-10-17T01:20:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d5119a700 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c69b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d480c9730 tx=0x7f1d4800d8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50995b14c5f16c58 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.865+0000 7f1d3cbff700 1 -- 172.21.15.45:0/2236980441 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48014020 con 0x5562736be190 2023-10-17T01:20:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.866+0000 7f1d3cbff700 1 -- 172.21.15.45:0/2236980441 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d48093e90 con 0x5562736be190 2023-10-17T01:20:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.866+0000 7f1d3cbff700 1 -- 172.21.15.45:0/2236980441 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48009d40 con 0x5562736be190 2023-10-17T01:20:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.867+0000 7f1d67427b80 1 -- 172.21.15.45:0/2236980441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 msgr2=0x5562736c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.867+0000 7f1d67427b80 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c69b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d480c9730 tx=0x7f1d4800d8d0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.868+0000 7f1d67427b80 1 -- 172.21.15.45:0/2236980441 shutdown_connections 2023-10-17T01:20:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.868+0000 7f1d67427b80 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bed90 0x5562736c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.868+0000 7f1d67427b80 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bd830 0x5562736b9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.868+0000 7f1d67427b80 1 --2- 172.21.15.45:0/2236980441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736c69b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.868+0000 7f1d67427b80 1 -- 172.21.15.45:0/2236980441 >> 172.21.15.45:0/2236980441 conn(0x55627361d070 msgr2=0x5562736c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.868+0000 7f1d67427b80 1 -- 172.21.15.45:0/2236980441 shutdown_connections 2023-10-17T01:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.869+0000 7f1d67427b80 1 -- 172.21.15.45:0/2236980441 wait complete. 2023-10-17T01:20:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.869+0000 7f1d67427b80 1 -- start start 2023-10-17T01:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.869+0000 7f1d67427b80 1 Processor -- start 2023-10-17T01:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bd830 0x5562736cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x556273723170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273723720 con 0x5562736bd830 2023-10-17T01:20:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273723890 con 0x5562736be190 2023-10-17T01:20:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273723a50 con 0x5562736bed90 2023-10-17T01:20:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d67427b80 10 client.0 _ll_register_callbacks cb 0x7f1d4000b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:20:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d5219c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x556273723170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d5219c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x556273723170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60346/0 (socket says 172.21.15.45:60346) 2023-10-17T01:20:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d5219c700 1 -- 172.21.15.45:0/3433776003 learned_addr learned my addr 172.21.15.45:0/3433776003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:20:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.870+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bd830 0x5562736cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.871+0000 7f1d5219c700 1 -- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 msgr2=0x5562736cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.871+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5219c700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5219c700 1 -- 172.21.15.45:0/3433776003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bd830 msgr2=0x5562736cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:20:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5219c700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bd830 0x5562736cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5219c700 1 -- 172.21.15.45:0/3433776003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c10dd10 con 0x5562736bed90 2023-10-17T01:20:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bd830 0x5562736cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:20:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5119a700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:20:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.872+0000 7f1d5219c700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x556273723170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d40000c00 tx=0x7f1d40010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa53e7e42c23cd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:20:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.873+0000 7f1d357fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:20:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.873+0000 7f1d357fa700 1 -- 172.21.15.45:0/3433776003 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4001aea0 con 0x5562736bed90 2023-10-17T01:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.873+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d44003e60 con 0x5562736bed90 2023-10-17T01:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.873+0000 7f1d357fa700 1 -- 172.21.15.45:0/3433776003 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1d4002b070 con 0x5562736bed90 2023-10-17T01:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.873+0000 7f1d357fa700 1 -- 172.21.15.45:0/3433776003 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d4001aea0 con 0x5562736bed90 2023-10-17T01:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.874+0000 7f1d67427b80 10 client.26095 fetch_fsmap learned FSMap version 40 2023-10-17T01:20:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.874+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55627343b250 con 0x5562736bed90 2023-10-17T01:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.874+0000 7f1d357fa700 1 -- 172.21.15.45:0/3433776003 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d44003e60 con 0x5562736bed90 2023-10-17T01:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.874+0000 7f1d357fa700 1 -- 172.21.15.45:0/3433776003 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1d40036070 con 0x5562736bed90 2023-10-17T01:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.876+0000 7f1d67427b80 10 client.26095 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.876+0000 7f1d67427b80 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:20:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.876+0000 7f1d67427b80 20 client.26095 populate_metadata read hostname 'smithi045' 2023-10-17T01:20:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.876+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562737252f0 0x556273727710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:20:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.876+0000 7f1d67427b80 4 client.26095 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:20:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.876+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556273607ce0 con 0x5562737252f0 2023-10-17T01:20:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.877+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562737252f0 0x556273727710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:20:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.878+0000 7f1d5199b700 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562737252f0 0x556273727710 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45b9ae062be1a87 server_cookie=260214290ceb7923 in_seq=0 out_seq=0 2023-10-17T01:20:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.878+0000 7f1d357fa700 10 client.26095 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.893+0000 7f1d357fa700 1 -- 172.21.15.45:0/3433776003 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556273607ce0 con 0x5562737252f0 2023-10-17T01:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.893+0000 7f1d357fa700 10 client.26095 handle_command_reply: tid=0 2023-10-17T01:20:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.893+0000 7f1d67427b80 1 client.26095 shutdown 2023-10-17T01:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 msgr2=0x556273723170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x556273723170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d40000c00 tx=0x7f1d40010b80 comp rx=0 tx=0).stop 2023-10-17T01:20:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 shutdown_connections 2023-10-17T01:20:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5562736bed90 0x556273723170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562736bd830 0x5562736cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5562737252f0 0x556273727710 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 --2- 172.21.15.45:0/3433776003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5562736be190 0x5562736cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 >> 172.21.15.45:0/3433776003 conn(0x55627361d070 msgr2=0x5562736c5260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d357fa700 0 client.26095 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:20:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.894+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 shutdown_connections 2023-10-17T01:20:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.895+0000 7f1d67427b80 1 -- 172.21.15.45:0/3433776003 wait complete. 2023-10-17T01:20:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.895+0000 7f1d67427b80 20 client.26095 trim_cache size 0 max 16384 2023-10-17T01:20:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:20:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:20:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1d380a0cf0 msgr2=0x7f1d380a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1d380a0cf0 0x7f1d380a31b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b830 tx=0x7f1d44006210 comp rx=0 tx=0).stop 2023-10-17T01:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c10a6c0 msgr2=0x7f1d4c10ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c10a6c0 0x7f1d4c10ba20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800d950 tx=0x7f1d4800dc60 comp rx=0 tx=0).stop 2023-10-17T01:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 shutdown_connections 2023-10-17T01:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1d380a0cf0 0x7f1d380a31b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10ab40 0x7f1d4c10c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d4c107d50 0x7f1d4c10b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 --2- 172.21.15.45:0/3509931564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d4c10a6c0 0x7f1d4c10ba20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 >> 172.21.15.45:0/3509931564 conn(0x7f1d4c06df20 msgr2=0x7f1d4c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.896+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 shutdown_connections 2023-10-17T01:20:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:20:51.897+0000 7f1d53bff700 1 -- 172.21.15.45:0/3509931564 wait complete. 2023-10-17T01:20:51.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:51 smithi181 ceph-mon[134520]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-17T01:20:52.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:51 smithi045 ceph-mon[98474]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-17T01:20:52.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:51 smithi060 ceph-mon[134136]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-17T01:20:54.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:53 smithi045 ceph-mon[98474]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-17T01:20:54.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:53 smithi060 ceph-mon[134136]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-17T01:20:54.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:53 smithi181 ceph-mon[134520]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-17T01:20:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 345.0 s, 2254.0 tps, lat 44.827 ms stddev 55.145 2023-10-17T01:20:55.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:55 smithi181 ceph-mon[134520]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 MiB/s wr, 831 op/s 2023-10-17T01:20:56.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:55 smithi045 ceph-mon[98474]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 MiB/s wr, 831 op/s 2023-10-17T01:20:56.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:55 smithi060 ceph-mon[134136]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 MiB/s wr, 831 op/s 2023-10-17T01:20:57.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:57 smithi181 ceph-mon[134520]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 713 op/s 2023-10-17T01:20:58.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:57 smithi045 ceph-mon[98474]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 713 op/s 2023-10-17T01:20:58.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:57 smithi060 ceph-mon[134136]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 713 op/s 2023-10-17T01:21:00.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:20:59 smithi045 ceph-mon[98474]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 699 op/s 2023-10-17T01:21:00.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:20:59 smithi060 ceph-mon[134136]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 699 op/s 2023-10-17T01:21:00.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:20:59 smithi181 ceph-mon[134520]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 699 op/s 2023-10-17T01:21:00.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 350.0 s, 1723.8 tps, lat 56.535 ms stddev 69.751 2023-10-17T01:21:02.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:01 smithi045 ceph-mon[98474]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 830 op/s 2023-10-17T01:21:02.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:01 smithi060 ceph-mon[134136]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 830 op/s 2023-10-17T01:21:02.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:21:02.163 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:01 smithi181 ceph-mon[134520]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 830 op/s 2023-10-17T01:21:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 -- 172.21.15.45:0/1027941495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb41040f0 msgr2=0x7fafb4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1027941495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb41040f0 0x7fafb4104570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009990 tx=0x7fafa8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 -- 172.21.15.45:0/1027941495 shutdown_connections 2023-10-17T01:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1027941495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 0x7fafb410b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1027941495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4102f20 0x7fafb4103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1027941495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb41040f0 0x7fafb4104570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.487+0000 7fafba7ff700 1 -- 172.21.15.45:0/1027941495 >> 172.21.15.45:0/1027941495 conn(0x7fafb40fe500 msgr2=0x7fafb4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.488+0000 7fafba7ff700 1 -- 172.21.15.45:0/1027941495 shutdown_connections 2023-10-17T01:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.488+0000 7fafba7ff700 1 -- 172.21.15.45:0/1027941495 wait complete. 2023-10-17T01:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.489+0000 7fafba7ff700 1 Processor -- start 2023-10-17T01:21:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.489+0000 7fafba7ff700 1 -- start start 2023-10-17T01:21:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.489+0000 7fafba7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 0x7fafb419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.489+0000 7fafba7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb41040f0 0x7fafb419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.489+0000 7fafba7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 0x7fafb41a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.489+0000 7fafba7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41a4b10 con 0x7fafb41040f0 2023-10-17T01:21:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafba7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb406a770 con 0x7fafb4102f20 2023-10-17T01:21:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafba7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb406a900 con 0x7fafb4104ab0 2023-10-17T01:21:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 0x7fafb419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 0x7fafb419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48802/0 (socket says 172.21.15.45:48802) 2023-10-17T01:21:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb3fff700 1 -- 172.21.15.45:0/1656813303 learned_addr learned my addr 172.21.15.45:0/1656813303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 0x7fafb41a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb37fe700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb41040f0 0x7fafb419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1656813303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 msgr2=0x7fafb419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 0x7fafb419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb41040f0 msgr2=0x7fafb419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb41040f0 0x7fafb419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1656813303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80095f0 con 0x7fafb4104ab0 2023-10-17T01:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.490+0000 7fafb3fff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 0x7fafb419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.491+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 0x7fafb41a4540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fafac00cc50 tx=0x7fafac00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.491+0000 7fafb17fa700 1 -- 172.21.15.45:0/1656813303 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac00aa90 con 0x7fafb4104ab0 2023-10-17T01:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.491+0000 7fafb17fa700 1 -- 172.21.15.45:0/1656813303 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafac00abf0 con 0x7fafb4104ab0 2023-10-17T01:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.491+0000 7fafb17fa700 1 -- 172.21.15.45:0/1656813303 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac0115e0 con 0x7fafb4104ab0 2023-10-17T01:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.491+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb406abe0 con 0x7fafb4104ab0 2023-10-17T01:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.491+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb406b0f0 con 0x7fafb4104ab0 2023-10-17T01:21:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.497+0000 7fafce162b80 1 Processor -- start 2023-10-17T01:21:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.497+0000 7fafb17fa700 1 -- 172.21.15.45:0/1656813303 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fafac012420 con 0x7fafb4104ab0 2023-10-17T01:21:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 -- start start 2023-10-17T01:21:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b902bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b902cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b902d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b8f83a10 con 0x5614b902d2b0 2023-10-17T01:21:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b8f813d0 con 0x5614b902c500 2023-10-17T01:21:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.498+0000 7fafce162b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b8f7ccc0 con 0x5614b902bba0 2023-10-17T01:21:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b902cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b902bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b902bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45940/0 (socket says 172.21.15.45:45940) 2023-10-17T01:21:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb37fe700 1 -- 172.21.15.45:0/2857953236 learned_addr learned my addr 172.21.15.45:0/2857953236 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb37fe700 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b902d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb3fff700 1 -- 172.21.15.45:0/2857953236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 msgr2=0x5614b902bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb3fff700 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b902bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb3fff700 1 -- 172.21.15.45:0/2857953236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 msgr2=0x5614b902d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.499+0000 7fafb3fff700 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b902d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7fafb3fff700 1 -- 172.21.15.45:0/2857953236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614b8daade0 con 0x5614b902c500 2023-10-17T01:21:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7fafb37fe700 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b902d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:21:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7fafb3fff700 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b902cd20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fafa400f4c0 tx=0x7fafa400f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74309150f06833d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7fafb17fa700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faf9c0a0ba0 0x7faf9c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7fafb17fa700 1 -- 172.21.15.45:0/1656813303 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fafac0cc1a0 con 0x7fafb4104ab0 2023-10-17T01:21:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7faf9a7fc700 1 -- 172.21.15.45:0/2857953236 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa400abb0 con 0x5614b902c500 2023-10-17T01:21:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7fafb37fe700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faf9c0a0ba0 0x7faf9c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7faf9a7fc700 1 -- 172.21.15.45:0/2857953236 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafa4004830 con 0x5614b902c500 2023-10-17T01:21:02.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.500+0000 7faf9a7fc700 1 -- 172.21.15.45:0/2857953236 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4005540 con 0x5614b902c500 2023-10-17T01:21:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.501+0000 7fafb37fe700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faf9c0a0ba0 0x7faf9c0a3060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5614b902b940 tx=0x7fafa8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.501+0000 7fafce162b80 1 -- 172.21.15.45:0/2857953236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 msgr2=0x5614b902cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.501+0000 7fafce162b80 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b902cd20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fafa400f4c0 tx=0x7fafa400f7d0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.502+0000 7fafce162b80 1 -- 172.21.15.45:0/2857953236 shutdown_connections 2023-10-17T01:21:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.502+0000 7fafce162b80 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b902d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.502+0000 7fafce162b80 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b902bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.502+0000 7fafce162b80 1 --2- 172.21.15.45:0/2857953236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b902cd20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.502+0000 7fafce162b80 1 -- 172.21.15.45:0/2857953236 >> 172.21.15.45:0/2857953236 conn(0x5614b8f8b0d0 msgr2=0x5614b9026950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.503+0000 7fafce162b80 1 -- 172.21.15.45:0/2857953236 shutdown_connections 2023-10-17T01:21:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.503+0000 7fafce162b80 1 -- 172.21.15.45:0/2857953236 wait complete. 2023-10-17T01:21:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.503+0000 7fafce162b80 1 -- start start 2023-10-17T01:21:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 Processor -- start 2023-10-17T01:21:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902bba0 0x5614b90358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902c500 0x5614b9035e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b903ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b8f19510 con 0x5614b902c500 2023-10-17T01:21:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b8f191b0 con 0x5614b902bba0 2023-10-17T01:21:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b8e00920 con 0x5614b902d2b0 2023-10-17T01:21:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.504+0000 7fafce162b80 10 client.0 _ll_register_callbacks cb 0x5614b8e66a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b903ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b903ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45954/0 (socket says 172.21.15.45:45954) 2023-10-17T01:21:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 -- 172.21.15.45:0/3042727877 learned_addr learned my addr 172.21.15.45:0/3042727877 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb3fff700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902bba0 0x5614b90358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb37fe700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902c500 0x5614b9035e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 -- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902bba0 msgr2=0x5614b90358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902bba0 0x5614b90358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 -- 172.21.15.45:0/3042727877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902c500 msgr2=0x5614b9035e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902c500 0x5614b9035e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 -- 172.21.15.45:0/3042727877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa400ad10 con 0x5614b902d2b0 2023-10-17T01:21:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b903ce40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafac096290 tx=0x7fafac099180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78ca816e8af14b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.505+0000 7faf937fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7faf937fe700 1 -- 172.21.15.45:0/3042727877 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac0b90b0 con 0x5614b902d2b0 2023-10-17T01:21:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7faf937fe700 1 -- 172.21.15.45:0/3042727877 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fafac0cb070 con 0x5614b902d2b0 2023-10-17T01:21:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614b8fa7d10 con 0x5614b902d2b0 2023-10-17T01:21:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7faf937fe700 1 -- 172.21.15.45:0/3042727877 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafac099f20 con 0x5614b902d2b0 2023-10-17T01:21:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 10 client.26104 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5614b8d93c60 con 0x5614b902d2b0 2023-10-17T01:21:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7faf937fe700 1 -- 172.21.15.45:0/3042727877 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafac07c080 con 0x5614b902d2b0 2023-10-17T01:21:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7faf937fe700 1 -- 172.21.15.45:0/3042727877 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fafac098d10 con 0x5614b902d2b0 2023-10-17T01:21:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 10 client.26104 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 20 client.26104 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b9041720 0x5614b9043b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 4 client.26104 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.506+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614b8ffad30 con 0x5614b9041720 2023-10-17T01:21:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.507+0000 7fafb3fff700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b9041720 0x5614b9043b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.507+0000 7fafb3fff700 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b9041720 0x5614b9043b40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18de29f2dfafd467 server_cookie=240d6904c89e5114 in_seq=0 out_seq=0 2023-10-17T01:21:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.507+0000 7faf937fe700 10 client.26104 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.508+0000 7faf937fe700 1 -- 172.21.15.45:0/3042727877 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5614b8ffad30 con 0x5614b9041720 2023-10-17T01:21:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.508+0000 7faf937fe700 10 client.26104 handle_command_reply: tid=0 2023-10-17T01:21:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.508+0000 7fafce162b80 1 client.26104 shutdown 2023-10-17T01:21:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.509+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 msgr2=0x5614b903ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.509+0000 7fafce162b80 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b903ce40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafac096290 tx=0x7fafac099180 comp rx=0 tx=0).stop 2023-10-17T01:21:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 shutdown_connections 2023-10-17T01:21:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7fafce162b80 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b903ce40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7fafce162b80 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b9041720 0x5614b9043b40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7fafce162b80 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902bba0 0x5614b90358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7fafce162b80 1 --2- 172.21.15.45:0/3042727877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902c500 0x5614b9035e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7faf937fe700 0 client.26104 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:02.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.510+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 >> 172.21.15.45:0/3042727877 conn(0x5614b8f8b0d0 msgr2=0x5614b9026020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:02.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.511+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 shutdown_connections 2023-10-17T01:21:02.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.512+0000 7fafce162b80 1 -- 172.21.15.45:0/3042727877 wait complete. 2023-10-17T01:21:02.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.512+0000 7fafce162b80 20 client.26104 trim_cache size 0 max 16384 2023-10-17T01:21:02.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.536+0000 7fafce162b80 1 Processor -- start 2023-10-17T01:21:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.537+0000 7fafce162b80 1 -- start start 2023-10-17T01:21:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.537+0000 7fafce162b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902bba0 0x5614b90345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.537+0000 7fafce162b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b9026020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.537+0000 7fafce162b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b9033750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.538+0000 7fafce162b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b903eb30 con 0x5614b902bba0 2023-10-17T01:21:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.538+0000 7fafce162b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b9036a40 con 0x5614b902c500 2023-10-17T01:21:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.538+0000 7fafce162b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b9023420 con 0x5614b902d2b0 2023-10-17T01:21:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.538+0000 7fafb8d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b9033750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.538+0000 7fafb8d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b9033750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45966/0 (socket says 172.21.15.45:45966) 2023-10-17T01:21:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.538+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1257520182 learned_addr learned my addr 172.21.15.45:0/1257520182 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1257520182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 msgr2=0x5614b9026020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:21:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b9026020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1257520182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902bba0 msgr2=0x5614b90345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902bba0 0x5614b90345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7fafb8d9c700 1 -- 172.21.15.45:0/1257520182 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614b8dd1250 con 0x5614b902d2b0 2023-10-17T01:21:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b9033750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafac095cd0 tx=0x7fafac018fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765c10d3d33b1aef server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7faf9affd700 1 -- 172.21.15.45:0/1257520182 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac07f070 con 0x5614b902d2b0 2023-10-17T01:21:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7faf9affd700 1 -- 172.21.15.45:0/1257520182 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614b8fa7d10 con 0x5614b902d2b0 2023-10-17T01:21:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.539+0000 7faf9affd700 1 -- 172.21.15.45:0/1257520182 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac09d650 con 0x5614b902d2b0 2023-10-17T01:21:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 -- 172.21.15.45:0/1257520182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 msgr2=0x5614b9033750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b9033750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafac095cd0 tx=0x7fafac018fb0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 -- 172.21.15.45:0/1257520182 shutdown_connections 2023-10-17T01:21:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902d2b0 0x5614b9033750 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902bba0 0x5614b90345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 --2- 172.21.15.45:0/1257520182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b9026020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.540+0000 7fafce162b80 1 -- 172.21.15.45:0/1257520182 >> 172.21.15.45:0/1257520182 conn(0x5614b8f8b0d0 msgr2=0x5614b9027650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.541+0000 7fafce162b80 1 -- 172.21.15.45:0/1257520182 shutdown_connections 2023-10-17T01:21:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.541+0000 7fafce162b80 1 -- 172.21.15.45:0/1257520182 wait complete. 2023-10-17T01:21:02.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.541+0000 7fafce162b80 1 -- start start 2023-10-17T01:21:02.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.541+0000 7fafce162b80 1 Processor -- start 2023-10-17T01:21:02.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b908b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b908b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b9092810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b9092d50 con 0x5614b902d2b0 2023-10-17T01:21:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b9092e90 con 0x5614b902c500 2023-10-17T01:21:02.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614b9093050 con 0x5614b902bba0 2023-10-17T01:21:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafce162b80 10 client.0 _ll_register_callbacks cb 0x7fafac099660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b908b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafb37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b908b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b908b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45980/0 (socket says 172.21.15.45:45980) 2023-10-17T01:21:02.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.542+0000 7fafb3fff700 1 -- 172.21.15.45:0/3240209688 learned_addr learned my addr 172.21.15.45:0/3240209688 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb8d9c700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b9092810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb3fff700 1 -- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 msgr2=0x5614b908b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb3fff700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b908b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb3fff700 1 -- 172.21.15.45:0/3240209688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 msgr2=0x5614b9092810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb3fff700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b9092810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb3fff700 1 -- 172.21.15.45:0/3240209688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614b8d93c60 con 0x5614b902bba0 2023-10-17T01:21:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.543+0000 7fafb37fe700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b908b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:21:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7fafb3fff700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b908b2e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fafa400f3e0 tx=0x7fafa400a670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f975b064d54c7e37 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7faf937fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7faf937fe700 1 -- 172.21.15.45:0/3240209688 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004040 con 0x5614b902bba0 2023-10-17T01:21:02.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7faf937fe700 1 -- 172.21.15.45:0/3240209688 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fafa40041a0 con 0x5614b902bba0 2023-10-17T01:21:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafa4004830 con 0x5614b902bba0 2023-10-17T01:21:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7faf937fe700 1 -- 172.21.15.45:0/3240209688 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafa4004360 con 0x5614b902bba0 2023-10-17T01:21:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7fafce162b80 10 client.26110 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.544+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5614b8da92b0 con 0x5614b902bba0 2023-10-17T01:21:02.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7faf937fe700 1 -- 172.21.15.45:0/3240209688 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafa4004830 con 0x5614b902bba0 2023-10-17T01:21:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7faf937fe700 1 -- 172.21.15.45:0/3240209688 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fafa4003950 con 0x5614b902bba0 2023-10-17T01:21:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafce162b80 10 client.26110 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafce162b80 10 client.26110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafce162b80 20 client.26110 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafce162b80 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b90949e0 0x5614b9096e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafce162b80 4 client.26110 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:21:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614b8f75d40 con 0x5614b90949e0 2023-10-17T01:21:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.545+0000 7fafb37fe700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b90949e0 0x5614b9096e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7fafb37fe700 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b90949e0 0x5614b9096e00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7161e46dde63094 server_cookie=901a024476b014a8 in_seq=0 out_seq=0 2023-10-17T01:21:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7faf937fe700 10 client.26110 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7faf937fe700 1 -- 172.21.15.45:0/3240209688 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5614b8f75d40 con 0x5614b90949e0 2023-10-17T01:21:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7faf937fe700 10 client.26110 handle_command_reply: tid=0 2023-10-17T01:21:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7fafce162b80 1 client.26110 shutdown 2023-10-17T01:21:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 msgr2=0x5614b908b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.546+0000 7fafce162b80 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b908b2e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fafa400f3e0 tx=0x7fafa400a670 comp rx=0 tx=0).stop 2023-10-17T01:21:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 shutdown_connections 2023-10-17T01:21:02.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614b902d2b0 0x5614b9092810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5614b902bba0 0x5614b908b2e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5614b90949e0 0x5614b9096e00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 --2- 172.21.15.45:0/3240209688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5614b902c500 0x5614b908b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7faf937fe700 0 client.26110 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:02.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 >> 172.21.15.45:0/3240209688 conn(0x5614b8f8b0d0 msgr2=0x5614b9026020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 shutdown_connections 2023-10-17T01:21:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 1 -- 172.21.15.45:0/3240209688 wait complete. 2023-10-17T01:21:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.547+0000 7fafce162b80 20 client.26110 trim_cache size 0 max 16384 2023-10-17T01:21:02.559 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:21:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:21:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:21:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:21:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:21:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:21:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:21:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:21:02.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:21:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:21:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faf9c0a0ba0 msgr2=0x7faf9c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faf9c0a0ba0 0x7faf9c0a3060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5614b902b940 tx=0x7fafa8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 msgr2=0x7fafb41a4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 0x7fafb41a4540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fafac00cc50 tx=0x7fafac00cf60 comp rx=0 tx=0).stop 2023-10-17T01:21:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 shutdown_connections 2023-10-17T01:21:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7faf9c0a0ba0 0x7faf9c0a3060 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafb4104ab0 0x7fafb41a4540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafb4102f20 0x7fafb419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 --2- 172.21.15.45:0/1656813303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb41040f0 0x7fafb419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:02.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 >> 172.21.15.45:0/1656813303 conn(0x7fafb40fe500 msgr2=0x7fafb40ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:02.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 shutdown_connections 2023-10-17T01:21:02.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:02.548+0000 7fafba7ff700 1 -- 172.21.15.45:0/1656813303 wait complete. 2023-10-17T01:21:02.581 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:21:03.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:02 smithi045 ceph-mon[98474]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 768 op/s 2023-10-17T01:21:03.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:02 smithi060 ceph-mon[134136]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 768 op/s 2023-10-17T01:21:03.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:02 smithi181 ceph-mon[134520]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 768 op/s 2023-10-17T01:21:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 -- 172.21.15.45:0/3354829827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc1041a0 msgr2=0x7fa3cc104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/3354829827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc104600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009920 tx=0x7fa3bc009c30 comp rx=0 tx=0).stop 2023-10-17T01:21:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 -- 172.21.15.45:0/3354829827 shutdown_connections 2023-10-17T01:21:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/3354829827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc104b40 0x7fa3cc10b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/3354829827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc1033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/3354829827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc104600 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 -- 172.21.15.45:0/3354829827 >> 172.21.15.45:0/3354829827 conn(0x7fa3cc0fe500 msgr2=0x7fa3cc100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 -- 172.21.15.45:0/3354829827 shutdown_connections 2023-10-17T01:21:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.108+0000 7fa3d3dff700 1 -- 172.21.15.45:0/3354829827 wait complete. 2023-10-17T01:21:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.109+0000 7fa3d3dff700 1 Processor -- start 2023-10-17T01:21:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.109+0000 7fa3d3dff700 1 -- start start 2023-10-17T01:21:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d3dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d3dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d3dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104b40 0x7fa3cc19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d3dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc19a850 con 0x7fa3cc102fb0 2023-10-17T01:21:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d3dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc19a9c0 con 0x7fa3cc104b40 2023-10-17T01:21:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d3dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc19ab80 con 0x7fa3cc1041a0 2023-10-17T01:21:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d139a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34438/0 (socket says 172.21.15.45:34438) 2023-10-17T01:21:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/681657050 learned_addr learned my addr 172.21.15.45:0/681657050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.110+0000 7fa3d239c700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104b40 0x7fa3cc19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d139a700 1 -- 172.21.15.45:0/681657050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104b40 msgr2=0x7fa3cc19a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d139a700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104b40 0x7fa3cc19a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d139a700 1 -- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 msgr2=0x7fa3cc198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d139a700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d139a700 1 -- 172.21.15.45:0/681657050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc009520 con 0x7fa3cc1041a0 2023-10-17T01:21:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.111+0000 7fa3d239c700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104b40 0x7fa3cc19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:21:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.112+0000 7fa3d139a700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc199410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc000c00 tx=0x7fa3bc00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.112+0000 7fa3c2ffd700 1 -- 172.21.15.45:0/681657050 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc01c070 con 0x7fa3cc1041a0 2023-10-17T01:21:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.112+0000 7fa3c2ffd700 1 -- 172.21.15.45:0/681657050 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3bc00f7d0 con 0x7fa3cc1041a0 2023-10-17T01:21:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.112+0000 7fa3c2ffd700 1 -- 172.21.15.45:0/681657050 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc003920 con 0x7fa3cc1041a0 2023-10-17T01:21:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.113+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc19ae00 con 0x7fa3cc1041a0 2023-10-17T01:21:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.113+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc1a1580 con 0x7fa3cc1041a0 2023-10-17T01:21:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.120+0000 7fa3c2ffd700 1 -- 172.21.15.45:0/681657050 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa3bc020030 con 0x7fa3cc1041a0 2023-10-17T01:21:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.121+0000 7fa3e77acb80 1 Processor -- start 2023-10-17T01:21:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.121+0000 7fa3e77acb80 1 -- start start 2023-10-17T01:21:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.121+0000 7fa3c2ffd700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa3b80a0ba0 0x7fa3b80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3c2ffd700 1 -- 172.21.15.45:0/681657050 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa3bc0cd100 con 0x7fa3cc1041a0 2023-10-17T01:21:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91eb2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91eb3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91e0a9b0 con 0x559a91eb2830 2023-10-17T01:21:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91e08370 con 0x559a91eb3d90 2023-10-17T01:21:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91e03c60 con 0x559a91eb3190 2023-10-17T01:21:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d239c700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa3b80a0ba0 0x7fa3b80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91eb2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91eb2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34446/0 (socket says 172.21.15.45:34446) 2023-10-17T01:21:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/1690999251 learned_addr learned my addr 172.21.15.45:0/1690999251 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d139a700 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91eb3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/1690999251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 msgr2=0x559a91eb3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91eb3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/1690999251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 msgr2=0x559a91eb4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.122+0000 7fa3d239c700 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/1690999251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a91c31d80 con 0x559a91eb2830 2023-10-17T01:21:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3d239c700 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:21:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91eb2c50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8011650 tx=0x7fa3c8011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534846a18f0a6b35 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3d239c700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa3b80a0ba0 0x7fa3b80a3060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x559a91eb4750 tx=0x7fa3c4009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3b7fff700 1 -- 172.21.15.45:0/1690999251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8004830 con 0x559a91eb2830 2023-10-17T01:21:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3b7fff700 1 -- 172.21.15.45:0/1690999251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3c8004e50 con 0x559a91eb2830 2023-10-17T01:21:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.123+0000 7fa3b7fff700 1 -- 172.21.15.45:0/1690999251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800f770 con 0x559a91eb2830 2023-10-17T01:21:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 -- 172.21.15.45:0/1690999251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 msgr2=0x559a91eb2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91eb2c50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8011650 tx=0x7fa3c8011960 comp rx=0 tx=0).stop 2023-10-17T01:21:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 -- 172.21.15.45:0/1690999251 shutdown_connections 2023-10-17T01:21:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91eb2c50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/1690999251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91eb3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 -- 172.21.15.45:0/1690999251 >> 172.21.15.45:0/1690999251 conn(0x559a91e12070 msgr2=0x559a91ead2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 -- 172.21.15.45:0/1690999251 shutdown_connections 2023-10-17T01:21:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.124+0000 7fa3e77acb80 1 -- 172.21.15.45:0/1690999251 wait complete. 2023-10-17T01:21:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 -- start start 2023-10-17T01:21:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 Processor -- start 2023-10-17T01:21:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91ec4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91ec4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91ecc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91da04b0 con 0x559a91eb2830 2023-10-17T01:21:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91da0150 con 0x559a91eb3d90 2023-10-17T01:21:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91c878c0 con 0x559a91eb3190 2023-10-17T01:21:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.125+0000 7fa3e77acb80 10 client.0 _ll_register_callbacks cb 0x559a91ceda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.126+0000 7fa3d139a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91ec4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.126+0000 7fa3d139a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91ec4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46018/0 (socket says 172.21.15.45:46018) 2023-10-17T01:21:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.126+0000 7fa3d139a700 1 -- 172.21.15.45:0/3423470354 learned_addr learned my addr 172.21.15.45:0/3423470354 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.126+0000 7fa3d239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91ecc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.127+0000 7fa3d139a700 1 -- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 msgr2=0x559a91ecc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.127+0000 7fa3d139a700 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91ecc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.127+0000 7fa3d139a700 1 -- 172.21.15.45:0/3423470354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 msgr2=0x559a91ec4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.126+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91ec4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.127+0000 7fa3d139a700 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91ec4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.127+0000 7fa3d139a700 1 -- 172.21.15.45:0/3423470354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c8009e40 con 0x559a91eb3190 2023-10-17T01:21:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.128+0000 7fa3d139a700 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91ec4fc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00fa40 tx=0x7fa3bc09a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca223d71f35a78c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.128+0000 7fa3b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.128+0000 7fa3b57fa700 1 -- 172.21.15.45:0/3423470354 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc09e5e0 con 0x559a91eb3190 2023-10-17T01:21:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.128+0000 7fa3b57fa700 1 -- 172.21.15.45:0/3423470354 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa3bc0a2790 con 0x559a91eb3190 2023-10-17T01:21:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.128+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a91e2ecb0 con 0x559a91eb3190 2023-10-17T01:21:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.129+0000 7fa3b57fa700 1 -- 172.21.15.45:0/3423470354 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3bc096ec0 con 0x559a91eb3190 2023-10-17T01:21:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.129+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91ec4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:21:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.129+0000 7fa3e77acb80 10 client.26122 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.129+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559a91c1ac00 con 0x559a91eb3190 2023-10-17T01:21:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.129+0000 7fa3b57fa700 1 -- 172.21.15.45:0/3423470354 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a91e2ecb0 con 0x559a91eb3190 2023-10-17T01:21:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.129+0000 7fa3b57fa700 1 -- 172.21.15.45:0/3423470354 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa3bc003a80 con 0x559a91eb3190 2023-10-17T01:21:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3e77acb80 10 client.26122 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3e77acb80 10 client.26122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3e77acb80 20 client.26122 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91ed0430 0x559a91ed2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3e77acb80 4 client.26122 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a91e81cd0 con 0x559a91ed0430 2023-10-17T01:21:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.130+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91ed0430 0x559a91ed2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.131+0000 7fa3b57fa700 10 client.26122 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.131+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91ed0430 0x559a91ed2850 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7587f15a89ced476 server_cookie=2dee9883df045492 in_seq=0 out_seq=0 2023-10-17T01:21:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.132+0000 7fa3b57fa700 1 -- 172.21.15.45:0/3423470354 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x559a91e81cd0 con 0x559a91ed0430 2023-10-17T01:21:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.132+0000 7fa3b57fa700 10 client.26122 handle_command_reply: tid=0 2023-10-17T01:21:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.132+0000 7fa3e77acb80 1 client.26122 shutdown 2023-10-17T01:21:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 msgr2=0x559a91ec4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91ec4fc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00fa40 tx=0x7fa3bc09a1c0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 shutdown_connections 2023-10-17T01:21:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91ed0430 0x559a91ed2850 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91ecc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb2830 0x559a91ec4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3423470354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3190 0x559a91ec4fc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3b57fa700 0 client.26122 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 >> 172.21.15.45:0/3423470354 conn(0x559a91e12070 msgr2=0x559a91eaca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 shutdown_connections 2023-10-17T01:21:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3423470354 wait complete. 2023-10-17T01:21:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.133+0000 7fa3e77acb80 20 client.26122 trim_cache size 0 max 16384 2023-10-17T01:21:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 Processor -- start 2023-10-17T01:21:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 -- start start 2023-10-17T01:21:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 0x559a91ed2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91e2ece0 con 0x559a91eb3190 2023-10-17T01:21:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91ec3db0 con 0x559a91eb3d90 2023-10-17T01:21:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91ec3f30 con 0x559a91eb2830 2023-10-17T01:21:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3d239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 0x559a91ed2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 0x559a91ed2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46022/0 (socket says 172.21.15.45:46022) 2023-10-17T01:21:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3d239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48878/0 (socket says 172.21.15.45:48878) 2023-10-17T01:21:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.154+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/3323565771 learned_addr learned my addr 172.21.15.45:0/3323565771 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d139a700 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/3323565771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 msgr2=0x559a91eb6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/3323565771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 msgr2=0x559a91ec3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/3323565771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a91c581f0 con 0x559a91eb2830 2023-10-17T01:21:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.155+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 0x559a91ed2820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8011fd0 tx=0x7fa3c8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a7cab6eb3c41a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.156+0000 7fa3c0bff700 1 -- 172.21.15.45:0/3323565771 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8028150 con 0x559a91eb2830 2023-10-17T01:21:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.156+0000 7fa3c0bff700 1 -- 172.21.15.45:0/3323565771 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3c802c070 con 0x559a91eb2830 2023-10-17T01:21:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.156+0000 7fa3c0bff700 1 -- 172.21.15.45:0/3323565771 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80038e0 con 0x559a91eb2830 2023-10-17T01:21:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.156+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3323565771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 msgr2=0x559a91ed2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.156+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 0x559a91ed2820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8011fd0 tx=0x7fa3c8000bc0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3323565771 shutdown_connections 2023-10-17T01:21:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb3d90 0x559a91eb6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb2830 0x559a91ed2820 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/3323565771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3323565771 >> 172.21.15.45:0/3323565771 conn(0x559a91e12070 msgr2=0x559a91eba100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3323565771 shutdown_connections 2023-10-17T01:21:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 -- 172.21.15.45:0/3323565771 wait complete. 2023-10-17T01:21:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 -- start start 2023-10-17T01:21:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.157+0000 7fa3e77acb80 1 Processor -- start 2023-10-17T01:21:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 0x559a91ec81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3d90 0x559a91ec90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91ec9620 con 0x559a91eb3190 2023-10-17T01:21:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91ec9790 con 0x559a91eb2830 2023-10-17T01:21:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a91ec9900 con 0x559a91eb3d90 2023-10-17T01:21:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3e77acb80 10 client.0 _ll_register_callbacks cb 0x559a91ceda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 0x559a91ec81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3d1b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 0x559a91ec81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48888/0 (socket says 172.21.15.45:48888) 2023-10-17T01:21:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/2378943234 learned_addr learned my addr 172.21.15.45:0/2378943234 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3d239c700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3d90 0x559a91ec90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.158+0000 7fa3d139a700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/2378943234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3d90 msgr2=0x559a91ec90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3d90 0x559a91ec90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/2378943234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 msgr2=0x559a91ec8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3d1b9b700 1 -- 172.21.15.45:0/2378943234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a91c31d80 con 0x559a91eb2830 2023-10-17T01:21:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3d1b9b700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 0x559a91ec81f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa3c801b040 tx=0x7fa3c8005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c9d737f37d4d0c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:21:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3b57fa700 1 -- 172.21.15.45:0/2378943234 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8028210 con 0x559a91eb2830 2023-10-17T01:21:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3c8004e50 con 0x559a91eb2830 2023-10-17T01:21:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.159+0000 7fa3b57fa700 1 -- 172.21.15.45:0/2378943234 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa3c80288b0 con 0x559a91eb2830 2023-10-17T01:21:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3b57fa700 1 -- 172.21.15.45:0/2378943234 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c8028370 con 0x559a91eb2830 2023-10-17T01:21:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 10 client.26147 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559a91c30250 con 0x559a91eb2830 2023-10-17T01:21:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3b57fa700 1 -- 172.21.15.45:0/2378943234 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3c8004e50 con 0x559a91eb2830 2023-10-17T01:21:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3b57fa700 1 -- 172.21.15.45:0/2378943234 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa3c800f480 con 0x559a91eb2830 2023-10-17T01:21:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 10 client.26147 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 20 client.26147 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91f229e0 0x559a91f24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 4 client.26147 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:21:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.160+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559a91dfcce0 con 0x559a91f229e0 2023-10-17T01:21:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.161+0000 7fa3d139a700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91f229e0 0x559a91f24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.162+0000 7fa3d139a700 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91f229e0 0x559a91f24e00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40cddd6613ce7a8e server_cookie=87808040ecc8ac53 in_seq=0 out_seq=0 2023-10-17T01:21:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.162+0000 7fa3b57fa700 10 client.26147 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3b57fa700 1 -- 172.21.15.45:0/2378943234 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559a91dfcce0 con 0x559a91f229e0 2023-10-17T01:21:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3b57fa700 10 client.26147 handle_command_reply: tid=0 2023-10-17T01:21:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 client.26147 shutdown 2023-10-17T01:21:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 msgr2=0x559a91ec81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 0x559a91ec81f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa3c801b040 tx=0x7fa3c8005700 comp rx=0 tx=0).stop 2023-10-17T01:21:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 shutdown_connections 2023-10-17T01:21:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a91eb3d90 0x559a91ec90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559a91f229e0 0x559a91f24e00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a91eb2830 0x559a91ec81f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 --2- 172.21.15.45:0/2378943234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a91eb3190 0x559a91ec8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3b57fa700 0 client.26147 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 >> 172.21.15.45:0/2378943234 conn(0x559a91e12070 msgr2=0x559a91eb6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 shutdown_connections 2023-10-17T01:21:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.179+0000 7fa3e77acb80 1 -- 172.21.15.45:0/2378943234 wait complete. 2023-10-17T01:21:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3e77acb80 20 client.26147 trim_cache size 0 max 16384 2023-10-17T01:21:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:21:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:21:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:21:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:21:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa3b80a0ba0 msgr2=0x7fa3b80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa3b80a0ba0 0x7fa3b80a3060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x559a91eb4750 tx=0x7fa3c4009920 comp rx=0 tx=0).stop 2023-10-17T01:21:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1041a0 msgr2=0x7fa3cc199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc199410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc000c00 tx=0x7fa3bc00b790 comp rx=0 tx=0).stop 2023-10-17T01:21:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 shutdown_connections 2023-10-17T01:21:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa3b80a0ba0 0x7fa3b80a3060 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3cc104b40 0x7fa3cc19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc102fb0 0x7fa3cc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 --2- 172.21.15.45:0/681657050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3cc1041a0 0x7fa3cc199410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.180+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 >> 172.21.15.45:0/681657050 conn(0x7fa3cc0fe500 msgr2=0x7fa3cc107780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.181+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 shutdown_connections 2023-10-17T01:21:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:05.181+0000 7fa3d3dff700 1 -- 172.21.15.45:0/681657050 wait complete. 2023-10-17T01:21:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 355.0 s, 1584.8 tps, lat 64.239 ms stddev 84.616 2023-10-17T01:21:05.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:05 smithi181 ceph-mon[134520]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 847 op/s 2023-10-17T01:21:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:05 smithi045 ceph-mon[98474]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 847 op/s 2023-10-17T01:21:06.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:05 smithi060 ceph-mon[134136]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 847 op/s 2023-10-17T01:21:08.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:07 smithi045 ceph-mon[98474]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 747 op/s 2023-10-17T01:21:08.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:07 smithi060 ceph-mon[134136]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 747 op/s 2023-10-17T01:21:08.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:07 smithi181 ceph-mon[134520]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 747 op/s 2023-10-17T01:21:10.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:09 smithi045 ceph-mon[98474]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 746 op/s 2023-10-17T01:21:10.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:09 smithi060 ceph-mon[134136]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 746 op/s 2023-10-17T01:21:10.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:09 smithi181 ceph-mon[134520]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 746 op/s 2023-10-17T01:21:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 360.0 s, 2032.8 tps, lat 49.153 ms stddev 56.022 2023-10-17T01:21:12.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:11 smithi045 ceph-mon[98474]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 855 op/s 2023-10-17T01:21:12.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:11 smithi060 ceph-mon[134136]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 855 op/s 2023-10-17T01:21:12.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:11 smithi181 ceph-mon[134520]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 855 op/s 2023-10-17T01:21:13.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:12 smithi045 ceph-mon[98474]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 MiB/s wr, 769 op/s 2023-10-17T01:21:13.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:12 smithi060 ceph-mon[134136]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 MiB/s wr, 769 op/s 2023-10-17T01:21:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:12 smithi181 ceph-mon[134520]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 MiB/s wr, 769 op/s 2023-10-17T01:21:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 365.0 s, 1783.4 tps, lat 55.884 ms stddev 66.585 2023-10-17T01:21:15.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:15 smithi181 ceph-mon[134520]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 845 op/s 2023-10-17T01:21:16.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:15 smithi045 ceph-mon[98474]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 845 op/s 2023-10-17T01:21:16.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:15 smithi060 ceph-mon[134136]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 845 op/s 2023-10-17T01:21:18.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:17 smithi045 ceph-mon[98474]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 713 op/s 2023-10-17T01:21:18.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:17 smithi060 ceph-mon[134136]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 713 op/s 2023-10-17T01:21:18.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:17 smithi181 ceph-mon[134520]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 713 op/s 2023-10-17T01:21:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 -- 172.21.15.45:0/1720394568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c104b40 msgr2=0x7fdc1c10b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1720394568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c104b40 0x7fdc1c10b3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009920 tx=0x7fdc0c009c30 comp rx=0 tx=0).stop 2023-10-17T01:21:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 -- 172.21.15.45:0/1720394568 shutdown_connections 2023-10-17T01:21:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1720394568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c104b40 0x7fdc1c10b3f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1720394568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c1033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1720394568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c1041a0 0x7fdc1c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.412+0000 7fdc209ff700 1 -- 172.21.15.45:0/1720394568 >> 172.21.15.45:0/1720394568 conn(0x7fdc1c0fe500 msgr2=0x7fdc1c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.413+0000 7fdc209ff700 1 -- 172.21.15.45:0/1720394568 shutdown_connections 2023-10-17T01:21:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.413+0000 7fdc209ff700 1 -- 172.21.15.45:0/1720394568 wait complete. 2023-10-17T01:21:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 Processor -- start 2023-10-17T01:21:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 -- start start 2023-10-17T01:21:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1041a0 0x7fdc1c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c104b40 0x7fdc1c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c19a820 con 0x7fdc1c1041a0 2023-10-17T01:21:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c19a990 con 0x7fdc1c102fb0 2023-10-17T01:21:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.414+0000 7fdc209ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c19ab50 con 0x7fdc1c104b40 2023-10-17T01:21:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.415+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.415+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48568/0 (socket says 172.21.15.45:48568) 2023-10-17T01:21:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.415+0000 7fdc1a59c700 1 -- 172.21.15.45:0/1038045088 learned_addr learned my addr 172.21.15.45:0/1038045088 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.415+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1041a0 0x7fdc1c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.416+0000 7fdc1a59c700 1 -- 172.21.15.45:0/1038045088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c104b40 msgr2=0x7fdc1c19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.416+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c104b40 0x7fdc1c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.416+0000 7fdc1a59c700 1 -- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1041a0 msgr2=0x7fdc1c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.416+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1041a0 0x7fdc1c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.416+0000 7fdc1a59c700 1 -- 172.21.15.45:0/1038045088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c009520 con 0x7fdc1c102fb0 2023-10-17T01:21:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.421+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c198e30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400d950 tx=0x7fdc1400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.422+0000 7fdc0b7fe700 1 -- 172.21.15.45:0/1038045088 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14009930 con 0x7fdc1c102fb0 2023-10-17T01:21:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.422+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c19ae30 con 0x7fdc1c102fb0 2023-10-17T01:21:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.422+0000 7fdc0b7fe700 1 -- 172.21.15.45:0/1038045088 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc14010460 con 0x7fdc1c102fb0 2023-10-17T01:21:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.422+0000 7fdc0b7fe700 1 -- 172.21.15.45:0/1038045088 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14009930 con 0x7fdc1c102fb0 2023-10-17T01:21:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.422+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c1a1670 con 0x7fdc1c102fb0 2023-10-17T01:21:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.426+0000 7fdc0b7fe700 1 -- 172.21.15.45:0/1038045088 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdc1401c400 con 0x7fdc1c102fb0 2023-10-17T01:21:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.428+0000 7fdc343c0b80 1 Processor -- start 2023-10-17T01:21:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.429+0000 7fdc0b7fe700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc040a0c80 0x7fdc040a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.429+0000 7fdc0b7fe700 1 -- 172.21.15.45:0/1038045088 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc140cbdd0 con 0x7fdc1c102fb0 2023-10-17T01:21:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.429+0000 7fdc343c0b80 1 -- start start 2023-10-17T01:21:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc040a0c80 0x7fdc040a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8190 0x563a76fe8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76fe9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76f3f9b0 con 0x563a76fe8d90 2023-10-17T01:21:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76f3d370 con 0x563a76fe7830 2023-10-17T01:21:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76f38c60 con 0x563a76fe8190 2023-10-17T01:21:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc19d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8190 0x563a76fe8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48576/0 (socket says 172.21.15.45:48576) 2023-10-17T01:21:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3240085591 learned_addr learned my addr 172.21.15.45:0/3240085591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.430+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76fe9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3240085591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8190 msgr2=0x563a76fe8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8190 0x563a76fe8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3240085591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 msgr2=0x563a76fe9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76fe9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc040a0c80 0x7fdc040a3140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009230 tx=0x7fdc0c00f480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3240085591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a76d66d80 con 0x563a76fe7830 2023-10-17T01:21:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.431+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe7c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc140a0f30 tx=0x7fdc14098fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c162d2c17ba09b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.432+0000 7fdc08ff9700 1 -- 172.21.15.45:0/3240085591 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140994a0 con 0x563a76fe7830 2023-10-17T01:21:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.432+0000 7fdc08ff9700 1 -- 172.21.15.45:0/3240085591 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1409d220 con 0x563a76fe7830 2023-10-17T01:21:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.432+0000 7fdc08ff9700 1 -- 172.21.15.45:0/3240085591 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14010820 con 0x563a76fe7830 2023-10-17T01:21:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3240085591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 msgr2=0x563a76fe7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe7c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc140a0f30 tx=0x7fdc14098fe0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3240085591 shutdown_connections 2023-10-17T01:21:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76fe9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe7c50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3240085591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8190 0x563a76fe8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3240085591 >> 172.21.15.45:0/3240085591 conn(0x563a76f47070 msgr2=0x563a76fe22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.433+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3240085591 shutdown_connections 2023-10-17T01:21:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.434+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3240085591 wait complete. 2023-10-17T01:21:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.434+0000 7fdc343c0b80 1 -- start start 2023-10-17T01:21:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.435+0000 7fdc343c0b80 1 Processor -- start 2023-10-17T01:21:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a76ff1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a76ff1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76ff89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76ed54b0 con 0x563a76fe8d90 2023-10-17T01:21:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76ed5150 con 0x563a76fe8190 2023-10-17T01:21:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76dbc8c0 con 0x563a76fe7830 2023-10-17T01:21:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a76ff1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a76ff1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46540/0 (socket says 172.21.15.45:46540) 2023-10-17T01:21:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3993838101 learned_addr learned my addr 172.21.15.45:0/3993838101 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.436+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a76ff1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.437+0000 7fdc343c0b80 10 client.0 _ll_register_callbacks cb 0x563a76e22a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.437+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76ff89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.437+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 msgr2=0x563a76ff1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.437+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a76ff1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3993838101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 msgr2=0x563a76ff89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76ff89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc1a59c700 1 -- 172.21.15.45:0/3993838101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c1a1670 con 0x563a76fe7830 2023-10-17T01:21:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a76ff1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76ff89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:21:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a76ff1420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdc140977f0 tx=0x7fdc1409d120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5590e4f9e37d19 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc01ffb700 1 -- 172.21.15.45:0/3993838101 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140994a0 con 0x563a76fe7830 2023-10-17T01:21:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc01ffb700 1 -- 172.21.15.45:0/3993838101 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc140c9920 con 0x563a76fe7830 2023-10-17T01:21:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a76f63cb0 con 0x563a76fe7830 2023-10-17T01:21:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc01ffb700 1 -- 172.21.15.45:0/3993838101 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc14098990 con 0x563a76fe7830 2023-10-17T01:21:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc343c0b80 10 client.26137 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.438+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563a76d4fc00 con 0x563a76fe7830 2023-10-17T01:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.439+0000 7fdc01ffb700 1 -- 172.21.15.45:0/3993838101 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a76f63cb0 con 0x563a76fe7830 2023-10-17T01:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.439+0000 7fdc01ffb700 1 -- 172.21.15.45:0/3993838101 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fdc140b1b90 con 0x563a76fe7830 2023-10-17T01:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.439+0000 7fdc343c0b80 10 client.26137 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.439+0000 7fdc343c0b80 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.440+0000 7fdc343c0b80 20 client.26137 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.440+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a76ffcd50 0x563a76fff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.440+0000 7fdc343c0b80 4 client.26137 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.440+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a76fb6cd0 con 0x563a76ffcd50 2023-10-17T01:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.440+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a76ffcd50 0x563a76fff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.441+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a76ffcd50 0x563a76fff170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813ff236156ff5bb server_cookie=397510f50385040e in_seq=0 out_seq=0 2023-10-17T01:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.441+0000 7fdc01ffb700 10 client.26137 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc01ffb700 1 -- 172.21.15.45:0/3993838101 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x563a76fb6cd0 con 0x563a76ffcd50 2023-10-17T01:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc01ffb700 10 client.26137 handle_command_reply: tid=0 2023-10-17T01:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 client.26137 shutdown 2023-10-17T01:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 msgr2=0x563a76ff1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a76ff1420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdc140977f0 tx=0x7fdc1409d120 comp rx=0 tx=0).stop 2023-10-17T01:21:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 shutdown_connections 2023-10-17T01:21:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a76ff89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a76ff1420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a76ffcd50 0x563a76fff170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/3993838101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a76ff1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 >> 172.21.15.45:0/3993838101 conn(0x563a76f47070 msgr2=0x563a76fe1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc01ffb700 0 client.26137 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 shutdown_connections 2023-10-17T01:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.442+0000 7fdc343c0b80 1 -- 172.21.15.45:0/3993838101 wait complete. 2023-10-17T01:21:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.443+0000 7fdc343c0b80 20 client.26137 trim_cache size 0 max 16384 2023-10-17T01:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.471+0000 7fdc343c0b80 1 Processor -- start 2023-10-17T01:21:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.471+0000 7fdc343c0b80 1 -- start start 2023-10-17T01:21:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8190 0x563a76ff0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 0x563a76ff0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76ff2610 con 0x563a76fe8190 2023-10-17T01:21:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76ff08d0 con 0x563a76fe7830 2023-10-17T01:21:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a76fe85c0 con 0x563a76fe8d90 2023-10-17T01:21:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 0x563a76ff0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 0x563a76ff0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46546/0 (socket says 172.21.15.45:46546) 2023-10-17T01:21:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48590/0 (socket says 172.21.15.45:48590) 2023-10-17T01:21:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 -- 172.21.15.45:0/24093696 learned_addr learned my addr 172.21.15.45:0/24093696 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8190 0x563a76ff0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 -- 172.21.15.45:0/24093696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 msgr2=0x563a76fe3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 -- 172.21.15.45:0/24093696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8190 msgr2=0x563a76ff0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8190 0x563a76ff0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.472+0000 7fdc1ad9d700 1 -- 172.21.15.45:0/24093696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a76d8d1f0 con 0x563a76fe8d90 2023-10-17T01:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.473+0000 7fdc1ad9d700 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 0x563a76ff0060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c016000 tx=0x7fdc0c024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b2c849f22be5f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.473+0000 7fdc097fa700 1 -- 172.21.15.45:0/24093696 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c01c040 con 0x563a76fe8d90 2023-10-17T01:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.473+0000 7fdc097fa700 1 -- 172.21.15.45:0/24093696 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0c00f950 con 0x563a76fe8d90 2023-10-17T01:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.473+0000 7fdc097fa700 1 -- 172.21.15.45:0/24093696 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c004a50 con 0x563a76fe8d90 2023-10-17T01:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.474+0000 7fdc343c0b80 1 -- 172.21.15.45:0/24093696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 msgr2=0x563a76ff0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.474+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 0x563a76ff0060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c016000 tx=0x7fdc0c024000 comp rx=0 tx=0).stop 2023-10-17T01:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 -- 172.21.15.45:0/24093696 shutdown_connections 2023-10-17T01:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe8d90 0x563a76ff0060 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe7830 0x563a76fe3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/24093696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8190 0x563a76ff0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 -- 172.21.15.45:0/24093696 >> 172.21.15.45:0/24093696 conn(0x563a76f47070 msgr2=0x563a76fe1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 -- 172.21.15.45:0/24093696 shutdown_connections 2023-10-17T01:21:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.475+0000 7fdc343c0b80 1 -- 172.21.15.45:0/24093696 wait complete. 2023-10-17T01:21:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.476+0000 7fdc343c0b80 1 -- start start 2023-10-17T01:21:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.477+0000 7fdc343c0b80 1 Processor -- start 2023-10-17T01:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.477+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a770486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a7704ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc343c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a7704d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7704d690 con 0x563a76fe8d90 2023-10-17T01:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7704d7d0 con 0x563a76fe8190 2023-10-17T01:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc343c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7704d990 con 0x563a76fe7830 2023-10-17T01:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc343c0b80 10 client.0 _ll_register_callbacks cb 0x7fdc0c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.478+0000 7fdc19d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a7704ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a770486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a770486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46556/0 (socket says 172.21.15.45:46556) 2023-10-17T01:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 -- 172.21.15.45:0/897086688 learned_addr learned my addr 172.21.15.45:0/897086688 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 -- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 msgr2=0x563a7704ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a7704ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 -- 172.21.15.45:0/897086688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 msgr2=0x563a7704d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a7704d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 -- 172.21.15.45:0/897086688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c10bd40 con 0x563a76fe7830 2023-10-17T01:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.479+0000 7fdc1a59c700 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a770486b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc1409d190 tx=0x7fdc14098340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e39ddf6595b794 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc01ffb700 1 -- 172.21.15.45:0/897086688 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14095640 con 0x563a76fe7830 2023-10-17T01:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc01ffb700 1 -- 172.21.15.45:0/897086688 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc14095400 con 0x563a76fe7830 2023-10-17T01:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc1409d220 con 0x563a76fe7830 2023-10-17T01:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc01ffb700 1 -- 172.21.15.45:0/897086688 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc14098fe0 con 0x563a76fe7830 2023-10-17T01:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc343c0b80 10 client.26143 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563a76d65250 con 0x563a76fe7830 2023-10-17T01:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.480+0000 7fdc01ffb700 1 -- 172.21.15.45:0/897086688 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc140a5020 con 0x563a76fe7830 2023-10-17T01:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc01ffb700 1 -- 172.21.15.45:0/897086688 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fdc14080030 con 0x563a76fe7830 2023-10-17T01:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc343c0b80 10 client.26143 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc343c0b80 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc343c0b80 20 client.26143 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a7704ecb0 0x563a770510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc343c0b80 4 client.26143 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a76f31ce0 con 0x563a7704ecb0 2023-10-17T01:21:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.481+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a7704ecb0 0x563a770510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.482+0000 7fdc19d9b700 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a7704ecb0 0x563a770510d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e01cd2b9884fec server_cookie=e6e4e82ce0fda156 in_seq=0 out_seq=0 2023-10-17T01:21:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.482+0000 7fdc01ffb700 10 client.26143 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.497+0000 7fdc01ffb700 1 -- 172.21.15.45:0/897086688 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a76f31ce0 con 0x563a7704ecb0 2023-10-17T01:21:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.497+0000 7fdc01ffb700 10 client.26143 handle_command_reply: tid=0 2023-10-17T01:21:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.497+0000 7fdc343c0b80 1 client.26143 shutdown 2023-10-17T01:21:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.497+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 msgr2=0x563a770486b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.497+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a770486b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc1409d190 tx=0x7fdc14098340 comp rx=0 tx=0).stop 2023-10-17T01:21:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 shutdown_connections 2023-10-17T01:21:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a76fe8d90 0x563a7704d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a76fe7830 0x563a770486b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a7704ecb0 0x563a770510d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 --2- 172.21.15.45:0/897086688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a76fe8190 0x563a7704ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 >> 172.21.15.45:0/897086688 conn(0x563a76f47070 msgr2=0x563a76ff91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc01ffb700 0 client.26143 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 shutdown_connections 2023-10-17T01:21:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.498+0000 7fdc343c0b80 1 -- 172.21.15.45:0/897086688 wait complete. 2023-10-17T01:21:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.499+0000 7fdc343c0b80 20 client.26143 trim_cache size 0 max 16384 2023-10-17T01:21:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:21:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:21:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:21:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:21:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc040a0c80 msgr2=0x7fdc040a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc040a0c80 0x7fdc040a3140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009230 tx=0x7fdc0c00f480 comp rx=0 tx=0).stop 2023-10-17T01:21:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 msgr2=0x7fdc1c198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c198e30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400d950 tx=0x7fdc1400dc60 comp rx=0 tx=0).stop 2023-10-17T01:21:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 shutdown_connections 2023-10-17T01:21:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc040a0c80 0x7fdc040a3140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c104b40 0x7fdc1c19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c102fb0 0x7fdc1c198e30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 --2- 172.21.15.45:0/1038045088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1041a0 0x7fdc1c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 >> 172.21.15.45:0/1038045088 conn(0x7fdc1c0fe500 msgr2=0x7fdc1c107660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.500+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 shutdown_connections 2023-10-17T01:21:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:18.501+0000 7fdc209ff700 1 -- 172.21.15.45:0/1038045088 wait complete. 2023-10-17T01:21:20.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:19 smithi045 ceph-mon[98474]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 987 op/s 2023-10-17T01:21:20.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:19 smithi060 ceph-mon[134136]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 987 op/s 2023-10-17T01:21:20.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:19 smithi181 ceph-mon[134520]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 987 op/s 2023-10-17T01:21:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 370.0 s, 1524.0 tps, lat 66.070 ms stddev 119.763 2023-10-17T01:21:21.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:20 smithi045 ceph-mon[98474]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2023-10-17T01:21:21.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:20 smithi060 ceph-mon[134136]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2023-10-17T01:21:21.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:20 smithi181 ceph-mon[134520]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2023-10-17T01:21:23.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:22 smithi045 ceph-mon[98474]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2023-10-17T01:21:23.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:22 smithi060 ceph-mon[134136]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2023-10-17T01:21:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:22 smithi181 ceph-mon[134520]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.90k op/s 2023-10-17T01:21:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 375.0 s, 1781.6 tps, lat 56.001 ms stddev 63.396 2023-10-17T01:21:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:25 smithi045 ceph-mon[98474]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.93k op/s 2023-10-17T01:21:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:25 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:21:26.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:25 smithi060 ceph-mon[134136]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.93k op/s 2023-10-17T01:21:26.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:25 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:21:26.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:25 smithi181 ceph-mon[134520]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.93k op/s 2023-10-17T01:21:26.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:25 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:21:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:21:27.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:21:27.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:21:27.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:26 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:21:27.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:26 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:21:27.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:26 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:21:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:21:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:21:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:21:28.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:27 smithi045 ceph-mon[98474]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.80k op/s 2023-10-17T01:21:28.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:27 smithi060 ceph-mon[134136]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.80k op/s 2023-10-17T01:21:28.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:27 smithi181 ceph-mon[134520]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.80k op/s 2023-10-17T01:21:29.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:28 smithi045 ceph-mon[98474]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2023-10-17T01:21:29.085 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:28 smithi060 ceph-mon[134136]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2023-10-17T01:21:29.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:28 smithi181 ceph-mon[134520]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2023-10-17T01:21:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 380.0 s, 1818.2 tps, lat 55.018 ms stddev 65.132 2023-10-17T01:21:31.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:30 smithi045 ceph-mon[98474]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.60k op/s 2023-10-17T01:21:31.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:30 smithi060 ceph-mon[134136]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.60k op/s 2023-10-17T01:21:31.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:30 smithi181 ceph-mon[134520]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.60k op/s 2023-10-17T01:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 -- 172.21.15.45:0/3508433752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4102910 msgr2=0x7fdce41091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/3508433752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4102910 0x7fdce41091c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009f90 tx=0x7fdcdc00b330 comp rx=0 tx=0).stop 2023-10-17T01:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 -- 172.21.15.45:0/3508433752 shutdown_connections 2023-10-17T01:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/3508433752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4102910 0x7fdce41091c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/3508433752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4100d80 0x7fdce41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/3508433752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4101f70 0x7fdce41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.736+0000 7fdceb7ff700 1 -- 172.21.15.45:0/3508433752 >> 172.21.15.45:0/3508433752 conn(0x7fdce40fc2d0 msgr2=0x7fdce40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.737+0000 7fdceb7ff700 1 -- 172.21.15.45:0/3508433752 shutdown_connections 2023-10-17T01:21:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.737+0000 7fdceb7ff700 1 -- 172.21.15.45:0/3508433752 wait complete. 2023-10-17T01:21:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.739+0000 7fdceb7ff700 1 Processor -- start 2023-10-17T01:21:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.739+0000 7fdceb7ff700 1 -- start start 2023-10-17T01:21:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdceb7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4100d80 0x7fdce4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdceb7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4101f70 0x7fdce41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdceb7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4102910 0x7fdce419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdceb7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce419a770 con 0x7fdce4102910 2023-10-17T01:21:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdceb7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce419a8e0 con 0x7fdce4100d80 2023-10-17T01:21:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdceb7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce419aaa0 con 0x7fdce4101f70 2023-10-17T01:21:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdce959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4100d80 0x7fdce4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdce959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4100d80 0x7fdce4198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42878/0 (socket says 172.21.15.45:42878) 2023-10-17T01:21:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.740+0000 7fdce959b700 1 -- 172.21.15.45:0/672319868 learned_addr learned my addr 172.21.15.45:0/672319868 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.741+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4102910 0x7fdce419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.742+0000 7fdce9d9c700 1 -- 172.21.15.45:0/672319868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4101f70 msgr2=0x7fdce41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.742+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4101f70 0x7fdce41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.742+0000 7fdce9d9c700 1 -- 172.21.15.45:0/672319868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4100d80 msgr2=0x7fdce4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.742+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4100d80 0x7fdce4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.742+0000 7fdce9d9c700 1 -- 172.21.15.45:0/672319868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc009c40 con 0x7fdce4102910 2023-10-17T01:21:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.742+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4102910 0x7fdce419a060 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009f90 tx=0x7fdcdc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.743+0000 7fdcda7fc700 1 -- 172.21.15.45:0/672319868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00e050 con 0x7fdce4102910 2023-10-17T01:21:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.743+0000 7fdcda7fc700 1 -- 172.21.15.45:0/672319868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcdc005570 con 0x7fdce4102910 2023-10-17T01:21:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.743+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce419ad20 con 0x7fdce4102910 2023-10-17T01:21:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.743+0000 7fdcda7fc700 1 -- 172.21.15.45:0/672319868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc01c9b0 con 0x7fdce4102910 2023-10-17T01:21:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.743+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce41a1490 con 0x7fdce4102910 2023-10-17T01:21:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.749+0000 7fdcda7fc700 1 -- 172.21.15.45:0/672319868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdcdc005040 con 0x7fdce4102910 2023-10-17T01:21:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.751+0000 7fdcff113b80 1 Processor -- start 2023-10-17T01:21:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 -- start start 2023-10-17T01:21:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcda7fc700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdcd00a0d20 0x7fdcd00a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcda7fc700 1 -- 172.21.15.45:0/672319868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdcdc0cd4c0 con 0x7fdce4102910 2023-10-17T01:21:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6129c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd612a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd612b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd60819b0 con 0x561bd6129830 2023-10-17T01:21:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd607f370 con 0x561bd612ad90 2023-10-17T01:21:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdcff113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd607ac60 con 0x561bd612a190 2023-10-17T01:21:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.752+0000 7fdce8d9a700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdcd00a0d20 0x7fdcd00a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd612a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6129c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd612a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55858/0 (socket says 172.21.15.45:55858) 2023-10-17T01:21:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce8d9a700 1 -- 172.21.15.45:0/2629047559 learned_addr learned my addr 172.21.15.45:0/2629047559 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6129c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60056/0 (socket says 172.21.15.45:60056) 2023-10-17T01:21:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd612b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 -- 172.21.15.45:0/2629047559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 msgr2=0x561bd612a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd612a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 -- 172.21.15.45:0/2629047559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 msgr2=0x561bd612b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd612b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.753+0000 7fdce959b700 1 -- 172.21.15.45:0/2629047559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd5ea8d80 con 0x561bd6129830 2023-10-17T01:21:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.754+0000 7fdce959b700 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6129c50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdce0005950 tx=0x7fdce0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a36dfb4e87e02f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.754+0000 7fdce8d9a700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdcd00a0d20 0x7fdcd00a31e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009fd0 tx=0x7fdcd4009b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.754+0000 7fdccf7fe700 1 -- 172.21.15.45:0/2629047559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011d80 con 0x561bd6129830 2023-10-17T01:21:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.754+0000 7fdccf7fe700 1 -- 172.21.15.45:0/2629047559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdce00044f0 con 0x561bd6129830 2023-10-17T01:21:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.754+0000 7fdccf7fe700 1 -- 172.21.15.45:0/2629047559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0005280 con 0x561bd6129830 2023-10-17T01:21:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.755+0000 7fdcff113b80 1 -- 172.21.15.45:0/2629047559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 msgr2=0x561bd6129c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.755+0000 7fdcff113b80 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6129c50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdce0005950 tx=0x7fdce0011440 comp rx=0 tx=0).stop 2023-10-17T01:21:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 -- 172.21.15.45:0/2629047559 shutdown_connections 2023-10-17T01:21:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd612b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6129c50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 --2- 172.21.15.45:0/2629047559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd612a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 -- 172.21.15.45:0/2629047559 >> 172.21.15.45:0/2629047559 conn(0x561bd6089070 msgr2=0x561bd61242c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 -- 172.21.15.45:0/2629047559 shutdown_connections 2023-10-17T01:21:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.756+0000 7fdcff113b80 1 -- 172.21.15.45:0/2629047559 wait complete. 2023-10-17T01:21:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.757+0000 7fdcff113b80 1 -- start start 2023-10-17T01:21:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 Processor -- start 2023-10-17T01:21:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd61332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd6133810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd613a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd60174b0 con 0x561bd6129830 2023-10-17T01:21:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6017150 con 0x561bd612ad90 2023-10-17T01:21:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.758+0000 7fdcff113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd5efe8c0 con 0x561bd612a190 2023-10-17T01:21:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.759+0000 7fdcff113b80 10 client.0 _ll_register_callbacks cb 0x561bd5f64a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.760+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd6133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.760+0000 7fdce959b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd61332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.760+0000 7fdce9d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd613a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.760+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd6133810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55866/0 (socket says 172.21.15.45:55866) 2023-10-17T01:21:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.760+0000 7fdce8d9a700 1 -- 172.21.15.45:0/3377137274 learned_addr learned my addr 172.21.15.45:0/3377137274 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.761+0000 7fdce8d9a700 1 -- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 msgr2=0x561bd613a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.761+0000 7fdce8d9a700 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd613a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.761+0000 7fdce8d9a700 1 -- 172.21.15.45:0/3377137274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 msgr2=0x561bd61332d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.761+0000 7fdce8d9a700 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd61332d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.761+0000 7fdce8d9a700 1 -- 172.21.15.45:0/3377137274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce0009970 con 0x561bd612a190 2023-10-17T01:21:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.761+0000 7fdce959b700 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd61332d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.762+0000 7fdce8d9a700 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd6133810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400e920 tx=0x7fdcd4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e3b363169fdb93 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.762+0000 7fdcccff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.764+0000 7fdcccff9700 1 -- 172.21.15.45:0/3377137274 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4018070 con 0x561bd612a190 2023-10-17T01:21:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.764+0000 7fdcccff9700 1 -- 172.21.15.45:0/3377137274 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdcd4020640 con 0x561bd612a190 2023-10-17T01:21:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.764+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bd60a5cb0 con 0x561bd612a190 2023-10-17T01:21:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.764+0000 7fdcccff9700 1 -- 172.21.15.45:0/3377137274 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcd4004020 con 0x561bd612a190 2023-10-17T01:21:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.764+0000 7fdcff113b80 10 client.26152 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.764+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561bd5e91c00 con 0x561bd612a190 2023-10-17T01:21:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.765+0000 7fdcccff9700 1 -- 172.21.15.45:0/3377137274 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bd60a5cb0 con 0x561bd612a190 2023-10-17T01:21:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.765+0000 7fdcccff9700 1 -- 172.21.15.45:0/3377137274 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fdcd40036c0 con 0x561bd612a190 2023-10-17T01:21:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.765+0000 7fdcff113b80 10 client.26152 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.765+0000 7fdcff113b80 10 client.26152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.765+0000 7fdcff113b80 20 client.26152 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.765+0000 7fdcff113b80 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd6150060 0x561bd6152480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.766+0000 7fdcff113b80 4 client.26152 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.766+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bd60f8cd0 con 0x561bd6150060 2023-10-17T01:21:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.766+0000 7fdce959b700 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd6150060 0x561bd6152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.767+0000 7fdcccff9700 10 client.26152 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.767+0000 7fdce959b700 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd6150060 0x561bd6152480 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5a15cdff357794 server_cookie=6e3f6b945263cfc2 in_seq=0 out_seq=0 2023-10-17T01:21:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.770+0000 7fdcccff9700 1 -- 172.21.15.45:0/3377137274 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x561bd60f8cd0 con 0x561bd6150060 2023-10-17T01:21:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.770+0000 7fdcccff9700 10 client.26152 handle_command_reply: tid=0 2023-10-17T01:21:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.770+0000 7fdcff113b80 1 client.26152 shutdown 2023-10-17T01:21:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 msgr2=0x561bd6133810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd6133810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400e920 tx=0x7fdcd4009310 comp rx=0 tx=0).stop 2023-10-17T01:21:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 shutdown_connections 2023-10-17T01:21:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd6150060 0x561bd6152480 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612ad90 0x561bd613a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd61332d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 --2- 172.21.15.45:0/3377137274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612a190 0x561bd6133810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 >> 172.21.15.45:0/3377137274 conn(0x561bd6089070 msgr2=0x561bd6123a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcccff9700 0 client.26152 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 shutdown_connections 2023-10-17T01:21:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.771+0000 7fdcff113b80 1 -- 172.21.15.45:0/3377137274 wait complete. 2023-10-17T01:21:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.772+0000 7fdcff113b80 20 client.26152 trim_cache size 0 max 16384 2023-10-17T01:21:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.792+0000 7fdcff113b80 1 Processor -- start 2023-10-17T01:21:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.792+0000 7fdcff113b80 1 -- start start 2023-10-17T01:21:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.793+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 0x561bd612b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.793+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd61311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.793+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd612ad90 0x561bd612dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.793+0000 7fdcff113b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6132780 con 0x561bd612ad90 2023-10-17T01:21:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.793+0000 7fdcff113b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6132e40 con 0x561bd612a190 2023-10-17T01:21:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.793+0000 7fdcff113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd612ac20 con 0x561bd6129830 2023-10-17T01:21:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce959b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 0x561bd612b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce959b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 0x561bd612b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55876/0 (socket says 172.21.15.45:55876) 2023-10-17T01:21:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce959b700 1 -- 172.21.15.45:0/657920652 learned_addr learned my addr 172.21.15.45:0/657920652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce8d9a700 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd61311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce959b700 1 -- 172.21.15.45:0/657920652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 msgr2=0x561bd61311a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce959b700 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd61311a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.794+0000 7fdce959b700 1 -- 172.21.15.45:0/657920652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd612ad90 msgr2=0x561bd612dcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:21:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.795+0000 7fdce959b700 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd612ad90 0x561bd612dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.795+0000 7fdce959b700 1 -- 172.21.15.45:0/657920652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd5ecf1f0 con 0x561bd6129830 2023-10-17T01:21:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.795+0000 7fdce959b700 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 0x561bd612b170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdce0005950 tx=0x7fdce00118c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0eeb10b299b457 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.795+0000 7fdccffff700 1 -- 172.21.15.45:0/657920652 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011ea0 con 0x561bd6129830 2023-10-17T01:21:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.796+0000 7fdccffff700 1 -- 172.21.15.45:0/657920652 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdce002a220 con 0x561bd6129830 2023-10-17T01:21:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.796+0000 7fdccffff700 1 -- 172.21.15.45:0/657920652 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0005630 con 0x561bd6129830 2023-10-17T01:21:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.796+0000 7fdcff113b80 1 -- 172.21.15.45:0/657920652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 msgr2=0x561bd612b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.796+0000 7fdcff113b80 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 0x561bd612b170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdce0005950 tx=0x7fdce00118c0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.796+0000 7fdcff113b80 1 -- 172.21.15.45:0/657920652 shutdown_connections 2023-10-17T01:21:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd612ad90 0x561bd612dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd6129830 0x561bd612b170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 --2- 172.21.15.45:0/657920652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd61311a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 -- 172.21.15.45:0/657920652 >> 172.21.15.45:0/657920652 conn(0x561bd6089070 msgr2=0x561bd6131ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 -- 172.21.15.45:0/657920652 shutdown_connections 2023-10-17T01:21:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 -- 172.21.15.45:0/657920652 wait complete. 2023-10-17T01:21:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 -- start start 2023-10-17T01:21:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.797+0000 7fdcff113b80 1 Processor -- start 2023-10-17T01:21:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6136a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd6136f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612ad90 0x561bd6137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6137ce0 con 0x561bd6129830 2023-10-17T01:21:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6137e50 con 0x561bd612a190 2023-10-17T01:21:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6138010 con 0x561bd612ad90 2023-10-17T01:21:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdcff113b80 10 client.0 _ll_register_callbacks cb 0x7fdcd400e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd6136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdce9d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612ad90 0x561bd6137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdce8d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd6136f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42914/0 (socket says 172.21.15.45:42914) 2023-10-17T01:21:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdce8d9a700 1 -- 172.21.15.45:0/1438139591 learned_addr learned my addr 172.21.15.45:0/1438139591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.798+0000 7fdce959b700 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdce9d9c700 1 -- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 msgr2=0x561bd6136f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd6136f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdce9d9c700 1 -- 172.21.15.45:0/1438139591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 msgr2=0x561bd6136a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6136a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdce9d9c700 1 -- 172.21.15.45:0/1438139591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bd5ea8d80 con 0x561bd612ad90 2023-10-17T01:21:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdce9d9c700 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612ad90 0x561bd6137600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc09a870 tx=0x7fdcdc01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6c0aa194ac59b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcccff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcccff9700 1 -- 172.21.15.45:0/1438139591 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc005970 con 0x561bd612ad90 2023-10-17T01:21:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdce00044f0 con 0x561bd612ad90 2023-10-17T01:21:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcccff9700 1 -- 172.21.15.45:0/1438139591 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdcdc09ab70 con 0x561bd612ad90 2023-10-17T01:21:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcccff9700 1 -- 172.21.15.45:0/1438139591 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcdc09e9a0 con 0x561bd612ad90 2023-10-17T01:21:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcff113b80 10 client.26158 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561bd5ea7250 con 0x561bd612ad90 2023-10-17T01:21:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.799+0000 7fdcccff9700 1 -- 172.21.15.45:0/1438139591 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdce00044f0 con 0x561bd612ad90 2023-10-17T01:21:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcccff9700 1 -- 172.21.15.45:0/1438139591 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fdcdc09a060 con 0x561bd612ad90 2023-10-17T01:21:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcff113b80 10 client.26158 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcff113b80 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcff113b80 20 client.26158 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcff113b80 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd61aad90 0x561bd61ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcff113b80 4 client.26158 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:21:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bd6073ce0 con 0x561bd61aad90 2023-10-17T01:21:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.800+0000 7fdce959b700 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd61aad90 0x561bd61ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.801+0000 7fdcccff9700 10 client.26158 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.801+0000 7fdce959b700 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd61aad90 0x561bd61ad1b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45b09fe9871db70 server_cookie=6c4a6b3962632272 in_seq=0 out_seq=0 2023-10-17T01:21:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcccff9700 1 -- 172.21.15.45:0/1438139591 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bd6073ce0 con 0x561bd61aad90 2023-10-17T01:21:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcccff9700 10 client.26158 handle_command_reply: tid=0 2023-10-17T01:21:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 client.26158 shutdown 2023-10-17T01:21:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612ad90 msgr2=0x561bd6137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612ad90 0x561bd6137600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc09a870 tx=0x7fdcdc01c400 comp rx=0 tx=0).stop 2023-10-17T01:21:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 shutdown_connections 2023-10-17T01:21:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561bd612ad90 0x561bd6137600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561bd6129830 0x561bd6136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561bd61aad90 0x561bd61ad1b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 --2- 172.21.15.45:0/1438139591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561bd612a190 0x561bd6136f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 >> 172.21.15.45:0/1438139591 conn(0x561bd6089070 msgr2=0x561bd61311a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcccff9700 0 client.26158 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 shutdown_connections 2023-10-17T01:21:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 1 -- 172.21.15.45:0/1438139591 wait complete. 2023-10-17T01:21:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.821+0000 7fdcff113b80 20 client.26158 trim_cache size 0 max 16384 2023-10-17T01:21:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:21:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:21:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:21:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:21:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdcd00a0d20 msgr2=0x7fdcd00a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdcd00a0d20 0x7fdcd00a31e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009fd0 tx=0x7fdcd4009b40 comp rx=0 tx=0).stop 2023-10-17T01:21:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4102910 msgr2=0x7fdce419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4102910 0x7fdce419a060 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009f90 tx=0x7fdcdc009b60 comp rx=0 tx=0).stop 2023-10-17T01:21:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 shutdown_connections 2023-10-17T01:21:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdcd00a0d20 0x7fdcd00a31e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce4102910 0x7fdce419a060 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce4100d80 0x7fdce4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 --2- 172.21.15.45:0/672319868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce4101f70 0x7fdce41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.822+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 >> 172.21.15.45:0/672319868 conn(0x7fdce40fc2d0 msgr2=0x7fdce40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.823+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 shutdown_connections 2023-10-17T01:21:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:31.823+0000 7fdceb7ff700 1 -- 172.21.15.45:0/672319868 wait complete. 2023-10-17T01:21:32.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:21:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 -- 172.21.15.45:0/3613214687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608100d40 msgr2=0x7f7608101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 --2- 172.21.15.45:0/3613214687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608100d40 0x7f7608101160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76040099c0 tx=0x7f7604009cd0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 -- 172.21.15.45:0/3613214687 shutdown_connections 2023-10-17T01:21:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 --2- 172.21.15.45:0/3613214687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 0x7f7608109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 --2- 172.21.15.45:0/3613214687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608100d40 0x7f7608101160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 --2- 172.21.15.45:0/3613214687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608101f30 0x7f7608102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 -- 172.21.15.45:0/3613214687 >> 172.21.15.45:0/3613214687 conn(0x7f76080fc290 msgr2=0x7f76080fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:32.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 -- 172.21.15.45:0/3613214687 shutdown_connections 2023-10-17T01:21:32.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.813+0000 7f760fdff700 1 -- 172.21.15.45:0/3613214687 wait complete. 2023-10-17T01:21:32.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.814+0000 7f760fdff700 1 Processor -- start 2023-10-17T01:21:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.814+0000 7f760fdff700 1 -- start start 2023-10-17T01:21:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760fdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 0x7f7608198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760fdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608101f30 0x7f7608199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760fdff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 0x7f76081a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760fdff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76081a0790 con 0x7f76081028d0 2023-10-17T01:21:32.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760fdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76081a0900 con 0x7f7608101f30 2023-10-17T01:21:32.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760fdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76081a0ac0 con 0x7f7608100d40 2023-10-17T01:21:32.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 0x7f7608198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 0x7f7608198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55898/0 (socket says 172.21.15.45:55898) 2023-10-17T01:21:32.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760d39a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608101f30 0x7f7608199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 -- 172.21.15.45:0/1472354646 learned_addr learned my addr 172.21.15.45:0/1472354646 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760e39c700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 0x7f76081a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 -- 172.21.15.45:0/1472354646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608101f30 msgr2=0x7f7608199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608101f30 0x7f7608199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 -- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 msgr2=0x7f76081a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 0x7f76081a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760db9b700 1 -- 172.21.15.45:0/1472354646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040095f0 con 0x7f7608100d40 2023-10-17T01:21:32.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.815+0000 7f760e39c700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 0x7f76081a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:32.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.816+0000 7f760db9b700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 0x7f7608198c50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f76040058e0 tx=0x7f760400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:32.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.816+0000 7f75feffd700 1 -- 172.21.15.45:0/1472354646 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760401d070 con 0x7f7608100d40 2023-10-17T01:21:32.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.816+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76081a0d40 con 0x7f7608100d40 2023-10-17T01:21:32.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.816+0000 7f75feffd700 1 -- 172.21.15.45:0/1472354646 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7604005050 con 0x7f7608100d40 2023-10-17T01:21:32.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.816+0000 7f75feffd700 1 -- 172.21.15.45:0/1472354646 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604004b30 con 0x7f7608100d40 2023-10-17T01:21:32.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.816+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76081a1160 con 0x7f7608100d40 2023-10-17T01:21:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.820+0000 7f75feffd700 1 -- 172.21.15.45:0/1472354646 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f76040051c0 con 0x7f7608100d40 2023-10-17T01:21:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.821+0000 7f76236f1b80 1 Processor -- start 2023-10-17T01:21:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 -- start start 2023-10-17T01:21:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2151d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2152730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a20a8a10 con 0x55b5a21522b0 2023-10-17T01:21:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a20a63d0 con 0x55b5a2151500 2023-10-17T01:21:32.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f75feffd700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f75f40a0b20 0x7f75f40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f76236f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a20a1cc0 con 0x55b5a2150ba0 2023-10-17T01:21:32.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f75feffd700 1 -- 172.21.15.45:0/1472354646 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f76040ccf30 con 0x7f7608100d40 2023-10-17T01:21:32.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f760e39c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f760e39c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55904/0 (socket says 172.21.15.45:55904) 2023-10-17T01:21:32.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f760e39c700 1 -- 172.21.15.45:0/1691801666 learned_addr learned my addr 172.21.15.45:0/1691801666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:32.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f760db9b700 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f760d39a700 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.822+0000 7f760e39c700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f75f40a0b20 0x7f75f40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760db9b700 1 -- 172.21.15.45:0/1691801666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 msgr2=0x55b5a2150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760db9b700 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760db9b700 1 -- 172.21.15.45:0/1691801666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 msgr2=0x55b5a2152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760db9b700 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760db9b700 1 -- 172.21.15.45:0/1691801666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5a1ecfde0 con 0x55b5a2151500 2023-10-17T01:21:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760e39c700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f75f40a0b20 0x7f75f40a2fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55b5a2150940 tx=0x7f760000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:32.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f760db9b700 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2151d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f760409a6f0 tx=0x7f7604096bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80bd62792024b193 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:32.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f75f3fff700 1 -- 172.21.15.45:0/1691801666 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760409ae60 con 0x55b5a2151500 2023-10-17T01:21:32.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f75f3fff700 1 -- 172.21.15.45:0/1691801666 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760409a760 con 0x55b5a2151500 2023-10-17T01:21:32.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.823+0000 7f75f3fff700 1 -- 172.21.15.45:0/1691801666 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604004ef0 con 0x55b5a2151500 2023-10-17T01:21:32.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 -- 172.21.15.45:0/1691801666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 msgr2=0x55b5a2151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2151d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f760409a6f0 tx=0x7f7604096bd0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 -- 172.21.15.45:0/1691801666 shutdown_connections 2023-10-17T01:21:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1691801666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2151d20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 -- 172.21.15.45:0/1691801666 >> 172.21.15.45:0/1691801666 conn(0x55b5a20b00d0 msgr2=0x55b5a214b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:32.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 -- 172.21.15.45:0/1691801666 shutdown_connections 2023-10-17T01:21:32.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.824+0000 7f76236f1b80 1 -- 172.21.15.45:0/1691801666 wait complete. 2023-10-17T01:21:32.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 -- start start 2023-10-17T01:21:32.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 Processor -- start 2023-10-17T01:21:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2150ba0 0x55b5a215a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 0x55b5a215aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a2161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a203e510 con 0x55b5a2150ba0 2023-10-17T01:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a203e1b0 con 0x55b5a21522b0 2023-10-17T01:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a1f25920 con 0x55b5a2151500 2023-10-17T01:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.825+0000 7f76236f1b80 10 client.0 _ll_register_callbacks cb 0x55b5a1f8ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 0x55b5a215aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a2161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 0x55b5a215aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55914/0 (socket says 172.21.15.45:55914) 2023-10-17T01:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 -- 172.21.15.45:0/846950509 learned_addr learned my addr 172.21.15.45:0/846950509 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760db9b700 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2150ba0 0x55b5a215a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 -- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 msgr2=0x55b5a2161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a2161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 -- 172.21.15.45:0/846950509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2150ba0 msgr2=0x55b5a215a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2150ba0 0x55b5a215a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 -- 172.21.15.45:0/846950509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76081a1160 con 0x55b5a2151500 2023-10-17T01:21:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f760d39a700 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 0x55b5a215aa70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f75f80117c0 tx=0x7f75f8011ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407a593fbf0a9397 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f75f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:21:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.826+0000 7f75f17fa700 1 -- 172.21.15.45:0/846950509 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8016070 con 0x55b5a2151500 2023-10-17T01:21:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5a20ccd10 con 0x55b5a2151500 2023-10-17T01:21:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f75f17fa700 1 -- 172.21.15.45:0/846950509 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f75f800f7b0 con 0x55b5a2151500 2023-10-17T01:21:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f75f17fa700 1 -- 172.21.15.45:0/846950509 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75f800f970 con 0x55b5a2151500 2023-10-17T01:21:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 10 client.26170 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b5a1eb8c60 con 0x55b5a2151500 2023-10-17T01:21:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f75f17fa700 1 -- 172.21.15.45:0/846950509 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5a20ccd10 con 0x55b5a2151500 2023-10-17T01:21:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f75f17fa700 1 -- 172.21.15.45:0/846950509 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f75f8023020 con 0x55b5a2151500 2023-10-17T01:21:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 10 client.26170 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 20 client.26170 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a2166320 0x55b5a2168740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 4 client.26170 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:32.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.827+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5a211fd30 con 0x55b5a2166320 2023-10-17T01:21:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.828+0000 7f760db9b700 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a2166320 0x55b5a2168740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.828+0000 7f760db9b700 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a2166320 0x55b5a2168740 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ee48b488191c96 server_cookie=4d86aea2544ec43 in_seq=0 out_seq=0 2023-10-17T01:21:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.828+0000 7f75f17fa700 10 client.26170 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f75f17fa700 1 -- 172.21.15.45:0/846950509 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55b5a211fd30 con 0x55b5a2166320 2023-10-17T01:21:32.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f75f17fa700 10 client.26170 handle_command_reply: tid=0 2023-10-17T01:21:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 client.26170 shutdown 2023-10-17T01:21:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 msgr2=0x55b5a215aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 0x55b5a215aa70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f75f80117c0 tx=0x7f75f8011ad0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 shutdown_connections 2023-10-17T01:21:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a2166320 0x55b5a2168740 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a2161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2150ba0 0x55b5a215a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 --2- 172.21.15.45:0/846950509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2151500 0x55b5a215aa70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f75f17fa700 0 client.26170 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 >> 172.21.15.45:0/846950509 conn(0x55b5a20b00d0 msgr2=0x55b5a214b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 shutdown_connections 2023-10-17T01:21:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.829+0000 7f76236f1b80 1 -- 172.21.15.45:0/846950509 wait complete. 2023-10-17T01:21:32.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.830+0000 7f76236f1b80 20 client.26170 trim_cache size 0 max 16384 2023-10-17T01:21:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 Processor -- start 2023-10-17T01:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 -- start start 2023-10-17T01:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2158640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2155180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a2162a50 con 0x55b5a21522b0 2023-10-17T01:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a215b6d0 con 0x55b5a2151500 2023-10-17T01:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.881+0000 7f76236f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a2143730 con 0x55b5a2150ba0 2023-10-17T01:21:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.882+0000 7f760e39c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.882+0000 7f760e39c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60144/0 (socket says 172.21.15.45:60144) 2023-10-17T01:21:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.882+0000 7f760e39c700 1 -- 172.21.15.45:0/1961193745 learned_addr learned my addr 172.21.15.45:0/1961193745 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.882+0000 7f760db9b700 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760d39a700 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760e39c700 1 -- 172.21.15.45:0/1961193745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 msgr2=0x55b5a2158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760e39c700 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760e39c700 1 -- 172.21.15.45:0/1961193745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 msgr2=0x55b5a2158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760e39c700 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760e39c700 1 -- 172.21.15.45:0/1961193745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5a1ef6250 con 0x55b5a21522b0 2023-10-17T01:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.883+0000 7f760d39a700 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.884+0000 7f760e39c700 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2155180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7600008b90 tx=0x7f7600009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3970d6a65875fdd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.884+0000 7f75fcbff700 1 -- 172.21.15.45:0/1961193745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600003e90 con 0x55b5a21522b0 2023-10-17T01:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.884+0000 7f75fcbff700 1 -- 172.21.15.45:0/1961193745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760000f040 con 0x55b5a21522b0 2023-10-17T01:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.884+0000 7f75fcbff700 1 -- 172.21.15.45:0/1961193745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760001b790 con 0x55b5a21522b0 2023-10-17T01:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.887+0000 7f76236f1b80 1 -- 172.21.15.45:0/1961193745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 msgr2=0x55b5a2155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.887+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2155180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7600008b90 tx=0x7f7600009920 comp rx=0 tx=0).stop 2023-10-17T01:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 -- 172.21.15.45:0/1961193745 shutdown_connections 2023-10-17T01:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a21522b0 0x55b5a2155180 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a2158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 --2- 172.21.15.45:0/1961193745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a2151500 0x55b5a2158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 -- 172.21.15.45:0/1961193745 >> 172.21.15.45:0/1961193745 conn(0x55b5a20b00d0 msgr2=0x55b5a2162340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:32.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 -- 172.21.15.45:0/1961193745 shutdown_connections 2023-10-17T01:21:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.888+0000 7f76236f1b80 1 -- 172.21.15.45:0/1961193745 wait complete. 2023-10-17T01:21:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.889+0000 7f76236f1b80 1 -- start start 2023-10-17T01:21:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.889+0000 7f76236f1b80 1 Processor -- start 2023-10-17T01:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.891+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a21b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.891+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2151500 0x55b5a21b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.891+0000 7f76236f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a21d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f76236f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a215fa60 con 0x55b5a2151500 2023-10-17T01:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f76236f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a215fbd0 con 0x55b5a21522b0 2023-10-17T01:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f76236f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a215fd90 con 0x55b5a2150ba0 2023-10-17T01:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f760e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a21d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f76236f1b80 10 client.0 _ll_register_callbacks cb 0x7f75f8009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f760e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a21d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42990/0 (socket says 172.21.15.45:42990) 2023-10-17T01:21:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f760e39c700 1 -- 172.21.15.45:0/2814906264 learned_addr learned my addr 172.21.15.45:0/2814906264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f760d39a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2151500 0x55b5a21b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.892+0000 7f760db9b700 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a21b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.893+0000 7f760e39c700 1 -- 172.21.15.45:0/2814906264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 msgr2=0x55b5a21b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.893+0000 7f760e39c700 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a21b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.893+0000 7f760e39c700 1 -- 172.21.15.45:0/2814906264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2151500 msgr2=0x55b5a21b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.893+0000 7f760e39c700 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2151500 0x55b5a21b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.893+0000 7f760e39c700 1 -- 172.21.15.45:0/2814906264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7600008930 con 0x55b5a21522b0 2023-10-17T01:21:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.894+0000 7f760e39c700 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a21d14f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7600009440 tx=0x7f7600006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d731847e8b23cac server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.894+0000 7f75f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:21:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.894+0000 7f75f17fa700 1 -- 172.21.15.45:0/2814906264 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600004720 con 0x55b5a21522b0 2023-10-17T01:21:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.894+0000 7f75f17fa700 1 -- 172.21.15.45:0/2814906264 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7600020040 con 0x55b5a21522b0 2023-10-17T01:21:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.894+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f760409a760 con 0x55b5a21522b0 2023-10-17T01:21:32.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.895+0000 7f75f17fa700 1 -- 172.21.15.45:0/2814906264 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7600004880 con 0x55b5a21522b0 2023-10-17T01:21:32.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.895+0000 7f76236f1b80 10 client.26195 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:32.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.895+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b5a1ece2b0 con 0x55b5a21522b0 2023-10-17T01:21:32.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.895+0000 7f75f17fa700 1 -- 172.21.15.45:0/2814906264 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760409a760 con 0x55b5a21522b0 2023-10-17T01:21:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.895+0000 7f75f17fa700 1 -- 172.21.15.45:0/2814906264 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f76000264a0 con 0x55b5a21522b0 2023-10-17T01:21:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f76236f1b80 10 client.26195 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f76236f1b80 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f76236f1b80 20 client.26195 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f76236f1b80 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a21d3180 0x55b5a21d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f76236f1b80 4 client.26195 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5a209ad40 con 0x55b5a21d3180 2023-10-17T01:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.896+0000 7f760db9b700 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a21d3180 0x55b5a21d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.898+0000 7f760db9b700 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a21d3180 0x55b5a21d55a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd46f74cc71356b9 server_cookie=a492f2e544c81b32 in_seq=0 out_seq=0 2023-10-17T01:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.898+0000 7f75f17fa700 10 client.26195 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.898+0000 7f75f17fa700 1 -- 172.21.15.45:0/2814906264 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b5a209ad40 con 0x55b5a21d3180 2023-10-17T01:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.898+0000 7f75f17fa700 10 client.26195 handle_command_reply: tid=0 2023-10-17T01:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.898+0000 7f76236f1b80 1 client.26195 shutdown 2023-10-17T01:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 msgr2=0x55b5a21d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a21d14f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7600009440 tx=0x7f7600006000 comp rx=0 tx=0).stop 2023-10-17T01:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 shutdown_connections 2023-10-17T01:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b5a21d3180 0x55b5a21d55a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a21522b0 0x55b5a21d14f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5a2150ba0 0x55b5a21b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f75f17fa700 0 client.26195 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 --2- 172.21.15.45:0/2814906264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5a2151500 0x55b5a21b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 >> 172.21.15.45:0/2814906264 conn(0x55b5a20b00d0 msgr2=0x55b5a2158640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 shutdown_connections 2023-10-17T01:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 1 -- 172.21.15.45:0/2814906264 wait complete. 2023-10-17T01:21:32.906 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:21:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:21:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:21:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:21:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:21:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:21:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:21:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:21:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:21:32.908 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.899+0000 7f76236f1b80 20 client.26195 trim_cache size 0 max 16384 2023-10-17T01:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f75f40a0b20 msgr2=0x7f75f40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f75f40a0b20 0x7f75f40a2fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55b5a2150940 tx=0x7f760000b040 comp rx=0 tx=0).stop 2023-10-17T01:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 msgr2=0x7f7608198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 0x7f7608198c50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f76040058e0 tx=0x7f760400f690 comp rx=0 tx=0).stop 2023-10-17T01:21:32.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 shutdown_connections 2023-10-17T01:21:32.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f75f40a0b20 0x7f75f40a2fe0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76081028d0 0x7f76081a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7608100d40 0x7f7608198c50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 --2- 172.21.15.45:0/1472354646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7608101f30 0x7f7608199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:32.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.900+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 >> 172.21.15.45:0/1472354646 conn(0x7f76080fc290 msgr2=0x7f76080fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:32.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.901+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 shutdown_connections 2023-10-17T01:21:32.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:21:32.901+0000 7f760fdff700 1 -- 172.21.15.45:0/1472354646 wait complete. 2023-10-17T01:21:32.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:32 smithi045 ceph-mon[98474]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 809 op/s 2023-10-17T01:21:32.933 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:21:33.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:32 smithi060 ceph-mon[134136]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 809 op/s 2023-10-17T01:21:33.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:32 smithi181 ceph-mon[134520]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 809 op/s 2023-10-17T01:21:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 385.0 s, 1673.5 tps, lat 59.574 ms stddev 75.439 2023-10-17T01:21:36.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:35 smithi045 ceph-mon[98474]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 MiB/s wr, 697 op/s 2023-10-17T01:21:36.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:35 smithi060 ceph-mon[134136]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 MiB/s wr, 697 op/s 2023-10-17T01:21:36.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:35 smithi181 ceph-mon[134520]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 MiB/s wr, 697 op/s 2023-10-17T01:21:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:36 smithi045 ceph-mon[98474]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 607 op/s 2023-10-17T01:21:37.085 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:36 smithi060 ceph-mon[134136]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 607 op/s 2023-10-17T01:21:37.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:36 smithi181 ceph-mon[134520]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 607 op/s 2023-10-17T01:21:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:38 smithi045 ceph-mon[98474]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 597 op/s 2023-10-17T01:21:39.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:38 smithi060 ceph-mon[134136]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 597 op/s 2023-10-17T01:21:39.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:38 smithi181 ceph-mon[134520]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 597 op/s 2023-10-17T01:21:40.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 390.0 s, 1958.0 tps, lat 51.188 ms stddev 56.781 2023-10-17T01:21:42.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:41 smithi045 ceph-mon[98474]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 683 op/s 2023-10-17T01:21:42.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:41 smithi060 ceph-mon[134136]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 683 op/s 2023-10-17T01:21:42.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:41 smithi181 ceph-mon[134520]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 683 op/s 2023-10-17T01:21:43.017 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:42 smithi060 ceph-mon[134136]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 638 op/s 2023-10-17T01:21:43.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:42 smithi045 ceph-mon[98474]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 638 op/s 2023-10-17T01:21:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:42 smithi181 ceph-mon[134520]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 638 op/s 2023-10-17T01:21:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 -- 172.21.15.45:0/4111605248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 msgr2=0x7fe8201091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 --2- 172.21.15.45:0/4111605248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe8201091f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe818009f90 tx=0x7fe81800b330 comp rx=0 tx=0).stop 2023-10-17T01:21:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 -- 172.21.15.45:0/4111605248 shutdown_connections 2023-10-17T01:21:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 --2- 172.21.15.45:0/4111605248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe8201091f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 --2- 172.21.15.45:0/4111605248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe820100db0 0x7fe8201011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 --2- 172.21.15.45:0/4111605248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 0x7fe820102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 -- 172.21.15.45:0/4111605248 >> 172.21.15.45:0/4111605248 conn(0x7fe8200fc300 msgr2=0x7fe8200fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 -- 172.21.15.45:0/4111605248 shutdown_connections 2023-10-17T01:21:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.051+0000 7fe8285ff700 1 -- 172.21.15.45:0/4111605248 wait complete. 2023-10-17T01:21:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.053+0000 7fe8285ff700 1 Processor -- start 2023-10-17T01:21:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.053+0000 7fe8285ff700 1 -- start start 2023-10-17T01:21:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.053+0000 7fe8285ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe820100db0 0x7fe820196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe8285ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 0x7fe820197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe8285ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe820197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe8285ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820198660 con 0x7fe820101fa0 2023-10-17T01:21:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe8285ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8201987d0 con 0x7fe820100db0 2023-10-17T01:21:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe8285ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820198990 con 0x7fe820102940 2023-10-17T01:21:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe820197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe82639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe820100db0 0x7fe820196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe820197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46862/0 (socket says 172.21.15.45:46862) 2023-10-17T01:21:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 -- 172.21.15.45:0/3105040807 learned_addr learned my addr 172.21.15.45:0/3105040807 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 0x7fe820197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 -- 172.21.15.45:0/3105040807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe820100db0 msgr2=0x7fe820196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe820100db0 0x7fe820196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 -- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 msgr2=0x7fe820197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 0x7fe820197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe826b9c700 1 -- 172.21.15.45:0/3105040807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818009c40 con 0x7fe820102940 2023-10-17T01:21:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.054+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 0x7fe820197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:21:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.055+0000 7fe826b9c700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe820197f60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe818009f60 tx=0x7fe818004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.055+0000 7fe8177fe700 1 -- 172.21.15.45:0/3105040807 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800e040 con 0x7fe820102940 2023-10-17T01:21:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.055+0000 7fe8177fe700 1 -- 172.21.15.45:0/3105040807 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe818008e80 con 0x7fe820102940 2023-10-17T01:21:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.055+0000 7fe8177fe700 1 -- 172.21.15.45:0/3105040807 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180052a0 con 0x7fe820102940 2023-10-17T01:21:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.055+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe820198c10 con 0x7fe820102940 2023-10-17T01:21:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.055+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe820073160 con 0x7fe820102940 2023-10-17T01:21:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.061+0000 7fe8177fe700 1 -- 172.21.15.45:0/3105040807 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fe81801d470 con 0x7fe820102940 2023-10-17T01:21:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.064+0000 7fe83bee1b80 1 Processor -- start 2023-10-17T01:21:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.064+0000 7fe8177fe700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe80c0a0b20 0x7fe80c0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.064+0000 7fe83bee1b80 1 -- start start 2023-10-17T01:21:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe8177fe700 1 -- 172.21.15.45:0/3105040807 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe8180cd8c0 con 0x7fe820102940 2023-10-17T01:21:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8075269b0 con 0x55f8075cfd90 2023-10-17T01:21:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe80c0a0b20 0x7fe80c0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f807524370 con 0x55f8075ce830 2023-10-17T01:21:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f80751fc60 con 0x55f8075cf190 2023-10-17T01:21:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe825b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe82639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe825b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46866/0 (socket says 172.21.15.45:46866) 2023-10-17T01:21:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.065+0000 7fe825b9a700 1 -- 172.21.15.45:0/147351517 learned_addr learned my addr 172.21.15.45:0/147351517 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe826b9c700 1 -- 172.21.15.45:0/147351517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 msgr2=0x55f8075cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe826b9c700 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe826b9c700 1 -- 172.21.15.45:0/147351517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 msgr2=0x55f8075cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe826b9c700 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe826b9c700 1 -- 172.21.15.45:0/147351517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f80734dd80 con 0x55f8075cfd90 2023-10-17T01:21:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe80c0a0b20 0x7fe80c0a2fe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe810013ab0 tx=0x7fe810009ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe826b9c700 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d0210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8180cb760 tx=0x7fe818009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1b87daca7cc407 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe814bfe700 1 -- 172.21.15.45:0/147351517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81809a930 con 0x55f8075cfd90 2023-10-17T01:21:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe814bfe700 1 -- 172.21.15.45:0/147351517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81802e350 con 0x55f8075cfd90 2023-10-17T01:21:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.066+0000 7fe814bfe700 1 -- 172.21.15.45:0/147351517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81801d720 con 0x55f8075cfd90 2023-10-17T01:21:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.067+0000 7fe83bee1b80 1 -- 172.21.15.45:0/147351517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 msgr2=0x55f8075d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.067+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d0210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8180cb760 tx=0x7fe818009bd0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 -- 172.21.15.45:0/147351517 shutdown_connections 2023-10-17T01:21:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d0210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/147351517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 -- 172.21.15.45:0/147351517 >> 172.21.15.45:0/147351517 conn(0x55f80752e070 msgr2=0x55f8075c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 -- 172.21.15.45:0/147351517 shutdown_connections 2023-10-17T01:21:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.068+0000 7fe83bee1b80 1 -- 172.21.15.45:0/147351517 wait complete. 2023-10-17T01:21:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.069+0000 7fe83bee1b80 1 -- start start 2023-10-17T01:21:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.069+0000 7fe83bee1b80 1 Processor -- start 2023-10-17T01:21:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075ce830 0x55f8075d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cf190 0x55f8075d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f8075df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8074bc4b0 con 0x55f8075cf190 2023-10-17T01:21:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8074bc150 con 0x55f8075cfd90 2023-10-17T01:21:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8073a38c0 con 0x55f8075ce830 2023-10-17T01:21:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.070+0000 7fe83bee1b80 10 client.0 _ll_register_callbacks cb 0x55f807409a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f8075df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f8075df650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48490/0 (socket says 172.21.15.45:48490) 2023-10-17T01:21:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 -- 172.21.15.45:0/1816077866 learned_addr learned my addr 172.21.15.45:0/1816077866 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe82639b700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075ce830 0x55f8075d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe825b9a700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cf190 0x55f8075d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 -- 172.21.15.45:0/1816077866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075ce830 msgr2=0x55f8075d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075ce830 0x55f8075d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 -- 172.21.15.45:0/1816077866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cf190 msgr2=0x55f8075d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cf190 0x55f8075d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.071+0000 7fe826b9c700 1 -- 172.21.15.45:0/1816077866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe820073160 con 0x55f8075cfd90 2023-10-17T01:21:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.072+0000 7fe82639b700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075ce830 0x55f8075d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.072+0000 7fe826b9c700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f8075df650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe818097a40 tx=0x7fe81802e4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0b1c0e2894fcf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.072+0000 7fe809ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:21:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.072+0000 7fe809ffb700 1 -- 172.21.15.45:0/1816077866 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818095de0 con 0x55f8075cfd90 2023-10-17T01:21:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.072+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f80754acb0 con 0x55f8075cfd90 2023-10-17T01:21:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.072+0000 7fe809ffb700 1 -- 172.21.15.45:0/1816077866 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe81809b6d0 con 0x55f8075cfd90 2023-10-17T01:21:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.073+0000 7fe809ffb700 1 -- 172.21.15.45:0/1816077866 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe81809ee00 con 0x55f8075cfd90 2023-10-17T01:21:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.073+0000 7fe83bee1b80 10 client.26207 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.073+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f807336c00 con 0x55f8075cfd90 2023-10-17T01:21:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.073+0000 7fe809ffb700 1 -- 172.21.15.45:0/1816077866 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f80754acb0 con 0x55f8075cfd90 2023-10-17T01:21:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.073+0000 7fe809ffb700 1 -- 172.21.15.45:0/1816077866 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe81809a5a0 con 0x55f8075cfd90 2023-10-17T01:21:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe83bee1b80 10 client.26207 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe83bee1b80 10 client.26207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe83bee1b80 20 client.26207 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f8075ec270 0x55f8075ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe83bee1b80 4 client.26207 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f80759dcd0 con 0x55f8075ec270 2023-10-17T01:21:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.074+0000 7fe82639b700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f8075ec270 0x55f8075ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.075+0000 7fe82639b700 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f8075ec270 0x55f8075ee690 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3485898183f7bd1b server_cookie=fcadd1fcfcb6ff74 in_seq=0 out_seq=0 2023-10-17T01:21:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.075+0000 7fe809ffb700 10 client.26207 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe809ffb700 1 -- 172.21.15.45:0/1816077866 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55f80759dcd0 con 0x55f8075ec270 2023-10-17T01:21:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe809ffb700 10 client.26207 handle_command_reply: tid=0 2023-10-17T01:21:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 client.26207 shutdown 2023-10-17T01:21:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 msgr2=0x55f8075df650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f8075df650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe818097a40 tx=0x7fe81802e4c0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 shutdown_connections 2023-10-17T01:21:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f8075ec270 0x55f8075ee690 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe809ffb700 0 client.26207 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f8075df650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075ce830 0x55f8075d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1816077866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cf190 0x55f8075d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 >> 172.21.15.45:0/1816077866 conn(0x55f80752e070 msgr2=0x55f8075c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.076+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 shutdown_connections 2023-10-17T01:21:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.077+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1816077866 wait complete. 2023-10-17T01:21:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.077+0000 7fe83bee1b80 20 client.26207 trim_cache size 0 max 16384 2023-10-17T01:21:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.103+0000 7fe83bee1b80 1 Processor -- start 2023-10-17T01:21:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 -- start start 2023-10-17T01:21:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8075d7ae0 con 0x55f8075cfd90 2023-10-17T01:21:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8075c6010 con 0x55f8075ce830 2023-10-17T01:21:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8075d8b10 con 0x55f8075cf190 2023-10-17T01:21:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe82639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe825b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe82639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075d8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48502/0 (socket says 172.21.15.45:48502) 2023-10-17T01:21:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe82639b700 1 -- 172.21.15.45:0/3776435554 learned_addr learned my addr 172.21.15.45:0/3776435554 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.104+0000 7fe826b9c700 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe825b9a700 1 -- 172.21.15.45:0/3776435554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 msgr2=0x55f8075d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe825b9a700 1 -- 172.21.15.45:0/3776435554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 msgr2=0x55f8075d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe825b9a700 1 -- 172.21.15.45:0/3776435554 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8073741f0 con 0x55f8075cf190 2023-10-17T01:21:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe82639b700 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe825b9a700 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075d85d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55f8075ce5d0 tx=0x7fe810021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=419fc5253a12359b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe8153ff700 1 -- 172.21.15.45:0/3776435554 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810013110 con 0x55f8075cf190 2023-10-17T01:21:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe8153ff700 1 -- 172.21.15.45:0/3776435554 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe810013270 con 0x55f8075cf190 2023-10-17T01:21:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.105+0000 7fe8153ff700 1 -- 172.21.15.45:0/3776435554 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810003af0 con 0x55f8075cf190 2023-10-17T01:21:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 -- 172.21.15.45:0/3776435554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 msgr2=0x55f8075d85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075d85d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55f8075ce5d0 tx=0x7fe810021000 comp rx=0 tx=0).stop 2023-10-17T01:21:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 -- 172.21.15.45:0/3776435554 shutdown_connections 2023-10-17T01:21:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075cfd90 0x55f8075d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075ce830 0x55f8075d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/3776435554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f8075d85d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 -- 172.21.15.45:0/3776435554 >> 172.21.15.45:0/3776435554 conn(0x55f80752e070 msgr2=0x55f8075d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 -- 172.21.15.45:0/3776435554 shutdown_connections 2023-10-17T01:21:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 -- 172.21.15.45:0/3776435554 wait complete. 2023-10-17T01:21:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.106+0000 7fe83bee1b80 1 -- start start 2023-10-17T01:21:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.107+0000 7fe83bee1b80 1 Processor -- start 2023-10-17T01:21:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075ce830 0x55f807635dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f807636310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f80763cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f80763d2a0 con 0x55f8075ce830 2023-10-17T01:21:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f80763d410 con 0x55f8075cfd90 2023-10-17T01:21:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f80763d5d0 con 0x55f8075cf190 2023-10-17T01:21:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe83bee1b80 10 client.0 _ll_register_callbacks cb 0x7fe81c0108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f80763cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.108+0000 7fe826b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f80763cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48510/0 (socket says 172.21.15.45:48510) 2023-10-17T01:21:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.109+0000 7fe826b9c700 1 -- 172.21.15.45:0/1815480534 learned_addr learned my addr 172.21.15.45:0/1815480534 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.109+0000 7fe82639b700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075ce830 0x55f807635dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.109+0000 7fe825b9a700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f807636310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.109+0000 7fe82639b700 1 -- 172.21.15.45:0/1815480534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 msgr2=0x55f807636310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.110+0000 7fe82639b700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f807636310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.110+0000 7fe82639b700 1 -- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 msgr2=0x55f80763cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.110+0000 7fe82639b700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f80763cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.110+0000 7fe82639b700 1 -- 172.21.15.45:0/1815480534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c010550 con 0x55f8075ce830 2023-10-17T01:21:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.110+0000 7fe825b9a700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f807636310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:21:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.110+0000 7fe826b9c700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f80763cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.113+0000 7fe82639b700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075ce830 0x55f807635dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe81c010890 tx=0x7fe81c004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ca85adccf40ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.113+0000 7fe809ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:21:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.113+0000 7fe809ffb700 1 -- 172.21.15.45:0/1815480534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c015040 con 0x55f8075ce830 2023-10-17T01:21:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.113+0000 7fe809ffb700 1 -- 172.21.15.45:0/1815480534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe81c004830 con 0x55f8075ce830 2023-10-17T01:21:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.114+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe81802e350 con 0x55f8075ce830 2023-10-17T01:21:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.114+0000 7fe809ffb700 1 -- 172.21.15.45:0/1815480534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe81c020e70 con 0x55f8075ce830 2023-10-17T01:21:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.114+0000 7fe83bee1b80 10 client.15957 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.114+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fe81001b040 con 0x55f8075ce830 2023-10-17T01:21:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe809ffb700 1 -- 172.21.15.45:0/1815480534 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81802e350 con 0x55f8075ce830 2023-10-17T01:21:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe809ffb700 1 -- 172.21.15.45:0/1815480534 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe81c030050 con 0x55f8075ce830 2023-10-17T01:21:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe83bee1b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe83bee1b80 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe83bee1b80 20 client.15957 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f80763f070 0x55f807641490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe83bee1b80 4 client.15957 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:21:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f807518ce0 con 0x55f80763f070 2023-10-17T01:21:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.115+0000 7fe825b9a700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f80763f070 0x55f807641490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.116+0000 7fe809ffb700 10 client.15957 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.116+0000 7fe825b9a700 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f80763f070 0x55f807641490 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395b50f4ced383f4 server_cookie=47272557996de160 in_seq=0 out_seq=0 2023-10-17T01:21:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.128+0000 7fe809ffb700 1 -- 172.21.15.45:0/1815480534 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f807518ce0 con 0x55f80763f070 2023-10-17T01:21:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.128+0000 7fe809ffb700 10 client.15957 handle_command_reply: tid=0 2023-10-17T01:21:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.128+0000 7fe83bee1b80 1 client.15957 shutdown 2023-10-17T01:21:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075ce830 msgr2=0x55f807635dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075ce830 0x55f807635dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe81c010890 tx=0x7fe81c004320 comp rx=0 tx=0).stop 2023-10-17T01:21:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 shutdown_connections 2023-10-17T01:21:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55f80763f070 0x55f807641490 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8075cfd90 0x55f80763cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8075ce830 0x55f807635dd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 --2- 172.21.15.45:0/1815480534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8075cf190 0x55f807636310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 >> 172.21.15.45:0/1815480534 conn(0x55f80752e070 msgr2=0x55f8075d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe809ffb700 0 client.15957 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 shutdown_connections 2023-10-17T01:21:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 1 -- 172.21.15.45:0/1815480534 wait complete. 2023-10-17T01:21:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.129+0000 7fe83bee1b80 20 client.15957 trim_cache size 0 max 16384 2023-10-17T01:21:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:21:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:21:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:21:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:21:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe80c0a0b20 msgr2=0x7fe80c0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe80c0a0b20 0x7fe80c0a2fe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe810013ab0 tx=0x7fe810009ca0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 msgr2=0x7fe820197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe820197f60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe818009f60 tx=0x7fe818004d30 comp rx=0 tx=0).stop 2023-10-17T01:21:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 shutdown_connections 2023-10-17T01:21:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe80c0a0b20 0x7fe80c0a2fe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe820102940 0x7fe820197f60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe820100db0 0x7fe820196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 --2- 172.21.15.45:0/3105040807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe820101fa0 0x7fe820197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 >> 172.21.15.45:0/3105040807 conn(0x7fe8200fc300 msgr2=0x7fe8201053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 shutdown_connections 2023-10-17T01:21:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:45.130+0000 7fe8285ff700 1 -- 172.21.15.45:0/3105040807 wait complete. 2023-10-17T01:21:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 395.0 s, 2038.3 tps, lat 48.085 ms stddev 59.544 2023-10-17T01:21:46.023 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:45 smithi060 ceph-mon[134136]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 722 op/s 2023-10-17T01:21:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:45 smithi045 ceph-mon[98474]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 722 op/s 2023-10-17T01:21:46.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:45 smithi181 ceph-mon[134520]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 722 op/s 2023-10-17T01:21:48.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:47 smithi045 ceph-mon[98474]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 622 op/s 2023-10-17T01:21:48.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:47 smithi060 ceph-mon[134136]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 622 op/s 2023-10-17T01:21:48.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:47 smithi181 ceph-mon[134520]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 622 op/s 2023-10-17T01:21:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:48 smithi045 ceph-mon[98474]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 633 op/s 2023-10-17T01:21:49.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:48 smithi060 ceph-mon[134136]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 633 op/s 2023-10-17T01:21:49.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:48 smithi181 ceph-mon[134520]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 633 op/s 2023-10-17T01:21:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 400.0 s, 1898.7 tps, lat 53.702 ms stddev 66.664 2023-10-17T01:21:52.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:51 smithi045 ceph-mon[98474]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 764 op/s 2023-10-17T01:21:52.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:51 smithi060 ceph-mon[134136]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 764 op/s 2023-10-17T01:21:52.165 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:51 smithi181 ceph-mon[134520]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 764 op/s 2023-10-17T01:21:54.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:53 smithi045 ceph-mon[98474]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 696 op/s 2023-10-17T01:21:54.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:53 smithi060 ceph-mon[134136]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 696 op/s 2023-10-17T01:21:54.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:53 smithi181 ceph-mon[134520]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 696 op/s 2023-10-17T01:21:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 405.0 s, 1819.4 tps, lat 54.834 ms stddev 68.879 2023-10-17T01:21:56.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:55 smithi045 ceph-mon[98474]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2023-10-17T01:21:56.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:55 smithi060 ceph-mon[134136]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2023-10-17T01:21:56.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:55 smithi181 ceph-mon[134520]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2023-10-17T01:21:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:56 smithi045 ceph-mon[98474]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 653 op/s 2023-10-17T01:21:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:56 smithi060 ceph-mon[134136]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 653 op/s 2023-10-17T01:21:57.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:56 smithi181 ceph-mon[134520]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 653 op/s 2023-10-17T01:21:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.362+0000 7f9e6ddff700 1 -- 172.21.15.45:0/275226319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68074d30 msgr2=0x7f9e68073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.362+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/275226319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68074d30 0x7f9e68073180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009f90 tx=0x7f9e5800b330 comp rx=0 tx=0).stop 2023-10-17T01:21:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 -- 172.21.15.45:0/275226319 shutdown_connections 2023-10-17T01:21:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/275226319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e68108c30 0x7f9e6810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/275226319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68074d30 0x7f9e68073180 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/275226319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68073750 0x7f9e68073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 -- 172.21.15.45:0/275226319 >> 172.21.15.45:0/275226319 conn(0x7f9e680fc290 msgr2=0x7f9e680fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 -- 172.21.15.45:0/275226319 shutdown_connections 2023-10-17T01:21:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.364+0000 7f9e6ddff700 1 -- 172.21.15.45:0/275226319 wait complete. 2023-10-17T01:21:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 Processor -- start 2023-10-17T01:21:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 -- start start 2023-10-17T01:21:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e68073750 0x7f9e68198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68074d30 0x7f9e68199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68108c30 0x7f9e6819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6819a950 con 0x7f9e68108c30 2023-10-17T01:21:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6819aac0 con 0x7f9e68074d30 2023-10-17T01:21:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.366+0000 7f9e6ddff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6819ac80 con 0x7f9e68073750 2023-10-17T01:21:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.367+0000 7f9e5effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68074d30 0x7f9e68199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.367+0000 7f9e5effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68074d30 0x7f9e68199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47908/0 (socket says 172.21.15.45:47908) 2023-10-17T01:21:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.367+0000 7f9e5effd700 1 -- 172.21.15.45:0/1604938434 learned_addr learned my addr 172.21.15.45:0/1604938434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.367+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68108c30 0x7f9e6819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.368+0000 7f9e67fff700 1 -- 172.21.15.45:0/1604938434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e68073750 msgr2=0x7f9e68198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.368+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e68073750 0x7f9e68198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.368+0000 7f9e67fff700 1 -- 172.21.15.45:0/1604938434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68074d30 msgr2=0x7f9e68199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.368+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68074d30 0x7f9e68199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.368+0000 7f9e67fff700 1 -- 172.21.15.45:0/1604938434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58009c40 con 0x7f9e68108c30 2023-10-17T01:21:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.370+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68108c30 0x7f9e6819a2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b3f0 tx=0x7f9e6000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.371+0000 7f9e657fa700 1 -- 172.21.15.45:0/1604938434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60011830 con 0x7f9e68108c30 2023-10-17T01:21:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.371+0000 7f9e657fa700 1 -- 172.21.15.45:0/1604938434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e60011e50 con 0x7f9e68108c30 2023-10-17T01:21:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.371+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6819af60 con 0x7f9e68108c30 2023-10-17T01:21:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.371+0000 7f9e657fa700 1 -- 172.21.15.45:0/1604938434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60010db0 con 0x7f9e68108c30 2023-10-17T01:21:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.372+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e681a1720 con 0x7f9e68108c30 2023-10-17T01:21:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.382+0000 7f9e657fa700 1 -- 172.21.15.45:0/1604938434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9e60011990 con 0x7f9e68108c30 2023-10-17T01:21:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.386+0000 7f9e657fa700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e540a0cf0 0x7f9e540a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.387+0000 7f9e657fa700 1 -- 172.21.15.45:0/1604938434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9e600cba70 con 0x7f9e68108c30 2023-10-17T01:21:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.387+0000 7f9e677fe700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e540a0cf0 0x7f9e540a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.388+0000 7f9e677fe700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e540a0cf0 0x7f9e540a31b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009f90 tx=0x7f9e58000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.389+0000 7f9e81678b80 1 Processor -- start 2023-10-17T01:21:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.389+0000 7f9e81678b80 1 -- start start 2023-10-17T01:21:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31533c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31534790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31535210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e81678b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3148b9b0 con 0x55fc31534d90 2023-10-17T01:21:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e81678b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31489370 con 0x55fc31534190 2023-10-17T01:21:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e81678b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31484c60 con 0x55fc31533830 2023-10-17T01:21:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31533c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31533c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36974/0 (socket says 172.21.15.45:36974) 2023-10-17T01:21:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e677fe700 1 -- 172.21.15.45:0/3331836467 learned_addr learned my addr 172.21.15.45:0/3331836467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.390+0000 7f9e5effd700 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31534790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.391+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31535210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.391+0000 7f9e677fe700 1 -- 172.21.15.45:0/3331836467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 msgr2=0x55fc31534790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.391+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31534790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.391+0000 7f9e677fe700 1 -- 172.21.15.45:0/3331836467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 msgr2=0x55fc31535210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.391+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31535210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.391+0000 7f9e677fe700 1 -- 172.21.15.45:0/3331836467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc312b2d80 con 0x55fc31533830 2023-10-17T01:21:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.392+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31533c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9e58007c30 tx=0x7f9e5801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560e27559e397e15 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.392+0000 7f9e5dffb700 1 -- 172.21.15.45:0/3331836467 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800e040 con 0x55fc31533830 2023-10-17T01:21:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.393+0000 7f9e5dffb700 1 -- 172.21.15.45:0/3331836467 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e58021070 con 0x55fc31533830 2023-10-17T01:21:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 -- 172.21.15.45:0/3331836467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 msgr2=0x55fc31533c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31533c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9e58007c30 tx=0x7f9e5801a000 comp rx=0 tx=0).stop 2023-10-17T01:21:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 -- 172.21.15.45:0/3331836467 shutdown_connections 2023-10-17T01:21:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31535210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31533c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3331836467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31534790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.394+0000 7f9e81678b80 1 -- 172.21.15.45:0/3331836467 >> 172.21.15.45:0/3331836467 conn(0x55fc31493070 msgr2=0x55fc3152e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.395+0000 7f9e81678b80 1 -- 172.21.15.45:0/3331836467 shutdown_connections 2023-10-17T01:21:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.395+0000 7f9e81678b80 1 -- 172.21.15.45:0/3331836467 wait complete. 2023-10-17T01:21:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.396+0000 7f9e81678b80 1 -- start start 2023-10-17T01:21:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.396+0000 7f9e81678b80 1 Processor -- start 2023-10-17T01:21:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.397+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc3153d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.397+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534190 0x55fc3153d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534d90 0x55fc315448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc3153d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc3153d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36984/0 (socket says 172.21.15.45:36984) 2023-10-17T01:21:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e81678b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc314214b0 con 0x55fc31534190 2023-10-17T01:21:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e81678b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31421150 con 0x55fc31534d90 2023-10-17T01:21:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e81678b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc313088c0 con 0x55fc31533830 2023-10-17T01:21:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e677fe700 1 -- 172.21.15.45:0/1749375318 learned_addr learned my addr 172.21.15.45:0/1749375318 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534d90 0x55fc315448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e81678b80 10 client.0 _ll_register_callbacks cb 0x55fc3136ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e5effd700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534190 0x55fc3153d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e67fff700 1 -- 172.21.15.45:0/1749375318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 msgr2=0x55fc3153d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc3153d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e67fff700 1 -- 172.21.15.45:0/1749375318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534190 msgr2=0x55fc3153d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.398+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534190 0x55fc3153d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.399+0000 7f9e67fff700 1 -- 172.21.15.45:0/1749375318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6810bcc0 con 0x55fc31534d90 2023-10-17T01:21:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.399+0000 7f9e67fff700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534d90 0x55fc315448e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e600c9540 tx=0x7f9e60098b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29c35acf0a13bc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.399+0000 7f9e46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:21:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.399+0000 7f9e46ffd700 1 -- 172.21.15.45:0/1749375318 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60099090 con 0x55fc31534d90 2023-10-17T01:21:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.399+0000 7f9e46ffd700 1 -- 172.21.15.45:0/1749375318 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9e6009cec0 con 0x55fc31534d90 2023-10-17T01:21:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.399+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc314afcb0 con 0x55fc31534d90 2023-10-17T01:21:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e46ffd700 1 -- 172.21.15.45:0/1749375318 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e600a1100 con 0x55fc31534d90 2023-10-17T01:21:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e81678b80 10 client.26225 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc3129bc00 con 0x55fc31534d90 2023-10-17T01:21:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e46ffd700 1 -- 172.21.15.45:0/1749375318 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc314afcb0 con 0x55fc31534d90 2023-10-17T01:21:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e46ffd700 1 -- 172.21.15.45:0/1749375318 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f9e600a0c70 con 0x55fc31534d90 2023-10-17T01:21:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e81678b80 10 client.26225 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e81678b80 10 client.26225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.400+0000 7f9e81678b80 20 client.26225 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.401+0000 7f9e81678b80 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc315490a0 0x55fc3154b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.401+0000 7f9e81678b80 4 client.26225 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:21:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.401+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc31502cd0 con 0x55fc315490a0 2023-10-17T01:21:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.403+0000 7f9e677fe700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc315490a0 0x55fc3154b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.404+0000 7f9e46ffd700 10 client.26225 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.404+0000 7f9e677fe700 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc315490a0 0x55fc3154b490 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a500bd1992fd6c50 server_cookie=6f9de0c60daa05f8 in_seq=0 out_seq=0 2023-10-17T01:21:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.405+0000 7f9e46ffd700 1 -- 172.21.15.45:0/1749375318 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55fc31502cd0 con 0x55fc315490a0 2023-10-17T01:21:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.405+0000 7f9e46ffd700 10 client.26225 handle_command_reply: tid=0 2023-10-17T01:21:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.405+0000 7f9e81678b80 1 client.26225 shutdown 2023-10-17T01:21:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534d90 msgr2=0x55fc315448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534d90 0x55fc315448e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e600c9540 tx=0x7f9e60098b20 comp rx=0 tx=0).stop 2023-10-17T01:21:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 shutdown_connections 2023-10-17T01:21:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc315490a0 0x55fc3154b490 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534d90 0x55fc315448e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc3153d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 --2- 172.21.15.45:0/1749375318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534190 0x55fc3153d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 >> 172.21.15.45:0/1749375318 conn(0x55fc31493070 msgr2=0x55fc3152da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e46ffd700 0 client.26225 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.406+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 shutdown_connections 2023-10-17T01:21:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.407+0000 7f9e81678b80 1 -- 172.21.15.45:0/1749375318 wait complete. 2023-10-17T01:21:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.407+0000 7f9e81678b80 20 client.26225 trim_cache size 0 max 16384 2023-10-17T01:21:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.437+0000 7f9e81678b80 1 Processor -- start 2023-10-17T01:21:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.437+0000 7f9e81678b80 1 -- start start 2023-10-17T01:21:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31533830 0x55fc3152da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 0x55fc3154b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc3153b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e81678b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3153cf20 con 0x55fc31534d90 2023-10-17T01:21:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e81678b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3153bef0 con 0x55fc31533830 2023-10-17T01:21:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e81678b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3152f600 con 0x55fc31534190 2023-10-17T01:21:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e5effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 0x55fc3154b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e5effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 0x55fc3154b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36986/0 (socket says 172.21.15.45:36986) 2023-10-17T01:21:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e5effd700 1 -- 172.21.15.45:0/3238816493 learned_addr learned my addr 172.21.15.45:0/3238816493 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31533830 0x55fc3152da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.438+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc3153b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e5effd700 1 -- 172.21.15.45:0/3238816493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31533830 msgr2=0x55fc3152da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e5effd700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31533830 0x55fc3152da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e5effd700 1 -- 172.21.15.45:0/3238816493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 msgr2=0x55fc3153b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e5effd700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc3153b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e5effd700 1 -- 172.21.15.45:0/3238816493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc312d91f0 con 0x55fc31534190 2023-10-17T01:21:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31533830 0x55fc3152da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc3153b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:21:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.439+0000 7f9e5effd700 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 0x55fc3154b0b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000be30 tx=0x7f9e50011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4574fe40f5e2ff server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.441+0000 7f9e5e7fc700 1 -- 172.21.15.45:0/3238816493 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50011d80 con 0x55fc31534190 2023-10-17T01:21:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.441+0000 7f9e5e7fc700 1 -- 172.21.15.45:0/3238816493 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e50017040 con 0x55fc31534190 2023-10-17T01:21:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.442+0000 7f9e5e7fc700 1 -- 172.21.15.45:0/3238816493 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50010760 con 0x55fc31534190 2023-10-17T01:21:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 -- 172.21.15.45:0/3238816493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 msgr2=0x55fc3154b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 0x55fc3154b0b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000be30 tx=0x7f9e50011720 comp rx=0 tx=0).stop 2023-10-17T01:21:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 -- 172.21.15.45:0/3238816493 shutdown_connections 2023-10-17T01:21:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc3153b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31533830 0x55fc3152da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3238816493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31534190 0x55fc3154b0b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.444+0000 7f9e81678b80 1 -- 172.21.15.45:0/3238816493 >> 172.21.15.45:0/3238816493 conn(0x55fc31493070 msgr2=0x55fc3153c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.445+0000 7f9e81678b80 1 -- 172.21.15.45:0/3238816493 shutdown_connections 2023-10-17T01:21:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.445+0000 7f9e81678b80 1 -- 172.21.15.45:0/3238816493 wait complete. 2023-10-17T01:21:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.445+0000 7f9e81678b80 1 -- start start 2023-10-17T01:21:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.445+0000 7f9e81678b80 1 Processor -- start 2023-10-17T01:21:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31592c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31593150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e81678b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31599a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e81678b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc31599fd0 con 0x55fc31534d90 2023-10-17T01:21:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e81678b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3159a110 con 0x55fc31534190 2023-10-17T01:21:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e81678b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3159a2d0 con 0x55fc31533830 2023-10-17T01:21:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31592c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31592c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36990/0 (socket says 172.21.15.45:36990) 2023-10-17T01:21:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.446+0000 7f9e677fe700 1 -- 172.21.15.45:0/3605646533 learned_addr learned my addr 172.21.15.45:0/3605646533 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:21:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e81678b80 10 client.0 _ll_register_callbacks cb 0x7f9e60098e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:21:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31599a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e5effd700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31593150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e67fff700 1 -- 172.21.15.45:0/3605646533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 msgr2=0x55fc31592c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31592c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e67fff700 1 -- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 msgr2=0x55fc31593150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:21:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31593150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e67fff700 1 -- 172.21.15.45:0/3605646533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc312b2d80 con 0x55fc31534d90 2023-10-17T01:21:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.447+0000 7f9e5effd700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31593150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:21:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.448+0000 7f9e67fff700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31599a90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e60010430 tx=0x7f9e600988e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b57cb23b4563d3c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:21:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.448+0000 7f9e46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:21:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.448+0000 7f9e46ffd700 1 -- 172.21.15.45:0/3605646533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6009cec0 con 0x55fc31534d90 2023-10-17T01:21:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.448+0000 7f9e46ffd700 1 -- 172.21.15.45:0/3605646533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9e60099090 con 0x55fc31534d90 2023-10-17T01:21:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.448+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e58021070 con 0x55fc31534d90 2023-10-17T01:21:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.449+0000 7f9e46ffd700 1 -- 172.21.15.45:0/3605646533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e600a1100 con 0x55fc31534d90 2023-10-17T01:21:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.450+0000 7f9e81678b80 10 client.15972 fetch_fsmap learned FSMap version 40 2023-10-17T01:21:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.450+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f9e5000bae0 con 0x55fc31534d90 2023-10-17T01:21:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e46ffd700 1 -- 172.21.15.45:0/3605646533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e58021070 con 0x55fc31534d90 2023-10-17T01:21:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e46ffd700 1 -- 172.21.15.45:0/3605646533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f9e600a9030 con 0x55fc31534d90 2023-10-17T01:21:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e81678b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:21:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e81678b80 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:21:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e81678b80 20 client.15972 populate_metadata read hostname 'smithi045' 2023-10-17T01:21:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc3159b8b0 0x55fc3159dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:21:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e81678b80 4 client.15972 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:21:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc3147dce0 con 0x55fc3159b8b0 2023-10-17T01:21:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.451+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc3159b8b0 0x55fc3159dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:21:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.452+0000 7f9e46ffd700 10 client.15972 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.452+0000 7f9e677fe700 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc3159b8b0 0x55fc3159dcd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e00649b7f9959ef server_cookie=638452f8fab7154d in_seq=0 out_seq=0 2023-10-17T01:21:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.466+0000 7f9e46ffd700 1 -- 172.21.15.45:0/3605646533 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc3147dce0 con 0x55fc3159b8b0 2023-10-17T01:21:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.466+0000 7f9e46ffd700 10 client.15972 handle_command_reply: tid=0 2023-10-17T01:21:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.466+0000 7f9e81678b80 1 client.15972 shutdown 2023-10-17T01:21:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 msgr2=0x55fc31599a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31599a90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e60010430 tx=0x7f9e600988e0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 shutdown_connections 2023-10-17T01:21:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc31534d90 0x55fc31599a90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fc31533830 0x55fc31592c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55fc3159b8b0 0x55fc3159dcd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 --2- 172.21.15.45:0/3605646533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc31534190 0x55fc31593150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.467+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 >> 172.21.15.45:0/3605646533 conn(0x55fc31493070 msgr2=0x55fc3152cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.468+0000 7f9e46ffd700 0 client.15972 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:21:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.468+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 shutdown_connections 2023-10-17T01:21:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.468+0000 7f9e81678b80 1 -- 172.21.15.45:0/3605646533 wait complete. 2023-10-17T01:21:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:21:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:21:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:21:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:21:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.468+0000 7f9e81678b80 20 client.15972 trim_cache size 0 max 16384 2023-10-17T01:21:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e540a0cf0 msgr2=0x7f9e540a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e540a0cf0 0x7f9e540a31b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009f90 tx=0x7f9e58000f40 comp rx=0 tx=0).stop 2023-10-17T01:21:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68108c30 msgr2=0x7f9e6819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:21:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68108c30 0x7f9e6819a2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b3f0 tx=0x7f9e6000b700 comp rx=0 tx=0).stop 2023-10-17T01:21:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 shutdown_connections 2023-10-17T01:21:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9e540a0cf0 0x7f9e540a31b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e68108c30 0x7f9e6819a2f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e68073750 0x7f9e68198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 --2- 172.21.15.45:0/1604938434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68074d30 0x7f9e68199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:21:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.469+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 >> 172.21.15.45:0/1604938434 conn(0x7f9e680fc290 msgr2=0x7f9e680fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:21:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.470+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 shutdown_connections 2023-10-17T01:21:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:21:58.470+0000 7f9e6ddff700 1 -- 172.21.15.45:0/1604938434 wait complete. 2023-10-17T01:21:59.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:21:58 smithi045 ceph-mon[98474]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 703 op/s 2023-10-17T01:21:59.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:21:58 smithi060 ceph-mon[134136]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 703 op/s 2023-10-17T01:21:59.163 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:21:58 smithi181 ceph-mon[134520]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 703 op/s 2023-10-17T01:22:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 410.0 s, 1148.2 tps, lat 86.408 ms stddev 98.778 2023-10-17T01:22:02.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:01 smithi045 ceph-mon[98474]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 856 op/s 2023-10-17T01:22:02.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:01 smithi060 ceph-mon[134136]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 856 op/s 2023-10-17T01:22:02.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:01 smithi181 ceph-mon[134520]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 856 op/s 2023-10-17T01:22:02.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:22:03.027 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:02 smithi045 ceph-mon[98474]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-17T01:22:03.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:02 smithi060 ceph-mon[134136]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-17T01:22:03.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:02 smithi181 ceph-mon[134520]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-17T01:22:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.265+0000 7fe784bff700 1 -- 172.21.15.45:0/1366065134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7800737b0 msgr2=0x7fe780073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.265+0000 7fe784bff700 1 --2- 172.21.15.45:0/1366065134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7800737b0 0x7fe780073c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe768009990 tx=0x7fe768009ca0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 -- 172.21.15.45:0/1366065134 shutdown_connections 2023-10-17T01:22:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 --2- 172.21.15.45:0/1366065134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78010d310 0x7fe78010f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 --2- 172.21.15.45:0/1366065134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe780074d90 0x7fe7800731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 --2- 172.21.15.45:0/1366065134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7800737b0 0x7fe780073c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 -- 172.21.15.45:0/1366065134 >> 172.21.15.45:0/1366065134 conn(0x7fe7800fc290 msgr2=0x7fe7800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 -- 172.21.15.45:0/1366065134 shutdown_connections 2023-10-17T01:22:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.266+0000 7fe784bff700 1 -- 172.21.15.45:0/1366065134 wait complete. 2023-10-17T01:22:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 Processor -- start 2023-10-17T01:22:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 -- start start 2023-10-17T01:22:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7800737b0 0x7fe78019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 0x7fe78019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78010d310 0x7fe7801a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801a4c10 con 0x7fe7800737b0 2023-10-17T01:22:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801a4d50 con 0x7fe780074d90 2023-10-17T01:22:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.267+0000 7fe784bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801a4ee0 con 0x7fe78010d310 2023-10-17T01:22:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 0x7fe78019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 0x7fe78019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47984/0 (socket says 172.21.15.45:47984) 2023-10-17T01:22:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 -- 172.21.15.45:0/2441681884 learned_addr learned my addr 172.21.15.45:0/2441681884 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77e59c700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7800737b0 0x7fe78019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78010d310 0x7fe7801a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 -- 172.21.15.45:0/2441681884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78010d310 msgr2=0x7fe7801a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78010d310 0x7fe7801a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 -- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7800737b0 msgr2=0x7fe78019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7800737b0 0x7fe78019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77dd9b700 1 -- 172.21.15.45:0/2441681884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680095f0 con 0x7fe780074d90 2023-10-17T01:22:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77e59c700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7800737b0 0x7fe78019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.268+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78010d310 0x7fe7801a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.269+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 0x7fe78019d5d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe768005720 tx=0x7fe768009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.270+0000 7fe7777fe700 1 -- 172.21.15.45:0/2441681884 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76801d070 con 0x7fe780074d90 2023-10-17T01:22:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.270+0000 7fe7777fe700 1 -- 172.21.15.45:0/2441681884 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe768021410 con 0x7fe780074d90 2023-10-17T01:22:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.270+0000 7fe7777fe700 1 -- 172.21.15.45:0/2441681884 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768017650 con 0x7fe780074d90 2023-10-17T01:22:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.270+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7801a5160 con 0x7fe780074d90 2023-10-17T01:22:03.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.270+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7801a5530 con 0x7fe780074d90 2023-10-17T01:22:03.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.276+0000 7fe7777fe700 1 -- 172.21.15.45:0/2441681884 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fe768021580 con 0x7fe780074d90 2023-10-17T01:22:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.277+0000 7fe79850fb80 1 Processor -- start 2023-10-17T01:22:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.277+0000 7fe79850fb80 1 -- start start 2023-10-17T01:22:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.278+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 0x55be55618fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.278+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55619500 0x55be55619d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.278+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be5561a2b0 0x55be5561a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.278+0000 7fe79850fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55570a10 con 0x55be5561a2b0 2023-10-17T01:22:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.278+0000 7fe79850fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5556e3d0 con 0x55be55618ba0 2023-10-17T01:22:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.278+0000 7fe79850fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55569cc0 con 0x55be55619500 2023-10-17T01:22:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 0x55be55618fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe7777fe700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe76c0a0c20 0x7fe76c0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 0x55be55618fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47996/0 (socket says 172.21.15.45:47996) 2023-10-17T01:22:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2214039 learned_addr learned my addr 172.21.15.45:0/2214039 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77e59c700 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55619500 0x55be55619d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be5561a2b0 0x55be5561a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe76c0a0c20 0x7fe76c0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2214039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55619500 msgr2=0x55be55619d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.279+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55619500 0x55be55619d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2214039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be5561a2b0 msgr2=0x55be5561a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be5561a2b0 0x55be5561a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2214039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be55397de0 con 0x55be55618ba0 2023-10-17T01:22:03.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe7777fe700 1 -- 172.21.15.45:0/2441681884 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe7680cd360 con 0x7fe780074d90 2023-10-17T01:22:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 0x55be55618fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe778004560 tx=0x7fe778004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e8d6580eff64cd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe76c0a0c20 0x7fe76c0a30e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe77800e7d0 tx=0x7fe77801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe774ff9700 1 -- 172.21.15.45:0/2214039 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778007700 con 0x55be55618ba0 2023-10-17T01:22:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe774ff9700 1 -- 172.21.15.45:0/2214039 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe77800f040 con 0x55be55618ba0 2023-10-17T01:22:03.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.280+0000 7fe774ff9700 1 -- 172.21.15.45:0/2214039 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7780228d0 con 0x55be55618ba0 2023-10-17T01:22:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 -- 172.21.15.45:0/2214039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 msgr2=0x55be55618fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 0x55be55618fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe778004560 tx=0x7fe778004870 comp rx=0 tx=0).stop 2023-10-17T01:22:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 -- 172.21.15.45:0/2214039 shutdown_connections 2023-10-17T01:22:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be5561a2b0 0x55be5561a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55618ba0 0x55be55618fc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2214039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55619500 0x55be55619d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 -- 172.21.15.45:0/2214039 >> 172.21.15.45:0/2214039 conn(0x55be555780d0 msgr2=0x55be55613950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:03.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 -- 172.21.15.45:0/2214039 shutdown_connections 2023-10-17T01:22:03.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.281+0000 7fe79850fb80 1 -- 172.21.15.45:0/2214039 wait complete. 2023-10-17T01:22:03.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.282+0000 7fe79850fb80 1 -- start start 2023-10-17T01:22:03.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.282+0000 7fe79850fb80 1 Processor -- start 2023-10-17T01:22:03.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be556228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55622de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be55629e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55506510 con 0x55be55619500 2023-10-17T01:22:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be555061b0 con 0x55be5561a2b0 2023-10-17T01:22:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be553ed920 con 0x55be55618ba0 2023-10-17T01:22:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe79850fb80 10 client.0 _ll_register_callbacks cb 0x55be55453a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be556228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be556228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37028/0 (socket says 172.21.15.45:37028) 2023-10-17T01:22:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 -- 172.21.15.45:0/1388665293 learned_addr learned my addr 172.21.15.45:0/1388665293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55622de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be55629e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 -- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 msgr2=0x55be55629e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be55629e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 -- 172.21.15.45:0/1388665293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 msgr2=0x55be55622de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55622de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.283+0000 7fe77e59c700 1 -- 172.21.15.45:0/1388665293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe778004140 con 0x55be55618ba0 2023-10-17T01:22:03.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.284+0000 7fe77e59c700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be556228a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe770011d10 tx=0x7fe77000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=444b5a5e4d0785e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:03.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.284+0000 7fe765ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:22:03.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.284+0000 7fe765ffb700 1 -- 172.21.15.45:0/1388665293 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770016070 con 0x55be55618ba0 2023-10-17T01:22:03.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.284+0000 7fe765ffb700 1 -- 172.21.15.45:0/1388665293 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe77000fcf0 con 0x55be55618ba0 2023-10-17T01:22:03.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.284+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be55594d10 con 0x55be55618ba0 2023-10-17T01:22:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.284+0000 7fe765ffb700 1 -- 172.21.15.45:0/1388665293 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe770004050 con 0x55be55618ba0 2023-10-17T01:22:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.285+0000 7fe79850fb80 10 client.26212 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.285+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55be55380c60 con 0x55be55618ba0 2023-10-17T01:22:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe765ffb700 1 -- 172.21.15.45:0/1388665293 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be55594d10 con 0x55be55618ba0 2023-10-17T01:22:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe765ffb700 1 -- 172.21.15.45:0/1388665293 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe770026480 con 0x55be55618ba0 2023-10-17T01:22:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe79850fb80 10 client.26212 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe79850fb80 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe79850fb80 20 client.26212 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe79850fb80 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be5562e620 0x55be55630a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe79850fb80 4 client.26212 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:22:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be555e7d30 con 0x55be5562e620 2023-10-17T01:22:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.287+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be5562e620 0x55be55630a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.288+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be5562e620 0x55be55630a40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9afcf22b3d0cad99 server_cookie=6fe5dcfa7887caef in_seq=0 out_seq=0 2023-10-17T01:22:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.288+0000 7fe765ffb700 10 client.26212 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.289+0000 7fe765ffb700 1 -- 172.21.15.45:0/1388665293 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55be555e7d30 con 0x55be5562e620 2023-10-17T01:22:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.289+0000 7fe765ffb700 10 client.26212 handle_command_reply: tid=0 2023-10-17T01:22:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.289+0000 7fe79850fb80 1 client.26212 shutdown 2023-10-17T01:22:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.289+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 msgr2=0x55be556228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.289+0000 7fe79850fb80 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be556228a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe770011d10 tx=0x7fe77000f6a0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 shutdown_connections 2023-10-17T01:22:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be5562e620 0x55be55630a40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe765ffb700 0 client.26212 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be55629e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be556228a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 --2- 172.21.15.45:0/1388665293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55622de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 >> 172.21.15.45:0/1388665293 conn(0x55be555780d0 msgr2=0x55be55613020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 shutdown_connections 2023-10-17T01:22:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 1 -- 172.21.15.45:0/1388665293 wait complete. 2023-10-17T01:22:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.290+0000 7fe79850fb80 20 client.26212 trim_cache size 0 max 16384 2023-10-17T01:22:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 Processor -- start 2023-10-17T01:22:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 -- start start 2023-10-17T01:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55618ba0 0x55be5561d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 0x55be5561d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be5561a2b0 0x55be55622650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be556204f0 con 0x55be55618ba0 2023-10-17T01:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55614650 con 0x55be55619500 2023-10-17T01:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.315+0000 7fe79850fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5562e370 con 0x55be5561a2b0 2023-10-17T01:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55618ba0 0x55be5561d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 0x55be5561d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 0x55be5561d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48024/0 (socket says 172.21.15.45:48024) 2023-10-17T01:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77dd9b700 1 -- 172.21.15.45:0/2388860545 learned_addr learned my addr 172.21.15.45:0/2388860545 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be5561a2b0 0x55be55622650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2388860545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 msgr2=0x55be5561d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 0x55be5561d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2388860545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55618ba0 msgr2=0x55be5561d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55618ba0 0x55be5561d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77ed9d700 1 -- 172.21.15.45:0/2388860545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be553be250 con 0x55be5561a2b0 2023-10-17T01:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77e59c700 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55618ba0 0x55be5561d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.316+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 0x55be5561d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.317+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be5561a2b0 0x55be55622650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe778003af0 tx=0x7fe778003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc87a8b40a8e4251 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.317+0000 7fe7757fa700 1 -- 172.21.15.45:0/2388860545 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778007700 con 0x55be5561a2b0 2023-10-17T01:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.317+0000 7fe7757fa700 1 -- 172.21.15.45:0/2388860545 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7780094e0 con 0x55be5561a2b0 2023-10-17T01:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.317+0000 7fe7757fa700 1 -- 172.21.15.45:0/2388860545 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77802b830 con 0x55be5561a2b0 2023-10-17T01:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.318+0000 7fe79850fb80 1 -- 172.21.15.45:0/2388860545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be5561a2b0 msgr2=0x55be55622650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.318+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be5561a2b0 0x55be55622650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe778003af0 tx=0x7fe778003e30 comp rx=0 tx=0).stop 2023-10-17T01:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.319+0000 7fe79850fb80 1 -- 172.21.15.45:0/2388860545 shutdown_connections 2023-10-17T01:22:03.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.319+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be5561a2b0 0x55be55622650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.319+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55618ba0 0x55be5561d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.319+0000 7fe79850fb80 1 --2- 172.21.15.45:0/2388860545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be55619500 0x55be5561d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.319+0000 7fe79850fb80 1 -- 172.21.15.45:0/2388860545 >> 172.21.15.45:0/2388860545 conn(0x55be555780d0 msgr2=0x55be5562a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:03.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.319+0000 7fe79850fb80 1 -- 172.21.15.45:0/2388860545 shutdown_connections 2023-10-17T01:22:03.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.320+0000 7fe79850fb80 1 -- 172.21.15.45:0/2388860545 wait complete. 2023-10-17T01:22:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.320+0000 7fe79850fb80 1 -- start start 2023-10-17T01:22:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.320+0000 7fe79850fb80 1 Processor -- start 2023-10-17T01:22:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be55678660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55678ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be5567fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55680120 con 0x55be55619500 2023-10-17T01:22:03.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55680260 con 0x55be5561a2b0 2023-10-17T01:22:03.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be556803f0 con 0x55be55618ba0 2023-10-17T01:22:03.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe79850fb80 10 client.0 _ll_register_callbacks cb 0x7fe7700117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:03.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be5567fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be55678660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be5567fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48026/0 (socket says 172.21.15.45:48026) 2023-10-17T01:22:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77ed9d700 1 -- 172.21.15.45:0/440777820 learned_addr learned my addr 172.21.15.45:0/440777820 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be55678660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37060/0 (socket says 172.21.15.45:37060) 2023-10-17T01:22:03.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55678ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77e59c700 1 -- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 msgr2=0x55be5567fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.321+0000 7fe77e59c700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be5567fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe77e59c700 1 -- 172.21.15.45:0/440777820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 msgr2=0x55be55678ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe77e59c700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55678ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe77e59c700 1 -- 172.21.15.45:0/440777820 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be55380c60 con 0x55be55618ba0 2023-10-17T01:22:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe77ed9d700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be5567fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe77e59c700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be55678660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55be55610490 tx=0x7fe770005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eecd4a7a9477fdff server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe765ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:22:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe765ffb700 1 -- 172.21.15.45:0/440777820 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770016040 con 0x55be55618ba0 2023-10-17T01:22:03.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe77800f040 con 0x55be55618ba0 2023-10-17T01:22:03.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.322+0000 7fe765ffb700 1 -- 172.21.15.45:0/440777820 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe77002b070 con 0x55be55618ba0 2023-10-17T01:22:03.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe765ffb700 1 -- 172.21.15.45:0/440777820 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe770026b50 con 0x55be55618ba0 2023-10-17T01:22:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 10 client.26218 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fe7801a5530 con 0x55be55618ba0 2023-10-17T01:22:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe765ffb700 1 -- 172.21.15.45:0/440777820 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe77800f040 con 0x55be55618ba0 2023-10-17T01:22:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe765ffb700 1 -- 172.21.15.45:0/440777820 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe77001a410 con 0x55be55618ba0 2023-10-17T01:22:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 10 client.26218 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:03.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 10 client.26218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 20 client.26218 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be55681d30 0x55be55684150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 4 client.26218 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:22:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be55562d40 con 0x55be55681d30 2023-10-17T01:22:03.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.323+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be55681d30 0x55be55684150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.325+0000 7fe77dd9b700 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be55681d30 0x55be55684150 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee2dc6fa4648de7 server_cookie=37ba45ee6da1fee7 in_seq=0 out_seq=0 2023-10-17T01:22:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.325+0000 7fe765ffb700 10 client.26218 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.325+0000 7fe765ffb700 1 -- 172.21.15.45:0/440777820 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55be55562d40 con 0x55be55681d30 2023-10-17T01:22:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.325+0000 7fe765ffb700 10 client.26218 handle_command_reply: tid=0 2023-10-17T01:22:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.325+0000 7fe79850fb80 1 client.26218 shutdown 2023-10-17T01:22:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 msgr2=0x55be55678660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be55678660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55be55610490 tx=0x7fe770005280 comp rx=0 tx=0).stop 2023-10-17T01:22:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 shutdown_connections 2023-10-17T01:22:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55be55681d30 0x55be55684150 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be5561a2b0 0x55be5567fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be55618ba0 0x55be55678660 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe765ffb700 0 client.26218 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 --2- 172.21.15.45:0/440777820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be55619500 0x55be55678ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.326+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 >> 172.21.15.45:0/440777820 conn(0x55be555780d0 msgr2=0x55be5561d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:03.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.327+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 shutdown_connections 2023-10-17T01:22:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.327+0000 7fe79850fb80 1 -- 172.21.15.45:0/440777820 wait complete. 2023-10-17T01:22:03.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.328+0000 7fe79850fb80 20 client.26218 trim_cache size 0 max 16384 2023-10-17T01:22:03.338 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:22:03.338 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:22:03.339 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:22:03.339 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:22:03.339 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:22:03.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:22:03.339 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:22:03.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:22:03.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:22:03.340 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:22:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.330+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe76c0a0c20 msgr2=0x7fe76c0a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.330+0000 7fe784bff700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe76c0a0c20 0x7fe76c0a30e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe77800e7d0 tx=0x7fe77801e000 comp rx=0 tx=0).stop 2023-10-17T01:22:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.330+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 msgr2=0x7fe78019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.330+0000 7fe784bff700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 0x7fe78019d5d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe768005720 tx=0x7fe768009580 comp rx=0 tx=0).stop 2023-10-17T01:22:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 shutdown_connections 2023-10-17T01:22:03.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe76c0a0c20 0x7fe76c0a30e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78010d310 0x7fe7801a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7800737b0 0x7fe78019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 --2- 172.21.15.45:0/2441681884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe780074d90 0x7fe78019d5d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 >> 172.21.15.45:0/2441681884 conn(0x7fe7800fc290 msgr2=0x7fe7800fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 shutdown_connections 2023-10-17T01:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:03.331+0000 7fe784bff700 1 -- 172.21.15.45:0/2441681884 wait complete. 2023-10-17T01:22:03.363 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:22:05.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:04 smithi045 ceph-mon[98474]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 878 op/s 2023-10-17T01:22:05.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:04 smithi060 ceph-mon[134136]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 878 op/s 2023-10-17T01:22:05.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:04 smithi181 ceph-mon[134520]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 878 op/s 2023-10-17T01:22:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 415.0 s, 1684.6 tps, lat 60.385 ms stddev 72.256 2023-10-17T01:22:07.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:06 smithi045 ceph-mon[98474]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 761 op/s 2023-10-17T01:22:07.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:06 smithi060 ceph-mon[134136]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 761 op/s 2023-10-17T01:22:07.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:06 smithi181 ceph-mon[134520]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 761 op/s 2023-10-17T01:22:09.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:08 smithi045 ceph-mon[98474]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-10-17T01:22:09.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:08 smithi060 ceph-mon[134136]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-10-17T01:22:09.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:08 smithi181 ceph-mon[134520]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-10-17T01:22:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 420.0 s, 1609.0 tps, lat 61.439 ms stddev 79.836 2023-10-17T01:22:11.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:10 smithi045 ceph-mon[98474]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 849 op/s 2023-10-17T01:22:11.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:10 smithi060 ceph-mon[134136]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 849 op/s 2023-10-17T01:22:11.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:10 smithi181 ceph-mon[134520]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 849 op/s 2023-10-17T01:22:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.718+0000 7f82cd7ff700 1 -- 172.21.15.45:0/1203937514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 msgr2=0x7f82c80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.718+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/1203937514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c80731e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009990 tx=0x7f82b8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 -- 172.21.15.45:0/1203937514 shutdown_connections 2023-10-17T01:22:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/1203937514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c810d330 0x7f82c810f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/1203937514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c80731e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/1203937514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80737b0 0x7f82c8073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 -- 172.21.15.45:0/1203937514 >> 172.21.15.45:0/1203937514 conn(0x7f82c80fc290 msgr2=0x7f82c80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 -- 172.21.15.45:0/1203937514 shutdown_connections 2023-10-17T01:22:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.719+0000 7f82cd7ff700 1 -- 172.21.15.45:0/1203937514 wait complete. 2023-10-17T01:22:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.720+0000 7f82cd7ff700 1 Processor -- start 2023-10-17T01:22:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.720+0000 7f82cd7ff700 1 -- start start 2023-10-17T01:22:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.720+0000 7f82cd7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80737b0 0x7f82c819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.720+0000 7f82cd7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.720+0000 7f82cd7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c810d330 0x7f82c819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82cd7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819ec90 con 0x7f82c810d330 2023-10-17T01:22:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82cd7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819ee00 con 0x7f82c8074d90 2023-10-17T01:22:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82cd7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819efc0 con 0x7f82c80737b0 2023-10-17T01:22:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80737b0 0x7f82c819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53716/0 (socket says 172.21.15.45:53716) 2023-10-17T01:22:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 -- 172.21.15.45:0/2938207373 learned_addr learned my addr 172.21.15.45:0/2938207373 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c77fe700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c810d330 0x7f82c819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 -- 172.21.15.45:0/2938207373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80737b0 msgr2=0x7f82c819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80737b0 0x7f82c819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 -- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c810d330 msgr2=0x7f82c819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c810d330 0x7f82c819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.721+0000 7f82c67fc700 1 -- 172.21.15.45:0/2938207373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80095f0 con 0x7f82c8074d90 2023-10-17T01:22:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.722+0000 7f82c67fc700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c819d7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b500 tx=0x7f82bc00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.722+0000 7f82affff700 1 -- 172.21.15.45:0/2938207373 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc011830 con 0x7f82c8074d90 2023-10-17T01:22:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.722+0000 7f82affff700 1 -- 172.21.15.45:0/2938207373 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82bc011e50 con 0x7f82c8074d90 2023-10-17T01:22:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.722+0000 7f82affff700 1 -- 172.21.15.45:0/2938207373 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc010dc0 con 0x7f82c8074d90 2023-10-17T01:22:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.722+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c819f2a0 con 0x7f82c8074d90 2023-10-17T01:22:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.722+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c81a5a60 con 0x7f82c8074d90 2023-10-17T01:22:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.726+0000 7f82affff700 1 -- 172.21.15.45:0/2938207373 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f82bc010380 con 0x7f82c8074d90 2023-10-17T01:22:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.729+0000 7f82affff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82b00a0ba0 0x7f82b00a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.729+0000 7f82affff700 1 -- 172.21.15.45:0/2938207373 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82bc0cbbd0 con 0x7f82c8074d90 2023-10-17T01:22:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.730+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82b00a0ba0 0x7f82b00a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.730+0000 7f82e1196b80 1 Processor -- start 2023-10-17T01:22:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 -- start start 2023-10-17T01:22:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82b00a0ba0 0x7f82b00a3060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82b8003680 tx=0x7f82b800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fb4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fb5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2f0b9b0 con 0x55c9b2fb4190 2023-10-17T01:22:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2f09370 con 0x55c9b2fb3830 2023-10-17T01:22:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.731+0000 7f82e1196b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2f04c60 con 0x55c9b2fb4d90 2023-10-17T01:22:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fb5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fb5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51828/0 (socket says 172.21.15.45:51828) 2023-10-17T01:22:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 -- 172.21.15.45:0/1043221303 learned_addr learned my addr 172.21.15.45:0/1043221303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c67fc700 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fb4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 -- 172.21.15.45:0/1043221303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 msgr2=0x55c9b2fb3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 -- 172.21.15.45:0/1043221303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 msgr2=0x55c9b2fb4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fb4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.732+0000 7f82c77fe700 1 -- 172.21.15.45:0/1043221303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b2d32d80 con 0x55c9b2fb4d90 2023-10-17T01:22:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.733+0000 7f82c77fe700 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fb5210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009970 tx=0x7f82c0009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42441ce9827d2291 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.733+0000 7f82ad7fa700 1 -- 172.21.15.45:0/1043221303 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0007500 con 0x55c9b2fb4d90 2023-10-17T01:22:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.733+0000 7f82ad7fa700 1 -- 172.21.15.45:0/1043221303 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c000f040 con 0x55c9b2fb4d90 2023-10-17T01:22:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.733+0000 7f82ad7fa700 1 -- 172.21.15.45:0/1043221303 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0015910 con 0x55c9b2fb4d90 2023-10-17T01:22:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.734+0000 7f82e1196b80 1 -- 172.21.15.45:0/1043221303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 msgr2=0x55c9b2fb5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.734+0000 7f82e1196b80 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fb5210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009970 tx=0x7f82c0009c80 comp rx=0 tx=0).stop 2023-10-17T01:22:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 -- 172.21.15.45:0/1043221303 shutdown_connections 2023-10-17T01:22:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fb5210 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 --2- 172.21.15.45:0/1043221303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fb4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 -- 172.21.15.45:0/1043221303 >> 172.21.15.45:0/1043221303 conn(0x55c9b2f13070 msgr2=0x55c9b2fae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 -- 172.21.15.45:0/1043221303 shutdown_connections 2023-10-17T01:22:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.735+0000 7f82e1196b80 1 -- 172.21.15.45:0/1043221303 wait complete. 2023-10-17T01:22:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.736+0000 7f82e1196b80 1 -- start start 2023-10-17T01:22:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.736+0000 7f82e1196b80 1 Processor -- start 2023-10-17T01:22:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fbd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2ea14b0 con 0x55c9b2fb4190 2023-10-17T01:22:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2ea1150 con 0x55c9b2fb3830 2023-10-17T01:22:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2d888c0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82e1196b80 10 client.0 _ll_register_callbacks cb 0x55c9b2deea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fbd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fbd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53736/0 (socket says 172.21.15.45:53736) 2023-10-17T01:22:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c6ffd700 1 -- 172.21.15.45:0/564416131 learned_addr learned my addr 172.21.15.45:0/564416131 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c77fe700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c67fc700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c77fe700 1 -- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 msgr2=0x55c9b2fbd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c77fe700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fbd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c77fe700 1 -- 172.21.15.45:0/564416131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 msgr2=0x55c9b2fbd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c77fe700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c77fe700 1 -- 172.21.15.45:0/564416131 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c0003c50 con 0x55c9b2fb4d90 2023-10-17T01:22:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.737+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fbd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82c77fe700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fc4970 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009970 tx=0x7f82c0015930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3e9d34748d165d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:22:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82aaff5700 1 -- 172.21.15.45:0/564416131 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0007500 con 0x55c9b2fb4d90 2023-10-17T01:22:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82aaff5700 1 -- 172.21.15.45:0/564416131 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82c0022070 con 0x55c9b2fb4d90 2023-10-17T01:22:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9b2f2fcb0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82aaff5700 1 -- 172.21.15.45:0/564416131 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c0007660 con 0x55c9b2fb4d90 2023-10-17T01:22:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82e1196b80 10 client.26227 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.738+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c9b2d1bc00 con 0x55c9b2fb4d90 2023-10-17T01:22:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82aaff5700 1 -- 172.21.15.45:0/564416131 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9b2f2fcb0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82aaff5700 1 -- 172.21.15.45:0/564416131 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f82c0027a10 con 0x55c9b2fb4d90 2023-10-17T01:22:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82e1196b80 10 client.26227 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82e1196b80 10 client.26227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82e1196b80 20 client.26227 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82e1196b80 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b2fc9060 0x55c9b2fcb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82e1196b80 4 client.26227 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:22:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.739+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9b2f82cd0 con 0x55c9b2fc9060 2023-10-17T01:22:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.740+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b2fc9060 0x55c9b2fcb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.741+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b2fc9060 0x55c9b2fcb480 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ef5941a5e06389 server_cookie=9c325e036cad6346 in_seq=0 out_seq=0 2023-10-17T01:22:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.741+0000 7f82aaff5700 10 client.26227 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.742+0000 7f82aaff5700 1 -- 172.21.15.45:0/564416131 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55c9b2f82cd0 con 0x55c9b2fc9060 2023-10-17T01:22:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.742+0000 7f82aaff5700 10 client.26227 handle_command_reply: tid=0 2023-10-17T01:22:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.742+0000 7f82e1196b80 1 client.26227 shutdown 2023-10-17T01:22:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 msgr2=0x55c9b2fc4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fc4970 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009970 tx=0x7f82c0015930 comp rx=0 tx=0).stop 2023-10-17T01:22:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 shutdown_connections 2023-10-17T01:22:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fc4970 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b2fc9060 0x55c9b2fcb480 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fbd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 --2- 172.21.15.45:0/564416131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 >> 172.21.15.45:0/564416131 conn(0x55c9b2f13070 msgr2=0x55c9b2fada00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.743+0000 7f82aaff5700 0 client.26227 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.744+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 shutdown_connections 2023-10-17T01:22:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.744+0000 7f82e1196b80 1 -- 172.21.15.45:0/564416131 wait complete. 2023-10-17T01:22:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.744+0000 7f82e1196b80 20 client.26227 trim_cache size 0 max 16384 2023-10-17T01:22:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.768+0000 7f82e1196b80 1 Processor -- start 2023-10-17T01:22:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.768+0000 7f82e1196b80 1 -- start start 2023-10-17T01:22:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.768+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fbd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82e1196b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2fa89e0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82e1196b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2fbc8a0 con 0x55c9b2fb3830 2023-10-17T01:22:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82e1196b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b2fbcf60 con 0x55c9b2fb4190 2023-10-17T01:22:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82c6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53738/0 (socket says 172.21.15.45:53738) 2023-10-17T01:22:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82c6ffd700 1 -- 172.21.15.45:0/2635159641 learned_addr learned my addr 172.21.15.45:0/2635159641 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82c6ffd700 1 -- 172.21.15.45:0/2635159641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4190 msgr2=0x55c9b2fbb200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:22:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.769+0000 7f82c67fc700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.770+0000 7f82c77fe700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fbd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.770+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.770+0000 7f82c6ffd700 1 -- 172.21.15.45:0/2635159641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4d90 msgr2=0x55c9b2fbd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.770+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fbd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.770+0000 7f82c6ffd700 1 -- 172.21.15.45:0/2635159641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b2d591f0 con 0x55c9b2fb3830 2023-10-17T01:22:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.771+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb5170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009990 tx=0x7f82b8020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64fb45604523dd93 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.771+0000 7f82c77fe700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fbd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:22:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.771+0000 7f82c67fc700 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:22:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.771+0000 7f82adffb700 1 -- 172.21.15.45:0/2635159641 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80342e0 con 0x55c9b2fb3830 2023-10-17T01:22:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.771+0000 7f82adffb700 1 -- 172.21.15.45:0/2635159641 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b802b040 con 0x55c9b2fb3830 2023-10-17T01:22:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.771+0000 7f82adffb700 1 -- 172.21.15.45:0/2635159641 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80248b0 con 0x55c9b2fb3830 2023-10-17T01:22:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 -- 172.21.15.45:0/2635159641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 msgr2=0x55c9b2fb5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb5170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009990 tx=0x7f82b8020000 comp rx=0 tx=0).stop 2023-10-17T01:22:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 -- 172.21.15.45:0/2635159641 shutdown_connections 2023-10-17T01:22:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb4d90 0x55c9b2fbd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb3830 0x55c9b2fb5170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 --2- 172.21.15.45:0/2635159641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4190 0x55c9b2fbb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.772+0000 7f82e1196b80 1 -- 172.21.15.45:0/2635159641 >> 172.21.15.45:0/2635159641 conn(0x55c9b2f13070 msgr2=0x55c9b2fbc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.773+0000 7f82e1196b80 1 -- 172.21.15.45:0/2635159641 shutdown_connections 2023-10-17T01:22:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.773+0000 7f82e1196b80 1 -- 172.21.15.45:0/2635159641 wait complete. 2023-10-17T01:22:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.773+0000 7f82e1196b80 1 -- start start 2023-10-17T01:22:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.773+0000 7f82e1196b80 1 Processor -- start 2023-10-17T01:22:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.774+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb3830 0x55c9b30149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.774+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb4190 0x55c9b3016f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82e1196b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b30194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82e1196b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b3019bb0 con 0x55c9b2fb3830 2023-10-17T01:22:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82e1196b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b3019d20 con 0x55c9b2fb4190 2023-10-17T01:22:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82e1196b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b3019ee0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82e1196b80 10 client.0 _ll_register_callbacks cb 0x7f82c0003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b30194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b30194a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51846/0 (socket says 172.21.15.45:51846) 2023-10-17T01:22:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.775+0000 7f82c77fe700 1 -- 172.21.15.45:0/561555270 learned_addr learned my addr 172.21.15.45:0/561555270 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb3830 0x55c9b30149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c67fc700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb4190 0x55c9b3016f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c77fe700 1 -- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb4190 msgr2=0x55c9b3016f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c77fe700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb4190 0x55c9b3016f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c77fe700 1 -- 172.21.15.45:0/561555270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb3830 msgr2=0x55c9b30149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c77fe700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb3830 0x55c9b30149c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.776+0000 7f82c77fe700 1 -- 172.21.15.45:0/561555270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c81100d0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.777+0000 7f82c77fe700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b30194a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f82c0003f40 tx=0x7f82c0009130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714a02a51ec0ef25 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.777+0000 7f82aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:22:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.777+0000 7f82aaff5700 1 -- 172.21.15.45:0/561555270 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0027480 con 0x55c9b2fb4d90 2023-10-17T01:22:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.777+0000 7f82aaff5700 1 -- 172.21.15.45:0/561555270 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82c0022070 con 0x55c9b2fb4d90 2023-10-17T01:22:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.777+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82c000f040 con 0x55c9b2fb4d90 2023-10-17T01:22:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82aaff5700 1 -- 172.21.15.45:0/561555270 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c00275e0 con 0x55c9b2fb4d90 2023-10-17T01:22:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 10 client.26230 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c9b2d31250 con 0x55c9b2fb4d90 2023-10-17T01:22:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82aaff5700 1 -- 172.21.15.45:0/561555270 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c000f040 con 0x55c9b2fb4d90 2023-10-17T01:22:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82aaff5700 1 -- 172.21.15.45:0/561555270 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f82c0007500 con 0x55c9b2fb4d90 2023-10-17T01:22:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 10 client.26230 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 20 client.26230 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b301b3a0 0x55c9b301d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 4 client.26230 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:22:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.778+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c9b2efdce0 con 0x55c9b301b3a0 2023-10-17T01:22:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.779+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b301b3a0 0x55c9b301d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.779+0000 7f82c6ffd700 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b301b3a0 0x55c9b301d7c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd9db517c377db3 server_cookie=4ae551617eefbe1b in_seq=0 out_seq=0 2023-10-17T01:22:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.779+0000 7f82aaff5700 10 client.26230 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.790+0000 7f82aaff5700 1 -- 172.21.15.45:0/561555270 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c9b2efdce0 con 0x55c9b301b3a0 2023-10-17T01:22:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.790+0000 7f82aaff5700 10 client.26230 handle_command_reply: tid=0 2023-10-17T01:22:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.790+0000 7f82e1196b80 1 client.26230 shutdown 2023-10-17T01:22:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.790+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 msgr2=0x55c9b30194a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.790+0000 7f82e1196b80 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b30194a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f82c0003f40 tx=0x7f82c0009130 comp rx=0 tx=0).stop 2023-10-17T01:22:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 shutdown_connections 2023-10-17T01:22:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c9b2fb4d90 0x55c9b30194a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9b2fb3830 0x55c9b30149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c9b301b3a0 0x55c9b301d7c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 --2- 172.21.15.45:0/561555270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c9b2fb4190 0x55c9b3016f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 >> 172.21.15.45:0/561555270 conn(0x55c9b2f13070 msgr2=0x55c9b2fbb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82aaff5700 0 client.26230 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 shutdown_connections 2023-10-17T01:22:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 1 -- 172.21.15.45:0/561555270 wait complete. 2023-10-17T01:22:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.791+0000 7f82e1196b80 20 client.26230 trim_cache size 0 max 16384 2023-10-17T01:22:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:22:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:22:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:22:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:22:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82b00a0ba0 msgr2=0x7f82b00a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82b00a0ba0 0x7f82b00a3060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82b8003680 tx=0x7f82b800b540 comp rx=0 tx=0).stop 2023-10-17T01:22:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 msgr2=0x7f82c819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c819d7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b500 tx=0x7f82bc00b810 comp rx=0 tx=0).stop 2023-10-17T01:22:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 shutdown_connections 2023-10-17T01:22:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82b00a0ba0 0x7f82b00a3060 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c810d330 0x7f82c819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80737b0 0x7f82c819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 --2- 172.21.15.45:0/2938207373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c8074d90 0x7f82c819d7b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 >> 172.21.15.45:0/2938207373 conn(0x7f82c80fc290 msgr2=0x7f82c80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 shutdown_connections 2023-10-17T01:22:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:11.792+0000 7f82cd7ff700 1 -- 172.21.15.45:0/2938207373 wait complete. 2023-10-17T01:22:13.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:12 smithi045 ceph-mon[98474]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 710 op/s 2023-10-17T01:22:13.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:12 smithi060 ceph-mon[134136]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 710 op/s 2023-10-17T01:22:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:12 smithi181 ceph-mon[134520]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 710 op/s 2023-10-17T01:22:15.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:14 smithi045 ceph-mon[98474]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 790 op/s 2023-10-17T01:22:15.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:14 smithi060 ceph-mon[134136]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 790 op/s 2023-10-17T01:22:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:14 smithi181 ceph-mon[134520]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 790 op/s 2023-10-17T01:22:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 425.0 s, 1655.0 tps, lat 60.768 ms stddev 70.628 2023-10-17T01:22:17.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:16 smithi045 ceph-mon[98474]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 660 op/s 2023-10-17T01:22:17.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:16 smithi060 ceph-mon[134136]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 660 op/s 2023-10-17T01:22:17.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:16 smithi181 ceph-mon[134520]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 660 op/s 2023-10-17T01:22:19.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:18 smithi045 ceph-mon[98474]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 950 op/s 2023-10-17T01:22:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:18 smithi060 ceph-mon[134136]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 950 op/s 2023-10-17T01:22:19.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:18 smithi181 ceph-mon[134520]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 950 op/s 2023-10-17T01:22:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 430.0 s, 1350.0 tps, lat 73.285 ms stddev 128.152 2023-10-17T01:22:22.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:21 smithi045 ceph-mon[98474]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2023-10-17T01:22:22.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:21 smithi060 ceph-mon[134136]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2023-10-17T01:22:22.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:21 smithi181 ceph-mon[134520]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 1.74k op/s 2023-10-17T01:22:23.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:22 smithi045 ceph-mon[98474]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.85k op/s 2023-10-17T01:22:23.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:22 smithi060 ceph-mon[134136]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.85k op/s 2023-10-17T01:22:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:22 smithi181 ceph-mon[134520]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.85k op/s 2023-10-17T01:22:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 -- 172.21.15.45:0/3347926519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be40737b0 msgr2=0x7f1be4073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 --2- 172.21.15.45:0/3347926519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be40737b0 0x7f1be4073c30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40098f0 tx=0x7f1bd4009c00 comp rx=0 tx=0).stop 2023-10-17T01:22:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 -- 172.21.15.45:0/3347926519 shutdown_connections 2023-10-17T01:22:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 --2- 172.21.15.45:0/3347926519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1be410d350 0x7f1be410f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 --2- 172.21.15.45:0/3347926519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 0x7f1be40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 --2- 172.21.15.45:0/3347926519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be40737b0 0x7f1be4073c30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 -- 172.21.15.45:0/3347926519 >> 172.21.15.45:0/3347926519 conn(0x7f1be40fc2d0 msgr2=0x7f1be40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 -- 172.21.15.45:0/3347926519 shutdown_connections 2023-10-17T01:22:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.012+0000 7f1bebbff700 1 -- 172.21.15.45:0/3347926519 wait complete. 2023-10-17T01:22:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.013+0000 7f1bebbff700 1 Processor -- start 2023-10-17T01:22:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 -- start start 2023-10-17T01:22:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1be40737b0 0x7f1be419d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 0x7f1be419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 0x7f1be419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419ec40 con 0x7f1be40737b0 2023-10-17T01:22:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419edb0 con 0x7f1be4074d90 2023-10-17T01:22:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.014+0000 7f1bebbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419ef70 con 0x7f1be410d350 2023-10-17T01:22:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1be919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 0x7f1be419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 0x7f1be419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 0x7f1be419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56598/0 (socket says 172.21.15.45:56598) 2023-10-17T01:22:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1bea19c700 1 -- 172.21.15.45:0/1277885293 learned_addr learned my addr 172.21.15.45:0/1277885293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1be999b700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1be40737b0 0x7f1be419d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1be919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 0x7f1be419d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60804/0 (socket says 172.21.15.45:60804) 2023-10-17T01:22:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1bea19c700 1 -- 172.21.15.45:0/1277885293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 msgr2=0x7f1be419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1bea19c700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 0x7f1be419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.015+0000 7f1bea19c700 1 -- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1be40737b0 msgr2=0x7f1be419d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bea19c700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1be40737b0 0x7f1be419d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bea19c700 1 -- 172.21.15.45:0/1277885293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4009520 con 0x7f1be410d350 2023-10-17T01:22:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bea19c700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 0x7f1be419e5e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00ef30 tx=0x7f1bdc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bdaffd700 1 -- 172.21.15.45:0/1277885293 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc018070 con 0x7f1be410d350 2023-10-17T01:22:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bdaffd700 1 -- 172.21.15.45:0/1277885293 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc00f040 con 0x7f1be410d350 2023-10-17T01:22:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be419f1f0 con 0x7f1be410d350 2023-10-17T01:22:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.016+0000 7f1bdaffd700 1 -- 172.21.15.45:0/1277885293 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00ccf0 con 0x7f1be410d350 2023-10-17T01:22:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.017+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be41a5a10 con 0x7f1be410d350 2023-10-17T01:22:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.020+0000 7f1bdaffd700 1 -- 172.21.15.45:0/1277885293 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f1bdc003a00 con 0x7f1be410d350 2023-10-17T01:22:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.022+0000 7f1bff5beb80 1 Processor -- start 2023-10-17T01:22:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.022+0000 7f1bdaffd700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1bd00a0b20 0x7f1bd00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.022+0000 7f1bff5beb80 1 -- start start 2023-10-17T01:22:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.022+0000 7f1bdaffd700 1 -- 172.21.15.45:0/1277885293 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1bdc0cba30 con 0x7f1be410d350 2023-10-17T01:22:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd7830 0x559b1fdd7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fdd8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdd9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fd2f9b0 con 0x559b1fdd8190 2023-10-17T01:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fd2d370 con 0x559b1fdd8d90 2023-10-17T01:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fd28c60 con 0x559b1fdd7830 2023-10-17T01:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1be999b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd7830 0x559b1fdd7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdd9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdd9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60810/0 (socket says 172.21.15.45:60810) 2023-10-17T01:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 -- 172.21.15.45:0/3371168965 learned_addr learned my addr 172.21.15.45:0/3371168965 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1be999b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd7830 0x559b1fdd7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56600/0 (socket says 172.21.15.45:56600) 2023-10-17T01:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1be919a700 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fdd8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1be919a700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1bd00a0b20 0x7f1bd00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 -- 172.21.15.45:0/3371168965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd7830 msgr2=0x559b1fdd7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd7830 0x559b1fdd7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 -- 172.21.15.45:0/3371168965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 msgr2=0x559b1fdd8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fdd8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.023+0000 7f1bea19c700 1 -- 172.21.15.45:0/3371168965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b1fb56d80 con 0x559b1fdd8d90 2023-10-17T01:22:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.024+0000 7f1bea19c700 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdd9210 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0a0bf0 tx=0x7f1bdc004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4fb216cb2d0975 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.024+0000 7f1bcffff700 1 -- 172.21.15.45:0/3371168965 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0133e0 con 0x559b1fdd8d90 2023-10-17T01:22:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.024+0000 7f1bcffff700 1 -- 172.21.15.45:0/3371168965 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc09f070 con 0x559b1fdd8d90 2023-10-17T01:22:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.024+0000 7f1bcffff700 1 -- 172.21.15.45:0/3371168965 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0a11b0 con 0x559b1fdd8d90 2023-10-17T01:22:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.024+0000 7f1be919a700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1bd00a0b20 0x7f1bd00a2fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009fd0 tx=0x7f1bd4005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 -- 172.21.15.45:0/3371168965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 msgr2=0x559b1fdd9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdd9210 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0a0bf0 tx=0x7f1bdc004dc0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 -- 172.21.15.45:0/3371168965 shutdown_connections 2023-10-17T01:22:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdd9210 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd7830 0x559b1fdd7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/3371168965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fdd8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 -- 172.21.15.45:0/3371168965 >> 172.21.15.45:0/3371168965 conn(0x559b1fd37070 msgr2=0x559b1fdd22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 -- 172.21.15.45:0/3371168965 shutdown_connections 2023-10-17T01:22:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 -- 172.21.15.45:0/3371168965 wait complete. 2023-10-17T01:22:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.025+0000 7f1bff5beb80 1 -- start start 2023-10-17T01:22:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 Processor -- start 2023-10-17T01:22:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdf2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 0x559b1fdf28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdf98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fcc54b0 con 0x559b1fdd7830 2023-10-17T01:22:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fcc5150 con 0x559b1fdd8d90 2023-10-17T01:22:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fbac8c0 con 0x559b1fdd8190 2023-10-17T01:22:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.026+0000 7f1bff5beb80 10 client.0 _ll_register_callbacks cb 0x559b1fc12a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be999b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdf2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 0x559b1fdf28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 0x559b1fdf28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56614/0 (socket says 172.21.15.45:56614) 2023-10-17T01:22:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 -- 172.21.15.45:0/1515921014 learned_addr learned my addr 172.21.15.45:0/1515921014 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1bea19c700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdf98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 -- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 msgr2=0x559b1fdf98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdf98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 -- 172.21.15.45:0/1515921014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 msgr2=0x559b1fdf2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdf2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be919a700 1 -- 172.21.15.45:0/1515921014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be41a5a10 con 0x559b1fdd8190 2023-10-17T01:22:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1bea19c700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdf98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.027+0000 7f1be999b700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdf2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.033+0000 7f1be919a700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 0x559b1fdf28b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1be4074ab0 tx=0x7f1bd4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8d8a3e88a82c9a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.033+0000 7f1bcd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:22:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.034+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/1515921014 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4005040 con 0x559b1fdd8190 2023-10-17T01:22:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.034+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/1515921014 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1bd40051a0 con 0x559b1fdd8190 2023-10-17T01:22:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.034+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b1fd53cb0 con 0x559b1fdd8190 2023-10-17T01:22:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.035+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/1515921014 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bd4005360 con 0x559b1fdd8190 2023-10-17T01:22:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.035+0000 7f1bff5beb80 10 client.26242 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.035+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559b1fb3fc00 con 0x559b1fdd8190 2023-10-17T01:22:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.035+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/1515921014 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b1fd53cb0 con 0x559b1fdd8190 2023-10-17T01:22:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/1515921014 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1bd4026420 con 0x559b1fdd8190 2023-10-17T01:22:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bff5beb80 10 client.26242 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bff5beb80 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bff5beb80 20 client.26242 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fdfe0d0 0x559b1fe004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bff5beb80 4 client.26242 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:22:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b1fda6cd0 con 0x559b1fdfe0d0 2023-10-17T01:22:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.036+0000 7f1be999b700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fdfe0d0 0x559b1fe004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.038+0000 7f1be999b700 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fdfe0d0 0x559b1fe004f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632aa9e5ff477c73 server_cookie=5c8cd5f6d2383b1 in_seq=0 out_seq=0 2023-10-17T01:22:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.038+0000 7f1bcd7fa700 10 client.26242 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/1515921014 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x559b1fda6cd0 con 0x559b1fdfe0d0 2023-10-17T01:22:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bcd7fa700 10 client.26242 handle_command_reply: tid=0 2023-10-17T01:22:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 client.26242 shutdown 2023-10-17T01:22:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 msgr2=0x559b1fdf28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 0x559b1fdf28b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1be4074ab0 tx=0x7f1bd4009c00 comp rx=0 tx=0).stop 2023-10-17T01:22:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 shutdown_connections 2023-10-17T01:22:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fdfe0d0 0x559b1fe004f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8d90 0x559b1fdf98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bcd7fa700 0 client.26242 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdf2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1515921014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8190 0x559b1fdf28b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 >> 172.21.15.45:0/1515921014 conn(0x559b1fd37070 msgr2=0x559b1fdd1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 shutdown_connections 2023-10-17T01:22:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.040+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1515921014 wait complete. 2023-10-17T01:22:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.041+0000 7f1bff5beb80 20 client.26242 trim_cache size 0 max 16384 2023-10-17T01:22:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 Processor -- start 2023-10-17T01:22:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 -- start start 2023-10-17T01:22:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdfa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8190 0x559b1fddbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fddc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fdd3d40 con 0x559b1fdd7830 2023-10-17T01:22:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fdfb5e0 con 0x559b1fdd8190 2023-10-17T01:22:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.061+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fdcf010 con 0x559b1fdd8d90 2023-10-17T01:22:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fddc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fddc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56618/0 (socket says 172.21.15.45:56618) 2023-10-17T01:22:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1bea19c700 1 -- 172.21.15.45:0/1919921111 learned_addr learned my addr 172.21.15.45:0/1919921111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1be919a700 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8190 0x559b1fddbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1be919a700 1 -- 172.21.15.45:0/1919921111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 msgr2=0x559b1fddc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1be919a700 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fddc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.062+0000 7f1be919a700 1 -- 172.21.15.45:0/1919921111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 msgr2=0x559b1fdfa120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:22:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1be919a700 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdfa120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1be919a700 1 -- 172.21.15.45:0/1919921111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b1fb7d1f0 con 0x559b1fdd8190 2023-10-17T01:22:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1bea19c700 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fddc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1be919a700 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8190 0x559b1fddbd50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009810 tx=0x7f1bd400f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c45f11c66996cf26 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1bd8bff700 1 -- 172.21.15.45:0/1919921111 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400f460 con 0x559b1fdd8190 2023-10-17T01:22:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1bd8bff700 1 -- 172.21.15.45:0/1919921111 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd4026aa0 con 0x559b1fdd8190 2023-10-17T01:22:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.063+0000 7f1bd8bff700 1 -- 172.21.15.45:0/1919921111 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4003680 con 0x559b1fdd8190 2023-10-17T01:22:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1919921111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8190 msgr2=0x559b1fddbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8190 0x559b1fddbd50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009810 tx=0x7f1bd400f960 comp rx=0 tx=0).stop 2023-10-17T01:22:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1919921111 shutdown_connections 2023-10-17T01:22:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fddc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd7830 0x559b1fdfa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/1919921111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd8190 0x559b1fddbd50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1919921111 >> 172.21.15.45:0/1919921111 conn(0x559b1fd37070 msgr2=0x559b1fdf1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1919921111 shutdown_connections 2023-10-17T01:22:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.064+0000 7f1bff5beb80 1 -- 172.21.15.45:0/1919921111 wait complete. 2023-10-17T01:22:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.065+0000 7f1bff5beb80 1 -- start start 2023-10-17T01:22:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.065+0000 7f1bff5beb80 1 Processor -- start 2023-10-17T01:22:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 0x559b1fe49a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fe4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fe4e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fe4ea30 con 0x559b1fdd8190 2023-10-17T01:22:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fe4eb70 con 0x559b1fdd7830 2023-10-17T01:22:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1fe4ecb0 con 0x559b1fdd8d90 2023-10-17T01:22:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bff5beb80 10 client.0 _ll_register_callbacks cb 0x7f1bd4010e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fe4e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 0x559b1fe49a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 0x559b1fe49a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:60836/0 (socket says 172.21.15.45:60836) 2023-10-17T01:22:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bea19c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fe4e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56622/0 (socket says 172.21.15.45:56622) 2023-10-17T01:22:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 -- 172.21.15.45:0/2908858111 learned_addr learned my addr 172.21.15.45:0/2908858111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be919a700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fe4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 -- 172.21.15.45:0/2908858111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 msgr2=0x559b1fe4e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fe4e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 -- 172.21.15.45:0/2908858111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 msgr2=0x559b1fe4bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fe4bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1be999b700 1 -- 172.21.15.45:0/2908858111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b1fb56d80 con 0x559b1fdd7830 2023-10-17T01:22:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.066+0000 7f1bea19c700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fe4e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.067+0000 7f1be999b700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 0x559b1fe49a50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x559b1fe00dc0 tx=0x7f1be0004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2e5f4c0a2cc5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.067+0000 7f1bcd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:22:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.067+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/2908858111 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0017070 con 0x559b1fdd7830 2023-10-17T01:22:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.067+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/2908858111 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1be0012cd0 con 0x559b1fdd7830 2023-10-17T01:22:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.067+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bdc09f070 con 0x559b1fdd7830 2023-10-17T01:22:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/2908858111 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be0012e90 con 0x559b1fdd7830 2023-10-17T01:22:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 10 client.26285 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559b1fb3fc00 con 0x559b1fdd7830 2023-10-17T01:22:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/2908858111 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc09f070 con 0x559b1fdd7830 2023-10-17T01:22:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/2908858111 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1be0003ba0 con 0x559b1fdd7830 2023-10-17T01:22:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 10 client.26285 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 10 client.26285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 20 client.26285 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fe50710 0x559b1fe52b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 4 client.26285 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:22:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.068+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b1fd21ce0 con 0x559b1fe50710 2023-10-17T01:22:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.069+0000 7f1be919a700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fe50710 0x559b1fe52b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.069+0000 7f1be919a700 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fe50710 0x559b1fe52b30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41088f822dea4ae2 server_cookie=a5317d48136cc573 in_seq=0 out_seq=0 2023-10-17T01:22:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.069+0000 7f1bcd7fa700 10 client.26285 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bcd7fa700 1 -- 172.21.15.45:0/2908858111 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b1fd21ce0 con 0x559b1fe50710 2023-10-17T01:22:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bcd7fa700 10 client.26285 handle_command_reply: tid=0 2023-10-17T01:22:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 client.26285 shutdown 2023-10-17T01:22:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 msgr2=0x559b1fe49a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 0x559b1fe49a50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x559b1fe00dc0 tx=0x7f1be0004550 comp rx=0 tx=0).stop 2023-10-17T01:22:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 shutdown_connections 2023-10-17T01:22:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559b1fdd8d90 0x559b1fe4e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559b1fe50710 0x559b1fe52b30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b1fdd7830 0x559b1fe49a50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 --2- 172.21.15.45:0/2908858111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b1fdd8190 0x559b1fe4bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bcd7fa700 0 client.26285 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 >> 172.21.15.45:0/2908858111 conn(0x559b1fd37070 msgr2=0x559b1fdfa120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 shutdown_connections 2023-10-17T01:22:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 1 -- 172.21.15.45:0/2908858111 wait complete. 2023-10-17T01:22:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.083+0000 7f1bff5beb80 20 client.26285 trim_cache size 0 max 16384 2023-10-17T01:22:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:22:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:22:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:22:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:22:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1bd00a0b20 msgr2=0x7f1bd00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1bd00a0b20 0x7f1bd00a2fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009fd0 tx=0x7f1bd4005dc0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 msgr2=0x7f1be419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 0x7f1be419e5e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00ef30 tx=0x7f1bdc00c3a0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 shutdown_connections 2023-10-17T01:22:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1bd00a0b20 0x7f1bd00a2fe0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1be410d350 0x7f1be419e5e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1be40737b0 0x7f1be419d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 --2- 172.21.15.45:0/1277885293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be4074d90 0x7f1be419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 >> 172.21.15.45:0/1277885293 conn(0x7f1be40fc2d0 msgr2=0x7f1be40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 shutdown_connections 2023-10-17T01:22:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:25.084+0000 7f1bebbff700 1 -- 172.21.15.45:0/1277885293 wait complete. 2023-10-17T01:22:25.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 435.0 s, 1802.4 tps, lat 56.431 ms stddev 65.643 2023-10-17T01:22:26.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:25 smithi045 ceph-mon[98474]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.93k op/s 2023-10-17T01:22:26.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:25 smithi060 ceph-mon[134136]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.93k op/s 2023-10-17T01:22:26.163 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:25 smithi181 ceph-mon[134520]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.93k op/s 2023-10-17T01:22:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:26 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:22:27.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:26 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:22:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:26 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:22:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:22:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:27 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:22:28.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:27 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:22:28.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:28.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:27 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi060 to 197.2M 2023-10-17T01:22:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-10-17T01:22:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: Unable to set osd_memory_target on smithi060 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-10-17T01:22:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-10-17T01:22:29.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: Adjusting osd_memory_target on smithi045 to 223.0M 2023-10-17T01:22:29.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: Unable to set osd_memory_target on smithi045 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-10-17T01:22:29.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:22:29.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:22:29.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:22:29.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:28 smithi045 ceph-mon[98474]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:22:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi060 to 197.2M 2023-10-17T01:22:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-10-17T01:22:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: Unable to set osd_memory_target on smithi060 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-10-17T01:22:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-10-17T01:22:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: Adjusting osd_memory_target on smithi045 to 223.0M 2023-10-17T01:22:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: Unable to set osd_memory_target on smithi045 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-10-17T01:22:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:22:29.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:22:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:22:29.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:28 smithi060 ceph-mon[134136]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:22:29.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi060 to 197.2M 2023-10-17T01:22:29.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-10-17T01:22:29.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: Unable to set osd_memory_target on smithi060 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-10-17T01:22:29.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: Unable to set osd_memory_target on smithi181 to 205737369: error parsing value: Value '205737369' is below minimum 939524096 2023-10-17T01:22:29.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-17T01:22:29.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: Adjusting osd_memory_target on smithi045 to 223.0M 2023-10-17T01:22:29.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: Unable to set osd_memory_target on smithi045 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-10-17T01:22:29.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:22:29.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:22:29.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:22:29.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:28 smithi181 ceph-mon[134520]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:22:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 440.0 s, 1826.4 tps, lat 54.506 ms stddev 72.021 2023-10-17T01:22:31.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:30 smithi045 ceph-mon[98474]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.62k op/s 2023-10-17T01:22:31.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:30 smithi060 ceph-mon[134136]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.62k op/s 2023-10-17T01:22:31.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:30 smithi181 ceph-mon[134520]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.62k op/s 2023-10-17T01:22:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:32 smithi045 ceph-mon[98474]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 840 op/s 2023-10-17T01:22:33.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:32 smithi060 ceph-mon[134136]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 840 op/s 2023-10-17T01:22:33.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:32 smithi181 ceph-mon[134520]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 840 op/s 2023-10-17T01:22:33.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:22:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 -- 172.21.15.45:0/2429689628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 msgr2=0x7f5e1c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 --2- 172.21.15.45:0/2429689628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c101290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009990 tx=0x7f5e08009ca0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 -- 172.21.15.45:0/2429689628 shutdown_connections 2023-10-17T01:22:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 --2- 172.21.15.45:0/2429689628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 --2- 172.21.15.45:0/2429689628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 0x7f5e1c100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 --2- 172.21.15.45:0/2429689628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c101290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 -- 172.21.15.45:0/2429689628 >> 172.21.15.45:0/2429689628 conn(0x7f5e1c0fb1b0 msgr2=0x7f5e1c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:33.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 -- 172.21.15.45:0/2429689628 shutdown_connections 2023-10-17T01:22:33.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.697+0000 7f5e227ff700 1 -- 172.21.15.45:0/2429689628 wait complete. 2023-10-17T01:22:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 Processor -- start 2023-10-17T01:22:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 -- start start 2023-10-17T01:22:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 0x7f5e1c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a0800 con 0x7f5e1c0ffc40 2023-10-17T01:22:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a0970 con 0x7f5e1c1017d0 2023-10-17T01:22:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.699+0000 7f5e227ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a0b30 con 0x7f5e1c100e30 2023-10-17T01:22:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c1a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38982/0 (socket says 172.21.15.45:38982) 2023-10-17T01:22:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 -- 172.21.15.45:0/4100906016 learned_addr learned my addr 172.21.15.45:0/4100906016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:33718/0 (socket says 172.21.15.45:33718) 2023-10-17T01:22:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 0x7f5e1c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 -- 172.21.15.45:0/4100906016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 msgr2=0x7f5e1c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 -- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 msgr2=0x7f5e1c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 0x7f5e1c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.700+0000 7f5e20d9c700 1 -- 172.21.15.45:0/4100906016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e080095f0 con 0x7f5e1c1017d0 2023-10-17T01:22:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.701+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 0x7f5e1c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:22:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.701+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c1a0230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e14007ee0 tx=0x7f5e1400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.701+0000 7f5e197fa700 1 -- 172.21.15.45:0/4100906016 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14017070 con 0x7f5e1c1017d0 2023-10-17T01:22:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.701+0000 7f5e197fa700 1 -- 172.21.15.45:0/4100906016 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e1400f040 con 0x7f5e1c1017d0 2023-10-17T01:22:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.701+0000 7f5e197fa700 1 -- 172.21.15.45:0/4100906016 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14005740 con 0x7f5e1c1017d0 2023-10-17T01:22:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.701+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c1020b0 con 0x7f5e1c1017d0 2023-10-17T01:22:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.702+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c1024b0 con 0x7f5e1c1017d0 2023-10-17T01:22:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.707+0000 7f5e197fa700 1 -- 172.21.15.45:0/4100906016 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f5e1400dc60 con 0x7f5e1c1017d0 2023-10-17T01:22:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 Processor -- start 2023-10-17T01:22:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 -- start start 2023-10-17T01:22:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e197fa700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5e0c0a0ba0 0x7f5e0c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84abcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84abdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84abe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84a14a10 con 0x555d84abd500 2023-10-17T01:22:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.710+0000 7f5e197fa700 1 -- 172.21.15.45:0/4100906016 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5e140cb8c0 con 0x7f5e1c1017d0 2023-10-17T01:22:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.709+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84a123d0 con 0x555d84abe2b0 2023-10-17T01:22:33.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.710+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84a0dcc0 con 0x555d84abcba0 2023-10-17T01:22:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.710+0000 7f5e1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84abcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.710+0000 7f5e1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84abcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:33726/0 (socket says 172.21.15.45:33726) 2023-10-17T01:22:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.710+0000 7f5e1bfff700 1 -- 172.21.15.45:0/93999359 learned_addr learned my addr 172.21.15.45:0/93999359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.710+0000 7f5e1bfff700 1 -- 172.21.15.45:0/93999359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 msgr2=0x555d84abe730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:22:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.711+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84abe730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.711+0000 7f5e1bfff700 1 -- 172.21.15.45:0/93999359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 msgr2=0x555d84abdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.711+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84abdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.711+0000 7f5e1bfff700 1 -- 172.21.15.45:0/93999359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8483bde0 con 0x555d84abcba0 2023-10-17T01:22:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.711+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84abcfc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009cb0 tx=0x7f5e1000e2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e523b0b57fd5b2c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.712+0000 7f5e067fc700 1 -- 172.21.15.45:0/93999359 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000ec20 con 0x555d84abcba0 2023-10-17T01:22:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.712+0000 7f5e067fc700 1 -- 172.21.15.45:0/93999359 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e1000ed80 con 0x555d84abcba0 2023-10-17T01:22:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.712+0000 7f5e067fc700 1 -- 172.21.15.45:0/93999359 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10010730 con 0x555d84abcba0 2023-10-17T01:22:33.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.713+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5e0c0a0ba0 0x7f5e0c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.713+0000 7f5e3611cb80 1 -- 172.21.15.45:0/93999359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 msgr2=0x555d84abcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.713+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84abcfc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009cb0 tx=0x7f5e1000e2f0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.713+0000 7f5e3611cb80 1 -- 172.21.15.45:0/93999359 shutdown_connections 2023-10-17T01:22:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.713+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84abe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.714+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84abcfc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.714+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/93999359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84abdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.714+0000 7f5e3611cb80 1 -- 172.21.15.45:0/93999359 >> 172.21.15.45:0/93999359 conn(0x555d84a1c0d0 msgr2=0x555d84ab7950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.714+0000 7f5e3611cb80 1 -- 172.21.15.45:0/93999359 shutdown_connections 2023-10-17T01:22:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.714+0000 7f5e3611cb80 1 -- 172.21.15.45:0/93999359 wait complete. 2023-10-17T01:22:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.714+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5e0c0a0ba0 0x7f5e0c0a3060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5e080095c0 tx=0x7f5e08009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.715+0000 7f5e3611cb80 1 -- start start 2023-10-17T01:22:33.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.715+0000 7f5e3611cb80 1 Processor -- start 2023-10-17T01:22:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84ac6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84ac6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84acdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d849aa510 con 0x555d84abd500 2023-10-17T01:22:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d849aa1b0 con 0x555d84abe2b0 2023-10-17T01:22:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84891920 con 0x555d84abcba0 2023-10-17T01:22:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e20d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84acdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e20d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84acdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38988/0 (socket says 172.21.15.45:38988) 2023-10-17T01:22:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.716+0000 7f5e20d9c700 1 -- 172.21.15.45:0/879801861 learned_addr learned my addr 172.21.15.45:0/879801861 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84ac6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e3611cb80 10 client.0 _ll_register_callbacks cb 0x555d848f7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e20d9c700 1 -- 172.21.15.45:0/879801861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 msgr2=0x555d84ac6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84ac6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e20d9c700 1 -- 172.21.15.45:0/879801861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 msgr2=0x555d84ac6e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:22:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84ac6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84ac6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e20d9c700 1 -- 172.21.15.45:0/879801861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e100098e0 con 0x555d84abe2b0 2023-10-17T01:22:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84ac6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:22:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.717+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d84a38d10 con 0x555d84abe2b0 2023-10-17T01:22:33.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84acdec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5e140a0a80 tx=0x7f5e140989d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47b83b3c01ca8e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5dfb7fe700 10 client.26294 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:22:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/879801861 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14005aa0 con 0x555d84abe2b0 2023-10-17T01:22:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/879801861 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5e1409cd70 con 0x555d84abe2b0 2023-10-17T01:22:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/879801861 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e140a0fb0 con 0x555d84abe2b0 2023-10-17T01:22:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5e3611cb80 10 client.26294 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x555d84824c60 con 0x555d84abe2b0 2023-10-17T01:22:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.718+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/879801861 <== mon.2 v2:172.21.15.60:3300/0 4 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5e140b1270 con 0x555d84abe2b0 2023-10-17T01:22:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e3611cb80 10 client.26294 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e3611cb80 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e3611cb80 20 client.26294 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84ad2510 0x555d84ad4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e3611cb80 4 client.26294 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:22:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d84a8bd30 con 0x555d84ad2510 2023-10-17T01:22:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.719+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84ad2510 0x555d84ad4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.720+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84ad2510 0x555d84ad4930 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14ca5a498ee65d1 server_cookie=55486f36b7daa8fa in_seq=0 out_seq=0 2023-10-17T01:22:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.720+0000 7f5dfb7fe700 10 client.26294 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.720+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/879801861 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x555d84a8bd30 con 0x555d84ad2510 2023-10-17T01:22:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.720+0000 7f5dfb7fe700 10 client.26294 handle_command_reply: tid=0 2023-10-17T01:22:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.720+0000 7f5e3611cb80 1 client.26294 shutdown 2023-10-17T01:22:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 msgr2=0x555d84acdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84acdec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5e140a0a80 tx=0x7f5e140989d0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 shutdown_connections 2023-10-17T01:22:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84ad2510 0x555d84ad4930 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84acdec0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abcba0 0x555d84ac6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5dfb7fe700 0 client.26294 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/879801861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abd500 0x555d84ac6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 >> 172.21.15.45:0/879801861 conn(0x555d84a1c0d0 msgr2=0x555d84ab7020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 shutdown_connections 2023-10-17T01:22:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.721+0000 7f5e3611cb80 1 -- 172.21.15.45:0/879801861 wait complete. 2023-10-17T01:22:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.722+0000 7f5e3611cb80 20 client.26294 trim_cache size 0 max 16384 2023-10-17T01:22:33.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.753+0000 7f5e3611cb80 1 Processor -- start 2023-10-17T01:22:33.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.753+0000 7f5e3611cb80 1 -- start start 2023-10-17T01:22:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abcba0 0x555d84ac5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84ac5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84ab7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84ab1e60 con 0x555d84abcba0 2023-10-17T01:22:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84ab8c80 con 0x555d84abe2b0 2023-10-17T01:22:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84ad22f0 con 0x555d84abd500 2023-10-17T01:22:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abcba0 0x555d84ac5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e20d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84ab7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e20d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84ab7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38990/0 (socket says 172.21.15.45:38990) 2023-10-17T01:22:33.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.754+0000 7f5e20d9c700 1 -- 172.21.15.45:0/1821596442 learned_addr learned my addr 172.21.15.45:0/1821596442 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:33.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84ac5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e20d9c700 1 -- 172.21.15.45:0/1821596442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 msgr2=0x555d84ac5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84ac5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e20d9c700 1 -- 172.21.15.45:0/1821596442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abcba0 msgr2=0x555d84ac5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abcba0 0x555d84ac5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e20d9c700 1 -- 172.21.15.45:0/1821596442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d84862250 con 0x555d84abe2b0 2023-10-17T01:22:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.755+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abcba0 0x555d84ac5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.756+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84ab7020 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009510 tx=0x7f5e1401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b307b3a5312a2186 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.756+0000 7f5e06ffd700 1 -- 172.21.15.45:0/1821596442 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1409cdf0 con 0x555d84abe2b0 2023-10-17T01:22:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.756+0000 7f5e06ffd700 1 -- 172.21.15.45:0/1821596442 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d84a38d10 con 0x555d84abe2b0 2023-10-17T01:22:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.756+0000 7f5e06ffd700 1 -- 172.21.15.45:0/1821596442 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14005b00 con 0x555d84abe2b0 2023-10-17T01:22:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 -- 172.21.15.45:0/1821596442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 msgr2=0x555d84ab7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84ab7020 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009510 tx=0x7f5e1401d000 comp rx=0 tx=0).stop 2023-10-17T01:22:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 -- 172.21.15.45:0/1821596442 shutdown_connections 2023-10-17T01:22:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abe2b0 0x555d84ab7020 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abcba0 0x555d84ac5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/1821596442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84ac5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 -- 172.21.15.45:0/1821596442 >> 172.21.15.45:0/1821596442 conn(0x555d84a1c0d0 msgr2=0x555d84ab78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 -- 172.21.15.45:0/1821596442 shutdown_connections 2023-10-17T01:22:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.757+0000 7f5e3611cb80 1 -- 172.21.15.45:0/1821596442 wait complete. 2023-10-17T01:22:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.758+0000 7f5e3611cb80 1 -- start start 2023-10-17T01:22:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 Processor -- start 2023-10-17T01:22:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abcba0 0x555d84acba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84aca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abe2b0 0x555d84aca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84acaac0 con 0x555d84abe2b0 2023-10-17T01:22:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84acac30 con 0x555d84abcba0 2023-10-17T01:22:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d84acadf0 con 0x555d84abd500 2023-10-17T01:22:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.759+0000 7f5e3611cb80 10 client.0 _ll_register_callbacks cb 0x7f5e14098de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.760+0000 7f5e1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84aca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.760+0000 7f5e1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84aca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:33746/0 (socket says 172.21.15.45:33746) 2023-10-17T01:22:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.760+0000 7f5e1bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abcba0 0x555d84acba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.760+0000 7f5e1b7fe700 1 -- 172.21.15.45:0/3530177440 learned_addr learned my addr 172.21.15.45:0/3530177440 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.760+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abe2b0 0x555d84aca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1b7fe700 1 -- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abcba0 msgr2=0x555d84acba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abcba0 0x555d84acba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1b7fe700 1 -- 172.21.15.45:0/3530177440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abe2b0 msgr2=0x555d84aca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abe2b0 0x555d84aca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1b7fe700 1 -- 172.21.15.45:0/3530177440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8483bde0 con 0x555d84abd500 2023-10-17T01:22:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abcba0 0x555d84acba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e20d9c700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abe2b0 0x555d84aca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:22:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.761+0000 7f5e1b7fe700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84aca040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098e0 tx=0x7f5e08024070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111ad6dbadb4c125 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:33.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5dfb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:22:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/3530177440 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08027040 con 0x555d84abd500 2023-10-17T01:22:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/3530177440 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5e08005050 con 0x555d84abd500 2023-10-17T01:22:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e1000ed80 con 0x555d84abd500 2023-10-17T01:22:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/3530177440 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e08005210 con 0x555d84abd500 2023-10-17T01:22:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5e3611cb80 10 client.26263 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.762+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555d84824c60 con 0x555d84abd500 2023-10-17T01:22:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.763+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/3530177440 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e1000ed80 con 0x555d84abd500 2023-10-17T01:22:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.763+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/3530177440 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5e08044b40 con 0x555d84abd500 2023-10-17T01:22:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.763+0000 7f5e3611cb80 10 client.26263 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.763+0000 7f5e3611cb80 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:33.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.763+0000 7f5e3611cb80 20 client.26263 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.764+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84b254a0 0x555d84b278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.765+0000 7f5e3611cb80 4 client.26263 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:22:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.765+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d84a06d40 con 0x555d84b254a0 2023-10-17T01:22:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.766+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84b254a0 0x555d84b278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.766+0000 7f5e1bfff700 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84b254a0 0x555d84b278c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dacb808761bf93c server_cookie=1750477451c59ebe in_seq=0 out_seq=0 2023-10-17T01:22:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.766+0000 7f5dfb7fe700 10 client.26263 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.766+0000 7f5dfb7fe700 1 -- 172.21.15.45:0/3530177440 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555d84a06d40 con 0x555d84b254a0 2023-10-17T01:22:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.766+0000 7f5dfb7fe700 10 client.26263 handle_command_reply: tid=0 2023-10-17T01:22:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.766+0000 7f5e3611cb80 1 client.26263 shutdown 2023-10-17T01:22:33.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 msgr2=0x555d84aca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84aca040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098e0 tx=0x7f5e08024070 comp rx=0 tx=0).stop 2023-10-17T01:22:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 shutdown_connections 2023-10-17T01:22:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d84abe2b0 0x555d84aca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555d84b254a0 0x555d84b278c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d84abcba0 0x555d84acba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 --2- 172.21.15.45:0/3530177440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d84abd500 0x555d84aca040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5dfb7fe700 0 client.26263 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 >> 172.21.15.45:0/3530177440 conn(0x555d84a1c0d0 msgr2=0x555d84ac46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 shutdown_connections 2023-10-17T01:22:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 1 -- 172.21.15.45:0/3530177440 wait complete. 2023-10-17T01:22:33.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.767+0000 7f5e3611cb80 20 client.26263 trim_cache size 0 max 16384 2023-10-17T01:22:33.776 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:22:33.776 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:22:33.776 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:22:33.776 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:22:33.777 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:22:33.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:22:33.777 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:22:33.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:22:33.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:22:33.778 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:22:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.768+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5e0c0a0ba0 msgr2=0x7f5e0c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.768+0000 7f5e227ff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5e0c0a0ba0 0x7f5e0c0a3060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5e080095c0 tx=0x7f5e08009530 comp rx=0 tx=0).stop 2023-10-17T01:22:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.768+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 msgr2=0x7f5e1c1a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.768+0000 7f5e227ff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c1a0230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e14007ee0 tx=0x7f5e1400d3b0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.769+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 shutdown_connections 2023-10-17T01:22:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.769+0000 7f5e227ff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5e0c0a0ba0 0x7f5e0c0a3060 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.769+0000 7f5e227ff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c1017d0 0x7f5e1c1a0230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.769+0000 7f5e227ff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e1c0ffc40 0x7f5e1c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.769+0000 7f5e227ff700 1 --2- 172.21.15.45:0/4100906016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c100e30 0x7f5e1c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.769+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 >> 172.21.15.45:0/4100906016 conn(0x7f5e1c0fb1b0 msgr2=0x7f5e1c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.770+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 shutdown_connections 2023-10-17T01:22:33.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:22:33.770+0000 7f5e227ff700 1 -- 172.21.15.45:0/4100906016 wait complete. 2023-10-17T01:22:33.800 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:22:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 445.0 s, 1769.4 tps, lat 56.340 ms stddev 61.571 2023-10-17T01:22:36.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:35 smithi045 ceph-mon[98474]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-10-17T01:22:36.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:35 smithi060 ceph-mon[134136]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-10-17T01:22:36.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:35 smithi181 ceph-mon[134520]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 698 op/s 2023-10-17T01:22:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:36 smithi045 ceph-mon[98474]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 582 op/s 2023-10-17T01:22:37.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:36 smithi060 ceph-mon[134136]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 582 op/s 2023-10-17T01:22:37.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:36 smithi181 ceph-mon[134520]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 582 op/s 2023-10-17T01:22:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 -- 172.21.15.45:0/470458063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c10a6c0 msgr2=0x7fa78c10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 --2- 172.21.15.45:0/470458063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c10a6c0 0x7fa78c10cb50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa774009990 tx=0x7fa774009ca0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 -- 172.21.15.45:0/470458063 shutdown_connections 2023-10-17T01:22:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 --2- 172.21.15.45:0/470458063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10d090 0x7fa78c10d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 --2- 172.21.15.45:0/470458063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa78c107d50 0x7fa78c10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 --2- 172.21.15.45:0/470458063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c10a6c0 0x7fa78c10cb50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 -- 172.21.15.45:0/470458063 >> 172.21.15.45:0/470458063 conn(0x7fa78c06db80 msgr2=0x7fa78c06ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.332+0000 7fa7929ff700 1 -- 172.21.15.45:0/470458063 shutdown_connections 2023-10-17T01:22:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.333+0000 7fa7929ff700 1 -- 172.21.15.45:0/470458063 wait complete. 2023-10-17T01:22:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.334+0000 7fa7929ff700 1 Processor -- start 2023-10-17T01:22:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.334+0000 7fa7929ff700 1 -- start start 2023-10-17T01:22:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.335+0000 7fa7929ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c107d50 0x7fa78c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.335+0000 7fa7929ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10a6c0 0x7fa78c1a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.335+0000 7fa7929ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa78c10d090 0x7fa78c1a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.335+0000 7fa7929ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a7340 con 0x7fa78c10d090 2023-10-17T01:22:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.335+0000 7fa7929ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a74b0 con 0x7fa78c107d50 2023-10-17T01:22:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.335+0000 7fa7929ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1a7670 con 0x7fa78c10a6c0 2023-10-17T01:22:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10a6c0 0x7fa78c1a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c107d50 0x7fa78c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10a6c0 0x7fa78c1a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51782/0 (socket says 172.21.15.45:51782) 2023-10-17T01:22:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78b7fe700 1 -- 172.21.15.45:0/1013436925 learned_addr learned my addr 172.21.15.45:0/1013436925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa790f9c700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa78c10d090 0x7fa78c1a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78bfff700 1 -- 172.21.15.45:0/1013436925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10a6c0 msgr2=0x7fa78c1a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78bfff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10a6c0 0x7fa78c1a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78bfff700 1 -- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa78c10d090 msgr2=0x7fa78c1a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78bfff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa78c10d090 0x7fa78c1a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.336+0000 7fa78bfff700 1 -- 172.21.15.45:0/1013436925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740095f0 con 0x7fa78c107d50 2023-10-17T01:22:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.337+0000 7fa78bfff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c107d50 0x7fa78c1a5920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9f0 tx=0x7fa77c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.337+0000 7fa7897fa700 1 -- 172.21.15.45:0/1013436925 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00eee0 con 0x7fa78c107d50 2023-10-17T01:22:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.337+0000 7fa7897fa700 1 -- 172.21.15.45:0/1013436925 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c00cbd0 con 0x7fa78c107d50 2023-10-17T01:22:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.337+0000 7fa7897fa700 1 -- 172.21.15.45:0/1013436925 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c018680 con 0x7fa78c107d50 2023-10-17T01:22:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.337+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c1a7950 con 0x7fa78c107d50 2023-10-17T01:22:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.337+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c1ae110 con 0x7fa78c107d50 2023-10-17T01:22:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.344+0000 7fa7897fa700 1 -- 172.21.15.45:0/1013436925 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa77c00cd40 con 0x7fa78c107d50 2023-10-17T01:22:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.345+0000 7fa7a639bb80 1 Processor -- start 2023-10-17T01:22:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 -- start start 2023-10-17T01:22:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce567cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce567d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce567e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce55d49b0 con 0x55cce567d190 2023-10-17T01:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce55d2370 con 0x55cce567c830 2023-10-17T01:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce55cdc60 con 0x55cce567dd90 2023-10-17T01:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce567e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.346+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce567e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51798/0 (socket says 172.21.15.45:51798) 2023-10-17T01:22:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78b7fe700 1 -- 172.21.15.45:0/33939519 learned_addr learned my addr 172.21.15.45:0/33939519 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78bfff700 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce567d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa790f9c700 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce567cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78b7fe700 1 -- 172.21.15.45:0/33939519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 msgr2=0x55cce567cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce567cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78b7fe700 1 -- 172.21.15.45:0/33939519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 msgr2=0x55cce567d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce567d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78b7fe700 1 -- 172.21.15.45:0/33939519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce53fbd80 con 0x55cce567dd90 2023-10-17T01:22:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.347+0000 7fa78bfff700 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce567d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce567e210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa774005f00 tx=0x7fa774005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2699a2fec6d203f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa7897fa700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa7780a0ba0 0x7fa7780a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa7897fa700 1 -- 172.21.15.45:0/1013436925 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa77c014070 con 0x7fa78c107d50 2023-10-17T01:22:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa790f9c700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa7780a0ba0 0x7fa7780a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa7827fc700 1 -- 172.21.15.45:0/33939519 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77401d070 con 0x55cce567dd90 2023-10-17T01:22:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa7827fc700 1 -- 172.21.15.45:0/33939519 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77401e040 con 0x55cce567dd90 2023-10-17T01:22:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.348+0000 7fa7827fc700 1 -- 172.21.15.45:0/33939519 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774004a70 con 0x55cce567dd90 2023-10-17T01:22:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.349+0000 7fa7a639bb80 1 -- 172.21.15.45:0/33939519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 msgr2=0x55cce567e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.349+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce567e210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa774005f00 tx=0x7fa774005c00 comp rx=0 tx=0).stop 2023-10-17T01:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.349+0000 7fa790f9c700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa7780a0ba0 0x7fa7780a3060 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa784007550 tx=0x7fa784009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.350+0000 7fa7a639bb80 1 -- 172.21.15.45:0/33939519 shutdown_connections 2023-10-17T01:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.350+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce567e210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.350+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce567cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.350+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/33939519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce567d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.350+0000 7fa7a639bb80 1 -- 172.21.15.45:0/33939519 >> 172.21.15.45:0/33939519 conn(0x55cce55dc070 msgr2=0x55cce56772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.350+0000 7fa7a639bb80 1 -- 172.21.15.45:0/33939519 shutdown_connections 2023-10-17T01:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.351+0000 7fa7a639bb80 1 -- 172.21.15.45:0/33939519 wait complete. 2023-10-17T01:22:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.351+0000 7fa7a639bb80 1 -- start start 2023-10-17T01:22:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.352+0000 7fa7a639bb80 1 Processor -- start 2023-10-17T01:22:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5686310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 0x55cce5686850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567dd90 0x55cce568d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 0x55cce5686850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce556a4b0 con 0x55cce567dd90 2023-10-17T01:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce556a150 con 0x55cce567c830 2023-10-17T01:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.353+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce54518c0 con 0x55cce567d190 2023-10-17T01:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5686310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 0x55cce5686850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51800/0 (socket says 172.21.15.45:51800) 2023-10-17T01:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78b7fe700 1 -- 172.21.15.45:0/2934043064 learned_addr learned my addr 172.21.15.45:0/2934043064 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa790f9c700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567dd90 0x55cce568d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa7a639bb80 10 client.0 _ll_register_callbacks cb 0x55cce54b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 -- 172.21.15.45:0/2934043064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 msgr2=0x55cce5686850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 0x55cce5686850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 -- 172.21.15.45:0/2934043064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567dd90 msgr2=0x55cce568d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567dd90 0x55cce568d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 -- 172.21.15.45:0/2934043064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c1ae110 con 0x55cce567c830 2023-10-17T01:22:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78bfff700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5686310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55cce567d590 tx=0x7fa77c097f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a69e252916c6290 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 0x55cce5686850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa790f9c700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567dd90 0x55cce568d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:22:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.354+0000 7fa76b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:22:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.355+0000 7fa76b7fe700 1 -- 172.21.15.45:0/2934043064 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c099490 con 0x55cce567c830 2023-10-17T01:22:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.355+0000 7fa76b7fe700 1 -- 172.21.15.45:0/2934043064 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa77c09a050 con 0x55cce567c830 2023-10-17T01:22:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.356+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cce55f8cb0 con 0x55cce567c830 2023-10-17T01:22:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.356+0000 7fa76b7fe700 1 -- 172.21.15.45:0/2934043064 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa77c0992b0 con 0x55cce567c830 2023-10-17T01:22:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.356+0000 7fa7a639bb80 10 client.26312 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.356+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fa78c10e090 con 0x55cce567c830 2023-10-17T01:22:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.356+0000 7fa76b7fe700 1 -- 172.21.15.45:0/2934043064 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cce55f8cb0 con 0x55cce567c830 2023-10-17T01:22:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa76b7fe700 1 -- 172.21.15.45:0/2934043064 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa77c098ed0 con 0x55cce567c830 2023-10-17T01:22:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa7a639bb80 10 client.26312 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa7a639bb80 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa7a639bb80 20 client.26312 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56920f0 0x55cce5694510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa7a639bb80 4 client.26312 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:22:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.357+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cce564bcd0 con 0x55cce56920f0 2023-10-17T01:22:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.358+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56920f0 0x55cce5694510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.358+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56920f0 0x55cce5694510 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4945976f4a9ce4d7 server_cookie=f94fd7feec9fbe94 in_seq=0 out_seq=0 2023-10-17T01:22:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.358+0000 7fa76b7fe700 10 client.26312 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.359+0000 7fa76b7fe700 1 -- 172.21.15.45:0/2934043064 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55cce564bcd0 con 0x55cce56920f0 2023-10-17T01:22:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.359+0000 7fa76b7fe700 10 client.26312 handle_command_reply: tid=0 2023-10-17T01:22:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.359+0000 7fa7a639bb80 1 client.26312 shutdown 2023-10-17T01:22:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.360+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 msgr2=0x55cce5686310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.360+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5686310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55cce567d590 tx=0x7fa77c097f00 comp rx=0 tx=0).stop 2023-10-17T01:22:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 shutdown_connections 2023-10-17T01:22:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567dd90 0x55cce568d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56920f0 0x55cce5694510 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5686310 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/2934043064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567d190 0x55cce5686850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 >> 172.21.15.45:0/2934043064 conn(0x55cce55dc070 msgr2=0x55cce5676a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa76b7fe700 0 client.26312 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.361+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 shutdown_connections 2023-10-17T01:22:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.362+0000 7fa7a639bb80 1 -- 172.21.15.45:0/2934043064 wait complete. 2023-10-17T01:22:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.362+0000 7fa7a639bb80 20 client.26312 trim_cache size 0 max 16384 2023-10-17T01:22:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.394+0000 7fa7a639bb80 1 Processor -- start 2023-10-17T01:22:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.394+0000 7fa7a639bb80 1 -- start start 2023-10-17T01:22:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5685030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce5676a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce5684e20 con 0x55cce567d190 2023-10-17T01:22:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce56857c0 con 0x55cce567c830 2023-10-17T01:22:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce567dbf0 con 0x55cce567dd90 2023-10-17T01:22:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa78bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5685030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa78bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5685030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33878/0 (socket says 172.21.15.45:33878) 2023-10-17T01:22:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa78bfff700 1 -- 172.21.15.45:0/3474923480 learned_addr learned my addr 172.21.15.45:0/3474923480 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa78bfff700 1 -- 172.21.15.45:0/3474923480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 msgr2=0x55cce568e060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:22:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.395+0000 7fa78bfff700 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.396+0000 7fa78bfff700 1 -- 172.21.15.45:0/3474923480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 msgr2=0x55cce5676a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.396+0000 7fa78bfff700 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce5676a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.396+0000 7fa78bfff700 1 -- 172.21.15.45:0/3474923480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce53e4c00 con 0x55cce567c830 2023-10-17T01:22:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.396+0000 7fa78bfff700 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5685030 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa77c095e50 tx=0x7fa77c0a18c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d75ca860aaf5bc70 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.397+0000 7fa782ffd700 1 -- 172.21.15.45:0/3474923480 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0992b0 con 0x55cce567c830 2023-10-17T01:22:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.397+0000 7fa782ffd700 1 -- 172.21.15.45:0/3474923480 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c099490 con 0x55cce567c830 2023-10-17T01:22:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.397+0000 7fa782ffd700 1 -- 172.21.15.45:0/3474923480 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c099950 con 0x55cce567c830 2023-10-17T01:22:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3474923480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 msgr2=0x55cce5685030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5685030 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa77c095e50 tx=0x7fa77c0a18c0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3474923480 shutdown_connections 2023-10-17T01:22:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567c830 0x55cce5685030 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3474923480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567d190 0x55cce5676a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3474923480 >> 172.21.15.45:0/3474923480 conn(0x55cce55dc070 msgr2=0x55cce5677290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3474923480 shutdown_connections 2023-10-17T01:22:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.398+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3474923480 wait complete. 2023-10-17T01:22:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.399+0000 7fa7a639bb80 1 -- start start 2023-10-17T01:22:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.399+0000 7fa7a639bb80 1 Processor -- start 2023-10-17T01:22:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567c830 0x55cce56892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 0x55cce5689800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce568ad50 con 0x55cce567c830 2023-10-17T01:22:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce568aec0 con 0x55cce567d190 2023-10-17T01:22:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce568b080 con 0x55cce567dd90 2023-10-17T01:22:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.401+0000 7fa7a639bb80 10 client.0 _ll_register_callbacks cb 0x7fa77c0cf860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.402+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 0x55cce5689800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.402+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 0x55cce5689800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33882/0 (socket says 172.21.15.45:33882) 2023-10-17T01:22:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.402+0000 7fa78b7fe700 1 -- 172.21.15.45:0/3095963040 learned_addr learned my addr 172.21.15.45:0/3095963040 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.402+0000 7fa790f9c700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.402+0000 7fa78bfff700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567c830 0x55cce56892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa78b7fe700 1 -- 172.21.15.45:0/3095963040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 msgr2=0x55cce568a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa78b7fe700 1 -- 172.21.15.45:0/3095963040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567c830 msgr2=0x55cce56892c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567c830 0x55cce56892c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa78b7fe700 1 -- 172.21.15.45:0/3095963040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c10e090 con 0x55cce567d190 2023-10-17T01:22:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa78b7fe700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 0x55cce5689800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa77400c010 tx=0x7fa774005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba05afe32ea90e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.403+0000 7fa76b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:22:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.404+0000 7fa76b7fe700 1 -- 172.21.15.45:0/3095963040 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77401d040 con 0x55cce567d190 2023-10-17T01:22:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.404+0000 7fa76b7fe700 1 -- 172.21.15.45:0/3095963040 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa774033920 con 0x55cce567d190 2023-10-17T01:22:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.404+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa77401e040 con 0x55cce567d190 2023-10-17T01:22:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.404+0000 7fa76b7fe700 1 -- 172.21.15.45:0/3095963040 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa774033ae0 con 0x55cce567d190 2023-10-17T01:22:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.404+0000 7fa7a639bb80 10 client.26318 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.404+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cce54221f0 con 0x55cce567d190 2023-10-17T01:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa76b7fe700 1 -- 172.21.15.45:0/3095963040 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77401e040 con 0x55cce567d190 2023-10-17T01:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa76b7fe700 1 -- 172.21.15.45:0/3095963040 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa77402c400 con 0x55cce567d190 2023-10-17T01:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa7a639bb80 10 client.26318 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa7a639bb80 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa7a639bb80 20 client.26318 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56e42b0 0x55cce56e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa7a639bb80 4 client.26318 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:22:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.405+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cce55c6ce0 con 0x55cce56e42b0 2023-10-17T01:22:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.407+0000 7fa78bfff700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56e42b0 0x55cce56e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.408+0000 7fa78bfff700 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56e42b0 0x55cce56e66d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b65be303bf2245 server_cookie=116a44a569d8dbdb in_seq=0 out_seq=0 2023-10-17T01:22:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.408+0000 7fa76b7fe700 10 client.26318 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.421+0000 7fa76b7fe700 1 -- 172.21.15.45:0/3095963040 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cce55c6ce0 con 0x55cce56e42b0 2023-10-17T01:22:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.421+0000 7fa76b7fe700 10 client.26318 handle_command_reply: tid=0 2023-10-17T01:22:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.421+0000 7fa7a639bb80 1 client.26318 shutdown 2023-10-17T01:22:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.421+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 msgr2=0x55cce5689800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.421+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 0x55cce5689800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa77400c010 tx=0x7fa774005bc0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 shutdown_connections 2023-10-17T01:22:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cce567dd90 0x55cce568a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cce567c830 0x55cce56892c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55cce56e42b0 0x55cce56e66d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa7a639bb80 1 --2- 172.21.15.45:0/3095963040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cce567d190 0x55cce5689800 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 >> 172.21.15.45:0/3095963040 conn(0x55cce55dc070 msgr2=0x55cce5684150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.422+0000 7fa76b7fe700 0 client.26318 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.423+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 shutdown_connections 2023-10-17T01:22:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.423+0000 7fa7a639bb80 1 -- 172.21.15.45:0/3095963040 wait complete. 2023-10-17T01:22:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.424+0000 7fa7a639bb80 20 client.26318 trim_cache size 0 max 16384 2023-10-17T01:22:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:22:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:22:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:22:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:22:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.425+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa7780a0ba0 msgr2=0x7fa7780a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.425+0000 7fa7929ff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa7780a0ba0 0x7fa7780a3060 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa784007550 tx=0x7fa784009040 comp rx=0 tx=0).stop 2023-10-17T01:22:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.425+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c107d50 msgr2=0x7fa78c1a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.425+0000 7fa7929ff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c107d50 0x7fa78c1a5920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9f0 tx=0x7fa77c00ed00 comp rx=0 tx=0).stop 2023-10-17T01:22:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 shutdown_connections 2023-10-17T01:22:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa7780a0ba0 0x7fa7780a3060 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa78c10d090 0x7fa78c1a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78c107d50 0x7fa78c1a5920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 --2- 172.21.15.45:0/1013436925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c10a6c0 0x7fa78c1a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 >> 172.21.15.45:0/1013436925 conn(0x7fa78c06db80 msgr2=0x7fa78c1095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 shutdown_connections 2023-10-17T01:22:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:38.426+0000 7fa7929ff700 1 -- 172.21.15.45:0/1013436925 wait complete. 2023-10-17T01:22:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:38 smithi045 ceph-mon[98474]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 593 op/s 2023-10-17T01:22:39.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:38 smithi060 ceph-mon[134136]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 593 op/s 2023-10-17T01:22:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:38 smithi181 ceph-mon[134520]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 593 op/s 2023-10-17T01:22:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 450.0 s, 2092.4 tps, lat 47.775 ms stddev 60.276 2023-10-17T01:22:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:40 smithi045 ceph-mon[98474]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 684 op/s 2023-10-17T01:22:41.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:40 smithi060 ceph-mon[134136]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 684 op/s 2023-10-17T01:22:41.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:40 smithi181 ceph-mon[134520]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 684 op/s 2023-10-17T01:22:43.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:42 smithi045 ceph-mon[98474]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 633 op/s 2023-10-17T01:22:43.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:42 smithi060 ceph-mon[134136]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 633 op/s 2023-10-17T01:22:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:42 smithi181 ceph-mon[134520]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 633 op/s 2023-10-17T01:22:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 455.0 s, 1775.9 tps, lat 55.935 ms stddev 62.962 2023-10-17T01:22:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:45 smithi045 ceph-mon[98474]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 695 op/s 2023-10-17T01:22:46.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:45 smithi060 ceph-mon[134136]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 695 op/s 2023-10-17T01:22:46.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:45 smithi181 ceph-mon[134520]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 695 op/s 2023-10-17T01:22:48.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:47 smithi045 ceph-mon[98474]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 634 op/s 2023-10-17T01:22:48.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:47 smithi060 ceph-mon[134136]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 634 op/s 2023-10-17T01:22:48.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:47 smithi181 ceph-mon[134520]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 634 op/s 2023-10-17T01:22:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:48 smithi045 ceph-mon[98474]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 671 op/s 2023-10-17T01:22:49.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:48 smithi060 ceph-mon[134136]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 671 op/s 2023-10-17T01:22:49.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:48 smithi181 ceph-mon[134520]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 671 op/s 2023-10-17T01:22:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 460.0 s, 2076.3 tps, lat 48.703 ms stddev 66.046 2023-10-17T01:22:51.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:50 smithi045 ceph-mon[98474]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 782 op/s 2023-10-17T01:22:51.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:50 smithi060 ceph-mon[134136]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 782 op/s 2023-10-17T01:22:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:50 smithi181 ceph-mon[134520]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 782 op/s 2023-10-17T01:22:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.659+0000 7fc8011ff700 1 -- 172.21.15.45:0/2780733984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 msgr2=0x7fc7fc100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.659+0000 7fc8011ff700 1 --2- 172.21.15.45:0/2780733984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 0x7fc7fc100400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00099c0 tx=0x7fc7f0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 -- 172.21.15.45:0/2780733984 shutdown_connections 2023-10-17T01:22:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 --2- 172.21.15.45:0/2780733984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 --2- 172.21.15.45:0/2780733984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7fc0ff620 0x7fc7fc0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 --2- 172.21.15.45:0/2780733984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 0x7fc7fc100400 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 -- 172.21.15.45:0/2780733984 >> 172.21.15.45:0/2780733984 conn(0x7fc7fc0fb1a0 msgr2=0x7fc7fc0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 -- 172.21.15.45:0/2780733984 shutdown_connections 2023-10-17T01:22:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.660+0000 7fc8011ff700 1 -- 172.21.15.45:0/2780733984 wait complete. 2023-10-17T01:22:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 Processor -- start 2023-10-17T01:22:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 -- start start 2023-10-17T01:22:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7fc0ff620 0x7fc7fc105310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 0x7fc7fc105850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc106720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc10baa0 con 0x7fc7fc0ff620 2023-10-17T01:22:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc106c60 con 0x7fc7fc0fff80 2023-10-17T01:22:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.662+0000 7fc8011ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc106e20 con 0x7fc7fc100940 2023-10-17T01:22:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 0x7fc7fc105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51460/0 (socket says 172.21.15.45:51460) 2023-10-17T01:22:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 -- 172.21.15.45:0/1352401700 learned_addr learned my addr 172.21.15.45:0/1352401700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7fc0ff620 0x7fc7fc105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 -- 172.21.15.45:0/1352401700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 msgr2=0x7fc7fc105850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 0x7fc7fc105850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 -- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7fc0ff620 msgr2=0x7fc7fc105310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7fc0ff620 0x7fc7fc105310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 -- 172.21.15.45:0/1352401700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00095f0 con 0x7fc7fc100940 2023-10-17T01:22:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc106720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400ef30 tx=0x7fc7f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.663+0000 7fc7e3fff700 1 -- 172.21.15.45:0/1352401700 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4018070 con 0x7fc7fc100940 2023-10-17T01:22:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.664+0000 7fc7e3fff700 1 -- 172.21.15.45:0/1352401700 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f400f040 con 0x7fc7fc100940 2023-10-17T01:22:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.664+0000 7fc7e3fff700 1 -- 172.21.15.45:0/1352401700 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400cdb0 con 0x7fc7fc100940 2023-10-17T01:22:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.664+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc107040 con 0x7fc7fc100940 2023-10-17T01:22:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.664+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc1074d0 con 0x7fc7fc100940 2023-10-17T01:22:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.669+0000 7fc7e3fff700 1 -- 172.21.15.45:0/1352401700 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fc7f4003a00 con 0x7fc7fc100940 2023-10-17T01:22:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.671+0000 7fc7e3fff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7e40a0ba0 0x7fc7e40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.671+0000 7fc7e3fff700 1 -- 172.21.15.45:0/1352401700 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc7f40cbd10 con 0x7fc7fc100940 2023-10-17T01:22:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.672+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7e40a0ba0 0x7fc7e40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.672+0000 7fc814b45b80 1 Processor -- start 2023-10-17T01:22:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.672+0000 7fc814b45b80 1 -- start start 2023-10-17T01:22:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dccdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dccf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc814b45b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dc259b0 con 0x56309dcced90 2023-10-17T01:22:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc814b45b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dc23370 con 0x56309dccd830 2023-10-17T01:22:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc814b45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dc1ec60 con 0x56309dcce190 2023-10-17T01:22:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7e40a0ba0 0x7fc7e40a3060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005ea0 tx=0x7fc7ec005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dccdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dccf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dccdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43118/0 (socket says 172.21.15.45:43118) 2023-10-17T01:22:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc7fad9d700 1 -- 172.21.15.45:0/1758725148 learned_addr learned my addr 172.21.15.45:0/1758725148 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.673+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fad9d700 1 -- 172.21.15.45:0/1758725148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 msgr2=0x56309dcce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fad9d700 1 -- 172.21.15.45:0/1758725148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 msgr2=0x56309dccf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dccf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fad9d700 1 -- 172.21.15.45:0/1758725148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56309da4cd80 con 0x56309dccd830 2023-10-17T01:22:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dccf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dccdc50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00a520 tx=0x7fc7ec011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7c2d76c52cd130 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.674+0000 7fc7e17fa700 1 -- 172.21.15.45:0/1758725148 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec010c20 con 0x56309dccd830 2023-10-17T01:22:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.675+0000 7fc7e17fa700 1 -- 172.21.15.45:0/1758725148 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7ec010d80 con 0x56309dccd830 2023-10-17T01:22:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.675+0000 7fc7e17fa700 1 -- 172.21.15.45:0/1758725148 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00f690 con 0x56309dccd830 2023-10-17T01:22:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.676+0000 7fc814b45b80 1 -- 172.21.15.45:0/1758725148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 msgr2=0x56309dccdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.676+0000 7fc814b45b80 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dccdc50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00a520 tx=0x7fc7ec011000 comp rx=0 tx=0).stop 2023-10-17T01:22:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.677+0000 7fc814b45b80 1 -- 172.21.15.45:0/1758725148 shutdown_connections 2023-10-17T01:22:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.677+0000 7fc814b45b80 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dccf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.677+0000 7fc814b45b80 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dccdc50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.677+0000 7fc814b45b80 1 --2- 172.21.15.45:0/1758725148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.677+0000 7fc814b45b80 1 -- 172.21.15.45:0/1758725148 >> 172.21.15.45:0/1758725148 conn(0x56309dc2d070 msgr2=0x56309dcc82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.678+0000 7fc814b45b80 1 -- 172.21.15.45:0/1758725148 shutdown_connections 2023-10-17T01:22:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.678+0000 7fc814b45b80 1 -- 172.21.15.45:0/1758725148 wait complete. 2023-10-17T01:22:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.679+0000 7fc814b45b80 1 -- start start 2023-10-17T01:22:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.680+0000 7fc814b45b80 1 Processor -- start 2023-10-17T01:22:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.680+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dcd7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.680+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.680+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dcde910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dcd7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dcd7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43124/0 (socket says 172.21.15.45:43124) 2023-10-17T01:22:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc7fad9d700 1 -- 172.21.15.45:0/3855568433 learned_addr learned my addr 172.21.15.45:0/3855568433 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dbbb4b0 con 0x56309dcced90 2023-10-17T01:22:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dbbb150 con 0x56309dccd830 2023-10-17T01:22:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309daa28c0 con 0x56309dcce190 2023-10-17T01:22:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.681+0000 7fc814b45b80 10 client.0 _ll_register_callbacks cb 0x56309db08a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.682+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dcde910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.682+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.683+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3855568433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 msgr2=0x56309dcd78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.683+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.683+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 msgr2=0x56309dcd7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.683+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dcd7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.683+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc10bcc0 con 0x56309dcced90 2023-10-17T01:22:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.683+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dcd7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.684+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dcde910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40c9890 tx=0x7fc7f4098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57181558801f11d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.684+0000 7fc7deff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:22:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.684+0000 7fc7deff5700 1 -- 172.21.15.45:0/3855568433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f401b320 con 0x56309dcced90 2023-10-17T01:22:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.684+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56309dc49cb0 con 0x56309dcced90 2023-10-17T01:22:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.685+0000 7fc7deff5700 1 -- 172.21.15.45:0/3855568433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc7f4099b20 con 0x56309dcced90 2023-10-17T01:22:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.685+0000 7fc7deff5700 1 -- 172.21.15.45:0/3855568433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f4099420 con 0x56309dcced90 2023-10-17T01:22:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.685+0000 7fc814b45b80 10 client.16047 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.685+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fc7fc1074d0 con 0x56309dcced90 2023-10-17T01:22:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.685+0000 7fc7deff5700 1 -- 172.21.15.45:0/3855568433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56309dc49cb0 con 0x56309dcced90 2023-10-17T01:22:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.685+0000 7fc7deff5700 1 -- 172.21.15.45:0/3855568433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc7f409f200 con 0x56309dcced90 2023-10-17T01:22:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc814b45b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc814b45b80 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc814b45b80 20 client.16047 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dce3030 0x56309dce5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc814b45b80 4 client.16047 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:22:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56309dc9ccd0 con 0x56309dce3030 2023-10-17T01:22:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.686+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dce3030 0x56309dce5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.687+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dce3030 0x56309dce5420 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1a8b57aafe2325 server_cookie=1416be6972035d41 in_seq=0 out_seq=0 2023-10-17T01:22:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.687+0000 7fc7deff5700 10 client.16047 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.688+0000 7fc7deff5700 1 -- 172.21.15.45:0/3855568433 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x56309dc9ccd0 con 0x56309dce3030 2023-10-17T01:22:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.688+0000 7fc7deff5700 10 client.16047 handle_command_reply: tid=0 2023-10-17T01:22:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.689+0000 7fc814b45b80 1 client.16047 shutdown 2023-10-17T01:22:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.690+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 msgr2=0x56309dcde910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.690+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dcde910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40c9890 tx=0x7fc7f4098d80 comp rx=0 tx=0).stop 2023-10-17T01:22:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 shutdown_connections 2023-10-17T01:22:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcced90 0x56309dcde910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dce3030 0x56309dce5420 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dccd830 0x56309dcd7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3855568433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 >> 172.21.15.45:0/3855568433 conn(0x56309dc2d070 msgr2=0x56309dcc7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc7deff5700 0 client.16047 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 shutdown_connections 2023-10-17T01:22:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.691+0000 7fc814b45b80 1 -- 172.21.15.45:0/3855568433 wait complete. 2023-10-17T01:22:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.692+0000 7fc814b45b80 20 client.16047 trim_cache size 0 max 16384 2023-10-17T01:22:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.716+0000 7fc814b45b80 1 Processor -- start 2023-10-17T01:22:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.716+0000 7fc814b45b80 1 -- start start 2023-10-17T01:22:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dccd830 0x56309dce5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dcc7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc814b45b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dcc95b0 con 0x56309dccd830 2023-10-17T01:22:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc814b45b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dcd6840 con 0x56309dcced90 2023-10-17T01:22:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc814b45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dcd6f30 con 0x56309dcce190 2023-10-17T01:22:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dcc7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dccd830 0x56309dce5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc7fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dcc7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43132/0 (socket says 172.21.15.45:43132) 2023-10-17T01:22:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.717+0000 7fc7fb59e700 1 -- 172.21.15.45:0/4235437390 learned_addr learned my addr 172.21.15.45:0/4235437390 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.718+0000 7fc7fa59c700 1 -- 172.21.15.45:0/4235437390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 msgr2=0x56309dcc7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.718+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dcc7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.718+0000 7fc7fa59c700 1 -- 172.21.15.45:0/4235437390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dccd830 msgr2=0x56309dce5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.718+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dccd830 0x56309dce5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.718+0000 7fc7fa59c700 1 -- 172.21.15.45:0/4235437390 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec00e9c0 con 0x56309dcce190 2023-10-17T01:22:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.718+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd6030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000c00 tx=0x7fc7f001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a884fada42d0cb10 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.719+0000 7fc7e1ffb700 1 -- 172.21.15.45:0/4235437390 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f001d040 con 0x56309dcce190 2023-10-17T01:22:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.719+0000 7fc7e1ffb700 1 -- 172.21.15.45:0/4235437390 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f001e040 con 0x56309dcce190 2023-10-17T01:22:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.719+0000 7fc7e1ffb700 1 -- 172.21.15.45:0/4235437390 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000f460 con 0x56309dcce190 2023-10-17T01:22:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 -- 172.21.15.45:0/4235437390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 msgr2=0x56309dcd6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd6030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000c00 tx=0x7fc7f001a040 comp rx=0 tx=0).stop 2023-10-17T01:22:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 -- 172.21.15.45:0/4235437390 shutdown_connections 2023-10-17T01:22:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dcc7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dccd830 0x56309dce5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 --2- 172.21.15.45:0/4235437390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dcce190 0x56309dcd6030 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 -- 172.21.15.45:0/4235437390 >> 172.21.15.45:0/4235437390 conn(0x56309dc2d070 msgr2=0x56309dcc8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 -- 172.21.15.45:0/4235437390 shutdown_connections 2023-10-17T01:22:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.720+0000 7fc814b45b80 1 -- 172.21.15.45:0/4235437390 wait complete. 2023-10-17T01:22:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.721+0000 7fc814b45b80 1 -- start start 2023-10-17T01:22:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.721+0000 7fc814b45b80 1 Processor -- start 2023-10-17T01:22:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dccd830 0x56309dd2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcce190 0x56309dd30eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dd33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dd33940 con 0x56309dcce190 2023-10-17T01:22:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dd33a80 con 0x56309dcced90 2023-10-17T01:22:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56309dd33c40 con 0x56309dccd830 2023-10-17T01:22:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc814b45b80 10 client.0 _ll_register_callbacks cb 0x7fc7f4099210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:22:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dd33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dccd830 0x56309dd2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.722+0000 7fc7fb59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dd33400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:43146/0 (socket says 172.21.15.45:43146) 2023-10-17T01:22:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3046427498 learned_addr learned my addr 172.21.15.45:0/3046427498 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:22:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dccd830 0x56309dd2e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51496/0 (socket says 172.21.15.45:51496) 2023-10-17T01:22:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcce190 0x56309dd30eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3046427498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dccd830 msgr2=0x56309dd2e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dccd830 0x56309dd2e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3046427498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcce190 msgr2=0x56309dd30eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:22:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcce190 0x56309dd30eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fb59e700 1 -- 172.21.15.45:0/3046427498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f000b890 con 0x56309dcced90 2023-10-17T01:22:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.723+0000 7fc7fa59c700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcce190 0x56309dd30eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:22:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.724+0000 7fc7fb59e700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dd33400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40c9bb0 tx=0x7fc7f4098f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7259da3d08d9886 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:22:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.724+0000 7fc7deff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:22:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.724+0000 7fc7deff5700 1 -- 172.21.15.45:0/3046427498 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4098a80 con 0x56309dcced90 2023-10-17T01:22:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.724+0000 7fc7deff5700 1 -- 172.21.15.45:0/3046427498 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc7f4099b20 con 0x56309dcced90 2023-10-17T01:22:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.725+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7ec010d80 con 0x56309dcced90 2023-10-17T01:22:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.725+0000 7fc7deff5700 1 -- 172.21.15.45:0/3046427498 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f401b320 con 0x56309dcced90 2023-10-17T01:22:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.725+0000 7fc814b45b80 10 client.26336 fetch_fsmap learned FSMap version 40 2023-10-17T01:22:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.725+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56309da35c00 con 0x56309dcced90 2023-10-17T01:22:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc7deff5700 1 -- 172.21.15.45:0/3046427498 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7ec010d80 con 0x56309dcced90 2023-10-17T01:22:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc7deff5700 1 -- 172.21.15.45:0/3046427498 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fc7f4095210 con 0x56309dcced90 2023-10-17T01:22:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc814b45b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:22:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc814b45b80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:22:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc814b45b80 20 client.26336 populate_metadata read hostname 'smithi045' 2023-10-17T01:22:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dd35020 0x56309dd37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:22:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc814b45b80 4 client.26336 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:22:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56309dc17ce0 con 0x56309dd35020 2023-10-17T01:22:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.726+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dd35020 0x56309dd37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:22:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.727+0000 7fc7deff5700 10 client.26336 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.727+0000 7fc7fad9d700 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dd35020 0x56309dd37440 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c96c0f728532f7e server_cookie=bc2a8248a4f7d24e in_seq=0 out_seq=0 2023-10-17T01:22:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc7deff5700 1 -- 172.21.15.45:0/3046427498 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56309dc17ce0 con 0x56309dd35020 2023-10-17T01:22:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc7deff5700 10 client.26336 handle_command_reply: tid=0 2023-10-17T01:22:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 client.26336 shutdown 2023-10-17T01:22:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 msgr2=0x56309dd33400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dd33400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40c9bb0 tx=0x7fc7f4098f60 comp rx=0 tx=0).stop 2023-10-17T01:22:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 shutdown_connections 2023-10-17T01:22:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56309dd35020 0x56309dd37440 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56309dcced90 0x56309dd33400 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56309dccd830 0x56309dd2e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 --2- 172.21.15.45:0/3046427498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56309dcce190 0x56309dd30eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc7deff5700 0 client.26336 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:22:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 >> 172.21.15.45:0/3046427498 conn(0x56309dc2d070 msgr2=0x56309dcd5220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 shutdown_connections 2023-10-17T01:22:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 1 -- 172.21.15.45:0/3046427498 wait complete. 2023-10-17T01:22:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.792+0000 7fc814b45b80 20 client.26336 trim_cache size 0 max 16384 2023-10-17T01:22:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:22:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:22:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:22:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:22:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7e40a0ba0 msgr2=0x7fc7e40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7e40a0ba0 0x7fc7e40a3060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005ea0 tx=0x7fc7ec005e30 comp rx=0 tx=0).stop 2023-10-17T01:22:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 msgr2=0x7fc7fc106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:22:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc106720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400ef30 tx=0x7fc7f400c3a0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 shutdown_connections 2023-10-17T01:22:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fc7e40a0ba0 0x7fc7e40a3060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7fc100940 0x7fc7fc106720 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7fc0ff620 0x7fc7fc105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 --2- 172.21.15.45:0/1352401700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7fc0fff80 0x7fc7fc105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:22:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 >> 172.21.15.45:0/1352401700 conn(0x7fc7fc0fb1a0 msgr2=0x7fc7fc0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:22:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 shutdown_connections 2023-10-17T01:22:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:22:51.793+0000 7fc8011ff700 1 -- 172.21.15.45:0/1352401700 wait complete. 2023-10-17T01:22:53.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:52 smithi045 ceph-mon[98474]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-17T01:22:53.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:52 smithi060 ceph-mon[134136]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-17T01:22:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:52 smithi181 ceph-mon[134520]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-17T01:22:55.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:54 smithi045 ceph-mon[98474]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2023-10-17T01:22:55.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:54 smithi060 ceph-mon[134136]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2023-10-17T01:22:55.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:54 smithi181 ceph-mon[134520]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2023-10-17T01:22:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 465.0 s, 1958.2 tps, lat 50.940 ms stddev 61.467 2023-10-17T01:22:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:56 smithi045 ceph-mon[98474]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 650 op/s 2023-10-17T01:22:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:56 smithi060 ceph-mon[134136]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 650 op/s 2023-10-17T01:22:57.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:56 smithi181 ceph-mon[134520]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 650 op/s 2023-10-17T01:22:59.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:22:58 smithi045 ceph-mon[98474]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-17T01:22:59.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:22:58 smithi060 ceph-mon[134136]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-17T01:22:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:22:58 smithi181 ceph-mon[134520]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-17T01:23:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 470.0 s, 1285.0 tps, lat 77.820 ms stddev 100.094 2023-10-17T01:23:02.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:01 smithi045 ceph-mon[98474]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 853 op/s 2023-10-17T01:23:02.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:01 smithi060 ceph-mon[134136]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 853 op/s 2023-10-17T01:23:02.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:01 smithi181 ceph-mon[134520]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 853 op/s 2023-10-17T01:23:03.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:02 smithi045 ceph-mon[98474]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 782 op/s 2023-10-17T01:23:03.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:02 smithi060 ceph-mon[134136]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 782 op/s 2023-10-17T01:23:03.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:02 smithi181 ceph-mon[134520]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 782 op/s 2023-10-17T01:23:03.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:23:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 -- 172.21.15.45:0/1389139139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 msgr2=0x7f8ee0077b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/1389139139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee0077b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800b0a0 tx=0x7f8ed800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 -- 172.21.15.45:0/1389139139 shutdown_connections 2023-10-17T01:23:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/1389139139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee0077b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/1389139139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee0075710 0x7f8ee0075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/1389139139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee0076dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 -- 172.21.15.45:0/1389139139 >> 172.21.15.45:0/1389139139 conn(0x7f8ee00fe500 msgr2=0x7f8ee0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 -- 172.21.15.45:0/1389139139 shutdown_connections 2023-10-17T01:23:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.113+0000 7f8ee73ff700 1 -- 172.21.15.45:0/1389139139 wait complete. 2023-10-17T01:23:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.114+0000 7f8ee73ff700 1 Processor -- start 2023-10-17T01:23:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.114+0000 7f8ee73ff700 1 -- start start 2023-10-17T01:23:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.116+0000 7f8ee73ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee0075710 0x7f8ee019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.116+0000 7f8ee73ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.116+0000 7f8ee73ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee01a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.116+0000 7f8ee499a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee73ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee01a4ce0 con 0x7f8ee0076950 2023-10-17T01:23:04.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee73ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee01a4e20 con 0x7f8ee0075710 2023-10-17T01:23:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee73ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee01a4fb0 con 0x7f8ee0077310 2023-10-17T01:23:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee499a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56766/0 (socket says 172.21.15.45:56766) 2023-10-17T01:23:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee499a700 1 -- 172.21.15.45:0/256305474 learned_addr learned my addr 172.21.15.45:0/256305474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee519b700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee0075710 0x7f8ee019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.117+0000 7f8ee599c700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee01a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee499a700 1 -- 172.21.15.45:0/256305474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 msgr2=0x7f8ee01a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee499a700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee01a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee499a700 1 -- 172.21.15.45:0/256305474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee0075710 msgr2=0x7f8ee019d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee499a700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee0075710 0x7f8ee019d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee499a700 1 -- 172.21.15.45:0/256305474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8009d00 con 0x7f8ee0076950 2023-10-17T01:23:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee599c700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee01a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ee499a700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee019d6a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000d980 tx=0x7f8ed000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.118+0000 7f8ed67fc700 1 -- 172.21.15.45:0/256305474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004220 con 0x7f8ee0076950 2023-10-17T01:23:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.119+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee01a5290 con 0x7f8ee0076950 2023-10-17T01:23:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.119+0000 7f8ed67fc700 1 -- 172.21.15.45:0/256305474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed0007640 con 0x7f8ee0076950 2023-10-17T01:23:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.119+0000 7f8ed67fc700 1 -- 172.21.15.45:0/256305474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000bd10 con 0x7f8ee0076950 2023-10-17T01:23:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.119+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee01a57f0 con 0x7f8ee0076950 2023-10-17T01:23:04.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.126+0000 7f8ed67fc700 1 -- 172.21.15.45:0/256305474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8ed0004380 con 0x7f8ee0076950 2023-10-17T01:23:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.127+0000 7f8efac39b80 1 Processor -- start 2023-10-17T01:23:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.127+0000 7f8efac39b80 1 -- start start 2023-10-17T01:23:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.127+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9e4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8efac39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f93aa10 con 0x55983f9e2ba0 2023-10-17T01:23:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8efac39b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f9383d0 con 0x55983f9e3500 2023-10-17T01:23:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ed67fc700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8ecc0a0ba0 0x7f8ecc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8efac39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f933cc0 con 0x55983f9e42b0 2023-10-17T01:23:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ee599c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ee599c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9e2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56774/0 (socket says 172.21.15.45:56774) 2023-10-17T01:23:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ed67fc700 1 -- 172.21.15.45:0/256305474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8ed00cbfe0 con 0x7f8ee0076950 2023-10-17T01:23:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ee519b700 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ee599c700 1 -- 172.21.15.45:0/183955485 learned_addr learned my addr 172.21.15.45:0/183955485 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ee599c700 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.128+0000 7f8ee499a700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8ecc0a0ba0 0x7f8ecc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee519b700 1 -- 172.21.15.45:0/183955485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 msgr2=0x55983f9e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee519b700 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9e4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee519b700 1 -- 172.21.15.45:0/183955485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 msgr2=0x55983f9e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee519b700 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee519b700 1 -- 172.21.15.45:0/183955485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983f761de0 con 0x55983f9e3500 2023-10-17T01:23:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee599c700 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.129+0000 7f8ee519b700 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9e3d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009c00 tx=0x7f8edc009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb54ff2f255265 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.130+0000 7f8ee499a700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8ecc0a0ba0 0x7f8ecc0a3060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0096070 tx=0x7f8ed000b6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.130+0000 7f8ecb7fe700 1 -- 172.21.15.45:0/183955485 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc011b70 con 0x55983f9e3500 2023-10-17T01:23:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.130+0000 7f8ecb7fe700 1 -- 172.21.15.45:0/183955485 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8edc011cd0 con 0x55983f9e3500 2023-10-17T01:23:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.130+0000 7f8ecb7fe700 1 -- 172.21.15.45:0/183955485 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00f6a0 con 0x55983f9e3500 2023-10-17T01:23:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 -- 172.21.15.45:0/183955485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 msgr2=0x55983f9e3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9e3d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009c00 tx=0x7f8edc009f10 comp rx=0 tx=0).stop 2023-10-17T01:23:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 -- 172.21.15.45:0/183955485 shutdown_connections 2023-10-17T01:23:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 --2- 172.21.15.45:0/183955485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9e3d20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 -- 172.21.15.45:0/183955485 >> 172.21.15.45:0/183955485 conn(0x55983f9420d0 msgr2=0x55983f9dd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:04.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 -- 172.21.15.45:0/183955485 shutdown_connections 2023-10-17T01:23:04.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.131+0000 7f8efac39b80 1 -- 172.21.15.45:0/183955485 wait complete. 2023-10-17T01:23:04.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.132+0000 7f8efac39b80 1 -- start start 2023-10-17T01:23:04.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.132+0000 7f8efac39b80 1 Processor -- start 2023-10-17T01:23:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9ec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f8d0510 con 0x55983f9e2ba0 2023-10-17T01:23:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f8d01b0 con 0x55983f9e3500 2023-10-17T01:23:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f7b7920 con 0x55983f9e42b0 2023-10-17T01:23:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8efac39b80 10 client.0 _ll_register_callbacks cb 0x55983f81da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8ee519b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8ee519b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9ece70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38016/0 (socket says 172.21.15.45:38016) 2023-10-17T01:23:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8ee519b700 1 -- 172.21.15.45:0/3677610214 learned_addr learned my addr 172.21.15.45:0/3677610214 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8ee599c700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9ec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.133+0000 7f8ee599c700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ee519b700 1 -- 172.21.15.45:0/3677610214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 msgr2=0x55983f9f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ee519b700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ee519b700 1 -- 172.21.15.45:0/3677610214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 msgr2=0x55983f9ec930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ee519b700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9ec930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ee519b700 1 -- 172.21.15.45:0/3677610214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee00785c0 con 0x55983f9e3500 2023-10-17T01:23:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ee519b700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9ece70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8edc01c040 tx=0x7f8edc012680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4328105ce9f55bb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ec8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:23:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/3677610214 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc011e40 con 0x55983f9e3500 2023-10-17T01:23:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/3677610214 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8edc004030 con 0x55983f9e3500 2023-10-17T01:23:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.134+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55983f95ed10 con 0x55983f9e3500 2023-10-17T01:23:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/3677610214 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8edc004230 con 0x55983f9e3500 2023-10-17T01:23:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 10 client.26348 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f8edc009830 con 0x55983f9e3500 2023-10-17T01:23:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/3677610214 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8edc032020 con 0x55983f9e3500 2023-10-17T01:23:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/3677610214 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8edc02a060 con 0x55983f9e3500 2023-10-17T01:23:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 10 client.26348 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 20 client.26348 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983f9f8600 0x55983f9faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 4 client.26348 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.135+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55983f9b1d30 con 0x55983f9f8600 2023-10-17T01:23:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.136+0000 7f8ee599c700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983f9f8600 0x55983f9faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.137+0000 7f8ee599c700 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983f9f8600 0x55983f9faa20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fb96e9a4f6d7fc server_cookie=a2cd13d900727be9 in_seq=0 out_seq=0 2023-10-17T01:23:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.137+0000 7f8ec8ff9700 10 client.26348 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.138+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/3677610214 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55983f9b1d30 con 0x55983f9f8600 2023-10-17T01:23:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.138+0000 7f8ec8ff9700 10 client.26348 handle_command_reply: tid=0 2023-10-17T01:23:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.138+0000 7f8efac39b80 1 client.26348 shutdown 2023-10-17T01:23:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 msgr2=0x55983f9ece70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9ece70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8edc01c040 tx=0x7f8edc012680 comp rx=0 tx=0).stop 2023-10-17T01:23:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 shutdown_connections 2023-10-17T01:23:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e42b0 0x55983f9f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9ec930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983f9f8600 0x55983f9faa20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 --2- 172.21.15.45:0/3677610214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e3500 0x55983f9ece70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 >> 172.21.15.45:0/3677610214 conn(0x55983f9420d0 msgr2=0x55983f9dd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8ec8ff9700 0 client.26348 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 shutdown_connections 2023-10-17T01:23:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.139+0000 7f8efac39b80 1 -- 172.21.15.45:0/3677610214 wait complete. 2023-10-17T01:23:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.140+0000 7f8efac39b80 20 client.26348 trim_cache size 0 max 16384 2023-10-17T01:23:04.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.166+0000 7f8efac39b80 1 Processor -- start 2023-10-17T01:23:04.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.166+0000 7f8efac39b80 1 -- start start 2023-10-17T01:23:04.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.167+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e2ba0 0x55983f9ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.167+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.167+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 0x55983f9ece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.167+0000 7f8efac39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f9ebde0 con 0x55983f9e42b0 2023-10-17T01:23:04.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.167+0000 7f8efac39b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f9ec4a0 con 0x55983f9e2ba0 2023-10-17T01:23:04.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.167+0000 7f8efac39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f9da420 con 0x55983f9e3500 2023-10-17T01:23:04.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.170+0000 7f8ee599c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 0x55983f9ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.170+0000 7f8ee599c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 0x55983f9ece30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56798/0 (socket says 172.21.15.45:56798) 2023-10-17T01:23:04.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.170+0000 7f8ee599c700 1 -- 172.21.15.45:0/2429054932 learned_addr learned my addr 172.21.15.45:0/2429054932 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:04.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee519b700 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee599c700 1 -- 172.21.15.45:0/2429054932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 msgr2=0x55983f9ec8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee599c700 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ec8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee599c700 1 -- 172.21.15.45:0/2429054932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e2ba0 msgr2=0x55983f9ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee599c700 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e2ba0 0x55983f9ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee599c700 1 -- 172.21.15.45:0/2429054932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983f74ac60 con 0x55983f9e42b0 2023-10-17T01:23:04.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.171+0000 7f8ee519b700 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:23:04.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.172+0000 7f8ee599c700 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 0x55983f9ece30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8007a60 tx=0x7f8ed80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347e2372389a8fbe server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:04.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.176+0000 7f8ecbfff700 1 -- 172.21.15.45:0/2429054932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8026070 con 0x55983f9e42b0 2023-10-17T01:23:04.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.176+0000 7f8ecbfff700 1 -- 172.21.15.45:0/2429054932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed8027040 con 0x55983f9e42b0 2023-10-17T01:23:04.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.176+0000 7f8ecbfff700 1 -- 172.21.15.45:0/2429054932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800e040 con 0x55983f9e42b0 2023-10-17T01:23:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 -- 172.21.15.45:0/2429054932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 msgr2=0x55983f9ece30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 0x55983f9ece30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8007a60 tx=0x7f8ed80099e0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 -- 172.21.15.45:0/2429054932 shutdown_connections 2023-10-17T01:23:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e42b0 0x55983f9ece30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e2ba0 0x55983f9ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2429054932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 -- 172.21.15.45:0/2429054932 >> 172.21.15.45:0/2429054932 conn(0x55983f9420d0 msgr2=0x55983f9eb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 -- 172.21.15.45:0/2429054932 shutdown_connections 2023-10-17T01:23:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.180+0000 7f8efac39b80 1 -- 172.21.15.45:0/2429054932 wait complete. 2023-10-17T01:23:04.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.181+0000 7f8efac39b80 1 -- start start 2023-10-17T01:23:04.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.181+0000 7f8efac39b80 1 Processor -- start 2023-10-17T01:23:04.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e42b0 0x55983f9eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983f9ef380 con 0x55983f9e2ba0 2023-10-17T01:23:04.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983fa5afa0 con 0x55983f9e42b0 2023-10-17T01:23:04.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983fa5b110 con 0x55983f9e3500 2023-10-17T01:23:04.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.182+0000 7f8efac39b80 10 client.0 _ll_register_callbacks cb 0x7f8edc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:04.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.183+0000 7f8ee599c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.183+0000 7f8ee599c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56814/0 (socket says 172.21.15.45:56814) 2023-10-17T01:23:04.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.183+0000 7f8ee599c700 1 -- 172.21.15.45:0/2601968462 learned_addr learned my addr 172.21.15.45:0/2601968462 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:04.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.185+0000 7f8ee519b700 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.185+0000 7f8ee519b700 1 -- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e42b0 msgr2=0x55983f9eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.185+0000 7f8ee519b700 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e42b0 0x55983f9eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.185+0000 7f8ee519b700 1 -- 172.21.15.45:0/2601968462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 msgr2=0x55983f9f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.185+0000 7f8ee519b700 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.185+0000 7f8ee519b700 1 -- 172.21.15.45:0/2601968462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed800ce20 con 0x55983f9e3500 2023-10-17T01:23:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.189+0000 7f8ee519b700 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ee7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55983f8d8460 tx=0x7f8edc0197c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c289f0ded248fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.189+0000 7f8ec8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:23:04.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.191+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/2601968462 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0092a0 con 0x55983f9e3500 2023-10-17T01:23:04.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.192+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/2601968462 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8edc000e40 con 0x55983f9e3500 2023-10-17T01:23:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.192+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8edc011cd0 con 0x55983f9e3500 2023-10-17T01:23:04.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.196+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/2601968462 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8edc012a00 con 0x55983f9e3500 2023-10-17T01:23:04.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.196+0000 7f8efac39b80 10 client.26311 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:04.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.196+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55983f788250 con 0x55983f9e3500 2023-10-17T01:23:04.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.196+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/2601968462 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8edc011cd0 con 0x55983f9e3500 2023-10-17T01:23:04.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/2601968462 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8edc0113e0 con 0x55983f9e3500 2023-10-17T01:23:04.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8efac39b80 10 client.26311 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:04.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8efac39b80 10 client.26311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:04.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8efac39b80 20 client.26311 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:04.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983fa5c700 0x55983fa5eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:04.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8efac39b80 4 client.26311 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:23:04.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.197+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55983f92cd40 con 0x55983fa5c700 2023-10-17T01:23:04.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.209+0000 7f8ee599c700 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983fa5c700 0x55983fa5eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:04.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.210+0000 7f8ec8ff9700 10 client.26311 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:04.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.210+0000 7f8ee599c700 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983fa5c700 0x55983fa5eb20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeef749899e584ce server_cookie=b707071200a90a22 in_seq=0 out_seq=0 2023-10-17T01:23:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.219+0000 7f8ec8ff9700 1 -- 172.21.15.45:0/2601968462 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55983f92cd40 con 0x55983fa5c700 2023-10-17T01:23:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.219+0000 7f8ec8ff9700 10 client.26311 handle_command_reply: tid=0 2023-10-17T01:23:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.219+0000 7f8efac39b80 1 client.26311 shutdown 2023-10-17T01:23:04.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 msgr2=0x55983f9ee7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ee7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55983f8d8460 tx=0x7f8edc0197c0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.223 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:23:04.224 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:23:04.224 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:23:04.224 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:23:04.224 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:23:04.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:23:04.224 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:23:04.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:23:04.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:23:04.225 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:23:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 shutdown_connections 2023-10-17T01:23:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55983fa5c700 0x55983fa5eb20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55983f9e42b0 0x55983f9eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8ec8ff9700 0 client.26311 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983f9e2ba0 0x55983f9f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 --2- 172.21.15.45:0/2601968462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55983f9e3500 0x55983f9ee7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 >> 172.21.15.45:0/2601968462 conn(0x55983f9420d0 msgr2=0x55983f9ea720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 shutdown_connections 2023-10-17T01:23:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.221+0000 7f8efac39b80 1 -- 172.21.15.45:0/2601968462 wait complete. 2023-10-17T01:23:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.222+0000 7f8efac39b80 20 client.26311 trim_cache size 0 max 16384 2023-10-17T01:23:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8ecc0a0ba0 msgr2=0x7f8ecc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8ecc0a0ba0 0x7f8ecc0a3060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0096070 tx=0x7f8ed000b6f0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 msgr2=0x7f8ee019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:04.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee019d6a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000d980 tx=0x7f8ed000dc90 comp rx=0 tx=0).stop 2023-10-17T01:23:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 shutdown_connections 2023-10-17T01:23:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8ecc0a0ba0 0x7f8ecc0a3060 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee0077310 0x7f8ee01a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee0075710 0x7f8ee019d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 --2- 172.21.15.45:0/256305474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ee0076950 0x7f8ee019d6a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.224+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 >> 172.21.15.45:0/256305474 conn(0x7f8ee00fe500 msgr2=0x7f8ee0100120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.226+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 shutdown_connections 2023-10-17T01:23:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:04.228+0000 7f8ee73ff700 1 -- 172.21.15.45:0/256305474 wait complete. 2023-10-17T01:23:04.263 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:23:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.030+0000 7fb64e1ff700 1 -- 172.21.15.45:0/3939694883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648100db0 msgr2=0x7fb6481011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.030+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/3939694883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648100db0 0x7fb6481011d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b0d0 tx=0x7fb63800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 -- 172.21.15.45:0/3939694883 shutdown_connections 2023-10-17T01:23:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/3939694883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648102940 0x7fb6481091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/3939694883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648100db0 0x7fb6481011d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/3939694883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb648101fa0 0x7fb648102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 -- 172.21.15.45:0/3939694883 >> 172.21.15.45:0/3939694883 conn(0x7fb6480fc300 msgr2=0x7fb6480fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 -- 172.21.15.45:0/3939694883 shutdown_connections 2023-10-17T01:23:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.031+0000 7fb64e1ff700 1 -- 172.21.15.45:0/3939694883 wait complete. 2023-10-17T01:23:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.033+0000 7fb64e1ff700 1 Processor -- start 2023-10-17T01:23:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 -- start start 2023-10-17T01:23:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 0x7fb648073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648101fa0 0x7fb648073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb648102940 0x7fb648073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a0cb0 con 0x7fb648101fa0 2023-10-17T01:23:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a0df0 con 0x7fb648102940 2023-10-17T01:23:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.034+0000 7fb64e1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481a0f30 con 0x7fb648100db0 2023-10-17T01:23:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.035+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 0x7fb648073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.035+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 0x7fb648073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56898/0 (socket says 172.21.15.45:56898) 2023-10-17T01:23:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.035+0000 7fb6477fe700 1 -- 172.21.15.45:0/4001375172 learned_addr learned my addr 172.21.15.45:0/4001375172 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.035+0000 7fb646ffd700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648101fa0 0x7fb648073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.036+0000 7fb647fff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb648102940 0x7fb648073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.036+0000 7fb6477fe700 1 -- 172.21.15.45:0/4001375172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb648102940 msgr2=0x7fb648073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.036+0000 7fb6477fe700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb648102940 0x7fb648073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.036+0000 7fb6477fe700 1 -- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648101fa0 msgr2=0x7fb648073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.036+0000 7fb6477fe700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648101fa0 0x7fb648073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.036+0000 7fb6477fe700 1 -- 172.21.15.45:0/4001375172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638009d00 con 0x7fb648100db0 2023-10-17T01:23:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.052+0000 7fb6477fe700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 0x7fb648073140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b820 tx=0x7fb638009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.055+0000 7fb644ff9700 1 -- 172.21.15.45:0/4001375172 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800e040 con 0x7fb648100db0 2023-10-17T01:23:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.055+0000 7fb644ff9700 1 -- 172.21.15.45:0/4001375172 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638003e10 con 0x7fb648100db0 2023-10-17T01:23:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.055+0000 7fb644ff9700 1 -- 172.21.15.45:0/4001375172 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638005610 con 0x7fb648100db0 2023-10-17T01:23:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.055+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6481a1160 con 0x7fb648100db0 2023-10-17T01:23:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.056+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6481a1600 con 0x7fb648100db0 2023-10-17T01:23:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.064+0000 7fb644ff9700 1 -- 172.21.15.45:0/4001375172 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb638005770 con 0x7fb648100db0 2023-10-17T01:23:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.066+0000 7fb644ff9700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6300a0ba0 0x7fb6300a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.066+0000 7fb644ff9700 1 -- 172.21.15.45:0/4001375172 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6380cd6b0 con 0x7fb648100db0 2023-10-17T01:23:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.067+0000 7fb646ffd700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6300a0ba0 0x7fb6300a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.067+0000 7fb661a7cb80 1 Processor -- start 2023-10-17T01:23:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.067+0000 7fb661a7cb80 1 -- start start 2023-10-17T01:23:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 0x562c09fd8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd9190 0x562c09fd9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c09fda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09f309b0 con 0x562c09fd9d90 2023-10-17T01:23:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09f2e370 con 0x562c09fd9190 2023-10-17T01:23:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09f29c60 con 0x562c09fd8830 2023-10-17T01:23:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb646ffd700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6300a0ba0 0x7fb6300a3060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005950 tx=0x7fb63c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 0x562c09fd8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 0x562c09fd8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56910/0 (socket says 172.21.15.45:56910) 2023-10-17T01:23:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb6477fe700 1 -- 172.21.15.45:0/36239455 learned_addr learned my addr 172.21.15.45:0/36239455 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.068+0000 7fb647fff700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c09fda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb6477fe700 1 -- 172.21.15.45:0/36239455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd9190 msgr2=0x562c09fd9790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:23:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb646ffd700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd9190 0x562c09fd9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb6477fe700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd9190 0x562c09fd9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb6477fe700 1 -- 172.21.15.45:0/36239455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 msgr2=0x562c09fda210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb6477fe700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c09fda210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb6477fe700 1 -- 172.21.15.45:0/36239455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c09d57d80 con 0x562c09fd8830 2023-10-17T01:23:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb647fff700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c09fda210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:23:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.069+0000 7fb646ffd700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd9190 0x562c09fd9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:23:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.070+0000 7fb6477fe700 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 0x562c09fd8c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb63809ade0 tx=0x7fb63800b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae400d9d259e561 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.070+0000 7fb62dffb700 1 -- 172.21.15.45:0/36239455 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63809ae50 con 0x562c09fd8830 2023-10-17T01:23:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.070+0000 7fb62dffb700 1 -- 172.21.15.45:0/36239455 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63809ec80 con 0x562c09fd8830 2023-10-17T01:23:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.070+0000 7fb62dffb700 1 -- 172.21.15.45:0/36239455 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380976b0 con 0x562c09fd8830 2023-10-17T01:23:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.072+0000 7fb661a7cb80 1 -- 172.21.15.45:0/36239455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 msgr2=0x562c09fd8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.072+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 0x562c09fd8c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb63809ade0 tx=0x7fb63800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.073+0000 7fb661a7cb80 1 -- 172.21.15.45:0/36239455 shutdown_connections 2023-10-17T01:23:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.073+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c09fda210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.073+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd8830 0x562c09fd8c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.073+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/36239455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd9190 0x562c09fd9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.073+0000 7fb661a7cb80 1 -- 172.21.15.45:0/36239455 >> 172.21.15.45:0/36239455 conn(0x562c09f38070 msgr2=0x562c09fd32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.074+0000 7fb661a7cb80 1 -- 172.21.15.45:0/36239455 shutdown_connections 2023-10-17T01:23:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.074+0000 7fb661a7cb80 1 -- 172.21.15.45:0/36239455 wait complete. 2023-10-17T01:23:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.075+0000 7fb661a7cb80 1 -- start start 2023-10-17T01:23:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.076+0000 7fb661a7cb80 1 Processor -- start 2023-10-17T01:23:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fe2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09ec64b0 con 0x562c09fd9190 2023-10-17T01:23:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09ec6150 con 0x562c09fd8830 2023-10-17T01:23:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09dad8c0 con 0x562c09fd9d90 2023-10-17T01:23:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.077+0000 7fb661a7cb80 10 client.0 _ll_register_callbacks cb 0x562c09e13a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.078+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fe2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.078+0000 7fb647fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fe2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38048/0 (socket says 172.21.15.45:38048) 2023-10-17T01:23:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 -- 172.21.15.45:0/678265727 learned_addr learned my addr 172.21.15.45:0/678265727 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 -- 172.21.15.45:0/678265727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 msgr2=0x562c09fe95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 -- 172.21.15.45:0/678265727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 msgr2=0x562c09fe2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.079+0000 7fb6477fe700 1 -- 172.21.15.45:0/678265727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6481a1600 con 0x562c09fd8830 2023-10-17T01:23:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.080+0000 7fb6477fe700 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fe2050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb638009670 tx=0x7fb6380a2ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa742ee65999552b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.080+0000 7fb626ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:23:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.080+0000 7fb626ffd700 1 -- 172.21.15.45:0/678265727 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638097730 con 0x562c09fd8830 2023-10-17T01:23:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.080+0000 7fb626ffd700 1 -- 172.21.15.45:0/678265727 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb638019560 con 0x562c09fd8830 2023-10-17T01:23:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.081+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c09f54cb0 con 0x562c09fd8830 2023-10-17T01:23:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.081+0000 7fb626ffd700 1 -- 172.21.15.45:0/678265727 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb63809ae50 con 0x562c09fd8830 2023-10-17T01:23:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.081+0000 7fb661a7cb80 10 client.26354 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.081+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562c09d40c00 con 0x562c09fd8830 2023-10-17T01:23:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.082+0000 7fb626ffd700 1 -- 172.21.15.45:0/678265727 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c09f54cb0 con 0x562c09fd8830 2023-10-17T01:23:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.082+0000 7fb626ffd700 1 -- 172.21.15.45:0/678265727 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb6380a29a0 con 0x562c09fd8830 2023-10-17T01:23:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.082+0000 7fb661a7cb80 10 client.26354 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.082+0000 7fb661a7cb80 10 client.26354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.082+0000 7fb661a7cb80 20 client.26354 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.082+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c09fed820 0x562c09fefc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.083+0000 7fb661a7cb80 4 client.26354 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.083+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c09fa7cd0 con 0x562c09fed820 2023-10-17T01:23:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.083+0000 7fb646ffd700 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c09fed820 0x562c09fefc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.084+0000 7fb646ffd700 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c09fed820 0x562c09fefc10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ed7a385431a8e1 server_cookie=38bdb229f901bba4 in_seq=0 out_seq=0 2023-10-17T01:23:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.084+0000 7fb626ffd700 10 client.26354 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.085+0000 7fb626ffd700 1 -- 172.21.15.45:0/678265727 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x562c09fa7cd0 con 0x562c09fed820 2023-10-17T01:23:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.085+0000 7fb626ffd700 10 client.26354 handle_command_reply: tid=0 2023-10-17T01:23:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.085+0000 7fb661a7cb80 1 client.26354 shutdown 2023-10-17T01:23:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 msgr2=0x562c09fe2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fe2050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb638009670 tx=0x7fb6380a2ee0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 shutdown_connections 2023-10-17T01:23:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c09fed820 0x562c09fefc10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fe2050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/678265727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 >> 172.21.15.45:0/678265727 conn(0x562c09f38070 msgr2=0x562c09fd2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb626ffd700 0 client.26354 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 shutdown_connections 2023-10-17T01:23:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.086+0000 7fb661a7cb80 1 -- 172.21.15.45:0/678265727 wait complete. 2023-10-17T01:23:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.087+0000 7fb661a7cb80 20 client.26354 trim_cache size 0 max 16384 2023-10-17T01:23:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.116+0000 7fb661a7cb80 1 Processor -- start 2023-10-17T01:23:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 -- start start 2023-10-17T01:23:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fd3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09f54cb0 con 0x562c09fd9190 2023-10-17T01:23:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09fe3240 con 0x562c09fd8830 2023-10-17T01:23:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c09fe1550 con 0x562c09fd9d90 2023-10-17T01:23:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fd3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb647fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fd3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38050/0 (socket says 172.21.15.45:38050) 2023-10-17T01:23:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.117+0000 7fb6477fe700 1 -- 172.21.15.45:0/774842734 learned_addr learned my addr 172.21.15.45:0/774842734 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb646ffd700 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb6477fe700 1 -- 172.21.15.45:0/774842734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 msgr2=0x562c09fe01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb6477fe700 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb6477fe700 1 -- 172.21.15.45:0/774842734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 msgr2=0x562c09fe0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb6477fe700 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb6477fe700 1 -- 172.21.15.45:0/774842734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c09d7e1f0 con 0x562c09fd8830 2023-10-17T01:23:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb646ffd700 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:23:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb6477fe700 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fd3290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb6380991a0 tx=0x7fb638009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1118ba1abc4b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.118+0000 7fb62e7fc700 1 -- 172.21.15.45:0/774842734 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380a16f0 con 0x562c09fd8830 2023-10-17T01:23:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.119+0000 7fb62e7fc700 1 -- 172.21.15.45:0/774842734 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638099e50 con 0x562c09fd8830 2023-10-17T01:23:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.119+0000 7fb62e7fc700 1 -- 172.21.15.45:0/774842734 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63809a110 con 0x562c09fd8830 2023-10-17T01:23:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.120+0000 7fb661a7cb80 1 -- 172.21.15.45:0/774842734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 msgr2=0x562c09fd3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.120+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fd3290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb6380991a0 tx=0x7fb638009e90 comp rx=0 tx=0).stop 2023-10-17T01:23:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 -- 172.21.15.45:0/774842734 shutdown_connections 2023-10-17T01:23:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9d90 0x562c09fe01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c09fd3290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/774842734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9190 0x562c09fe0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 -- 172.21.15.45:0/774842734 >> 172.21.15.45:0/774842734 conn(0x562c09f38070 msgr2=0x562c09fd2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 -- 172.21.15.45:0/774842734 shutdown_connections 2023-10-17T01:23:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.121+0000 7fb661a7cb80 1 -- 172.21.15.45:0/774842734 wait complete. 2023-10-17T01:23:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.122+0000 7fb661a7cb80 1 -- start start 2023-10-17T01:23:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.122+0000 7fb661a7cb80 1 Processor -- start 2023-10-17T01:23:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.122+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c0a039150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.122+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 0x562c0a03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb661a7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c0a03dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0a03e2e0 con 0x562c09fd9d90 2023-10-17T01:23:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0a03e450 con 0x562c09fd8830 2023-10-17T01:23:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb661a7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0a03e610 con 0x562c09fd9190 2023-10-17T01:23:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb661a7cb80 10 client.0 _ll_register_callbacks cb 0x562c09e13a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb646ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 0x562c0a03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c0a039150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c0a039150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38060/0 (socket says 172.21.15.45:38060) 2023-10-17T01:23:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb6477fe700 1 -- 172.21.15.45:0/4071839883 learned_addr learned my addr 172.21.15.45:0/4071839883 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb646ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 0x562c0a03b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:56942/0 (socket says 172.21.15.45:56942) 2023-10-17T01:23:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.123+0000 7fb647fff700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c0a03dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.124+0000 7fb646ffd700 1 -- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 msgr2=0x562c0a039150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.124+0000 7fb646ffd700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c0a039150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.124+0000 7fb646ffd700 1 -- 172.21.15.45:0/4071839883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 msgr2=0x562c0a03dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.124+0000 7fb646ffd700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c0a03dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.124+0000 7fb646ffd700 1 -- 172.21.15.45:0/4071839883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c09d40c00 con 0x562c09fd9190 2023-10-17T01:23:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.124+0000 7fb6477fe700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c0a039150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.125+0000 7fb647fff700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c0a03dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:23:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.125+0000 7fb646ffd700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 0x562c0a03b6a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562c09ff04f0 tx=0x7fb63c00ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946d7fef7020ed86 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.125+0000 7fb626ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:23:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.126+0000 7fb626ffd700 1 -- 172.21.15.45:0/4071839883 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c024040 con 0x562c09fd9190 2023-10-17T01:23:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.126+0000 7fb626ffd700 1 -- 172.21.15.45:0/4071839883 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb63c011ad0 con 0x562c09fd9190 2023-10-17T01:23:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.126+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb63809ec80 con 0x562c09fd9190 2023-10-17T01:23:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.126+0000 7fb626ffd700 1 -- 172.21.15.45:0/4071839883 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb63c024040 con 0x562c09fd9190 2023-10-17T01:23:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 10 client.26326 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562c09d56250 con 0x562c09fd9190 2023-10-17T01:23:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb626ffd700 1 -- 172.21.15.45:0/4071839883 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63809ec80 con 0x562c09fd9190 2023-10-17T01:23:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb626ffd700 1 -- 172.21.15.45:0/4071839883 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb63c01f450 con 0x562c09fd9190 2023-10-17T01:23:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 10 client.26326 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 20 client.26326 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c0a040000 0x562c0a042420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 4 client.26326 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:23:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.127+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c09f22ce0 con 0x562c0a040000 2023-10-17T01:23:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.128+0000 7fb6477fe700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c0a040000 0x562c0a042420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.129+0000 7fb6477fe700 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c0a040000 0x562c0a042420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f4123e31e45053 server_cookie=341d9be5b7567b3e in_seq=0 out_seq=0 2023-10-17T01:23:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.129+0000 7fb626ffd700 10 client.26326 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.142+0000 7fb626ffd700 1 -- 172.21.15.45:0/4071839883 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c09f22ce0 con 0x562c0a040000 2023-10-17T01:23:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.142+0000 7fb626ffd700 10 client.26326 handle_command_reply: tid=0 2023-10-17T01:23:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 client.26326 shutdown 2023-10-17T01:23:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 msgr2=0x562c0a03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 0x562c0a03b6a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562c09ff04f0 tx=0x7fb63c00ede0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 shutdown_connections 2023-10-17T01:23:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c09fd9d90 0x562c0a03dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562c0a040000 0x562c0a042420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562c09fd8830 0x562c0a039150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 --2- 172.21.15.45:0/4071839883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c09fd9190 0x562c0a03b6a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb626ffd700 0 client.26326 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 >> 172.21.15.45:0/4071839883 conn(0x562c09f38070 msgr2=0x562c09fe0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 shutdown_connections 2023-10-17T01:23:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 1 -- 172.21.15.45:0/4071839883 wait complete. 2023-10-17T01:23:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.143+0000 7fb661a7cb80 20 client.26326 trim_cache size 0 max 16384 2023-10-17T01:23:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:23:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:23:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:23:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:23:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6300a0ba0 msgr2=0x7fb6300a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6300a0ba0 0x7fb6300a3060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005950 tx=0x7fb63c0058e0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 msgr2=0x7fb648073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 0x7fb648073140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b820 tx=0x7fb638009570 comp rx=0 tx=0).stop 2023-10-17T01:23:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 shutdown_connections 2023-10-17T01:23:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6300a0ba0 0x7fb6300a3060 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb648102940 0x7fb648073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb648100db0 0x7fb648073140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 --2- 172.21.15.45:0/4001375172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb648101fa0 0x7fb648073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 >> 172.21.15.45:0/4001375172 conn(0x7fb6480fc300 msgr2=0x7fb648105450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 shutdown_connections 2023-10-17T01:23:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:05.144+0000 7fb64e1ff700 1 -- 172.21.15.45:0/4001375172 wait complete. 2023-10-17T01:23:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 475.0 s, 1576.6 tps, lat 63.473 ms stddev 71.771 2023-10-17T01:23:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:05 smithi045 ceph-mon[98474]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 900 op/s 2023-10-17T01:23:06.085 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:05 smithi060 ceph-mon[134136]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 900 op/s 2023-10-17T01:23:06.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:05 smithi181 ceph-mon[134520]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 900 op/s 2023-10-17T01:23:07.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:06 smithi045 ceph-mon[98474]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 778 op/s 2023-10-17T01:23:07.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:06 smithi060 ceph-mon[134136]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 778 op/s 2023-10-17T01:23:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:06 smithi181 ceph-mon[134520]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 778 op/s 2023-10-17T01:23:09.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:08 smithi045 ceph-mon[98474]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 801 op/s 2023-10-17T01:23:09.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:08 smithi060 ceph-mon[134136]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 801 op/s 2023-10-17T01:23:09.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:08 smithi181 ceph-mon[134520]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 801 op/s 2023-10-17T01:23:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 480.0 s, 1589.6 tps, lat 62.054 ms stddev 68.113 2023-10-17T01:23:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:10 smithi045 ceph-mon[98474]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 843 op/s 2023-10-17T01:23:11.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:10 smithi060 ceph-mon[134136]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 843 op/s 2023-10-17T01:23:11.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:10 smithi181 ceph-mon[134520]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 843 op/s 2023-10-17T01:23:13.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:12 smithi045 ceph-mon[98474]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 685 op/s 2023-10-17T01:23:13.086 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:12 smithi060 ceph-mon[134136]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 685 op/s 2023-10-17T01:23:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:12 smithi181 ceph-mon[134520]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 685 op/s 2023-10-17T01:23:15.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:14 smithi045 ceph-mon[98474]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 759 op/s 2023-10-17T01:23:15.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:14 smithi060 ceph-mon[134136]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 759 op/s 2023-10-17T01:23:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:14 smithi181 ceph-mon[134520]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 759 op/s 2023-10-17T01:23:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 485.0 s, 1433.6 tps, lat 68.223 ms stddev 81.178 2023-10-17T01:23:17.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:16 smithi045 ceph-mon[98474]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 632 op/s 2023-10-17T01:23:17.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:16 smithi060 ceph-mon[134136]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 632 op/s 2023-10-17T01:23:17.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:16 smithi181 ceph-mon[134520]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 632 op/s 2023-10-17T01:23:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.369+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/1969105401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 msgr2=0x7f7e48106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/1969105401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48106590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0098f0 tx=0x7f7e3c009c00 comp rx=0 tx=0).stop 2023-10-17T01:23:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/1969105401 shutdown_connections 2023-10-17T01:23:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/1969105401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 0x7f7e48108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/1969105401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48101870 0x7f7e48103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/1969105401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48106590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/1969105401 >> 172.21.15.45:0/1969105401 conn(0x7f7e480fb1a0 msgr2=0x7f7e480fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/1969105401 shutdown_connections 2023-10-17T01:23:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.370+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/1969105401 wait complete. 2023-10-17T01:23:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 Processor -- start 2023-10-17T01:23:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 -- start start 2023-10-17T01:23:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48101870 0x7f7e48192040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48192580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 0x7f7e48193450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48193ab0 con 0x7f7e481041a0 2023-10-17T01:23:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48193c20 con 0x7f7e48101870 2023-10-17T01:23:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.372+0000 7f7e4e7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48193de0 con 0x7f7e48106ad0 2023-10-17T01:23:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.373+0000 7f7e4cd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 0x7f7e48193450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.373+0000 7f7e477fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48192580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.373+0000 7f7e4cd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 0x7f7e48193450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51858/0 (socket says 172.21.15.45:51858) 2023-10-17T01:23:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.373+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/3011914293 learned_addr learned my addr 172.21.15.45:0/3011914293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.373+0000 7f7e47fff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48101870 0x7f7e48192040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.374+0000 7f7e477fe700 1 -- 172.21.15.45:0/3011914293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 msgr2=0x7f7e48193450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.374+0000 7f7e477fe700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 0x7f7e48193450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.374+0000 7f7e477fe700 1 -- 172.21.15.45:0/3011914293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48101870 msgr2=0x7f7e48192040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.374+0000 7f7e477fe700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48101870 0x7f7e48192040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.374+0000 7f7e477fe700 1 -- 172.21.15.45:0/3011914293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c009520 con 0x7f7e481041a0 2023-10-17T01:23:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.374+0000 7f7e477fe700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48192580 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009230 tx=0x7f7e3c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.375+0000 7f7e457fa700 1 -- 172.21.15.45:0/3011914293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c01c070 con 0x7f7e481041a0 2023-10-17T01:23:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.375+0000 7f7e457fa700 1 -- 172.21.15.45:0/3011914293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e3c00f7d0 con 0x7f7e481041a0 2023-10-17T01:23:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.375+0000 7f7e457fa700 1 -- 172.21.15.45:0/3011914293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c003a80 con 0x7f7e481041a0 2023-10-17T01:23:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.375+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e48194060 con 0x7f7e481041a0 2023-10-17T01:23:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.375+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e48194870 con 0x7f7e481041a0 2023-10-17T01:23:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.378+0000 7f7e457fa700 1 -- 172.21.15.45:0/3011914293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f7e3c020030 con 0x7f7e481041a0 2023-10-17T01:23:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.380+0000 7f7e457fa700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7e300a0ba0 0x7f7e300a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.380+0000 7f7e457fa700 1 -- 172.21.15.45:0/3011914293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7e3c0cd060 con 0x7f7e481041a0 2023-10-17T01:23:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.380+0000 7f7e620f1b80 1 Processor -- start 2023-10-17T01:23:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.380+0000 7f7e47fff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7e300a0ba0 0x7f7e300a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.380+0000 7f7e620f1b80 1 -- start start 2023-10-17T01:23:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7190 0x5565ba4d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba42e9b0 con 0x5565ba4d6830 2023-10-17T01:23:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba42c370 con 0x5565ba4d7d90 2023-10-17T01:23:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba427c60 con 0x5565ba4d7190 2023-10-17T01:23:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e4cd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e4cd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4d8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51586/0 (socket says 172.21.15.45:51586) 2023-10-17T01:23:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.381+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/3130208287 learned_addr learned my addr 172.21.15.45:0/3130208287 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e477fe700 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7190 0x5565ba4d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/3130208287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7190 msgr2=0x5565ba4d7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7190 0x5565ba4d7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/3130208287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 msgr2=0x5565ba4d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/3130208287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565ba255d80 con 0x5565ba4d7d90 2023-10-17T01:23:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e47fff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7e300a0ba0 0x7f7e300a3060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e48191db0 tx=0x7f7e38011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.382+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4d8210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e400094d0 tx=0x7f7e40009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b9dce40fc33e43 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.383+0000 7f7e2e7fc700 1 -- 172.21.15.45:0/3130208287 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000e070 con 0x5565ba4d7d90 2023-10-17T01:23:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.383+0000 7f7e2e7fc700 1 -- 172.21.15.45:0/3130208287 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e4000f040 con 0x5565ba4d7d90 2023-10-17T01:23:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.383+0000 7f7e2e7fc700 1 -- 172.21.15.45:0/3130208287 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40005280 con 0x5565ba4d7d90 2023-10-17T01:23:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.385+0000 7f7e620f1b80 1 -- 172.21.15.45:0/3130208287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 msgr2=0x5565ba4d8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.385+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4d8210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e400094d0 tx=0x7f7e40009920 comp rx=0 tx=0).stop 2023-10-17T01:23:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 -- 172.21.15.45:0/3130208287 shutdown_connections 2023-10-17T01:23:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4d8210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/3130208287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7190 0x5565ba4d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 -- 172.21.15.45:0/3130208287 >> 172.21.15.45:0/3130208287 conn(0x5565ba436070 msgr2=0x5565ba4d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 -- 172.21.15.45:0/3130208287 shutdown_connections 2023-10-17T01:23:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.386+0000 7f7e620f1b80 1 -- 172.21.15.45:0/3130208287 wait complete. 2023-10-17T01:23:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.387+0000 7f7e620f1b80 1 -- start start 2023-10-17T01:23:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.387+0000 7f7e620f1b80 1 Processor -- start 2023-10-17T01:23:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba4e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7d90 0x5565ba4e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba3c44b0 con 0x5565ba4d6830 2023-10-17T01:23:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba3c4150 con 0x5565ba4d7190 2023-10-17T01:23:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba2ab8c0 con 0x5565ba4d7d90 2023-10-17T01:23:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.388+0000 7f7e620f1b80 10 client.0 _ll_register_callbacks cb 0x5565ba311a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.389+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.389+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4e0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47722/0 (socket says 172.21.15.45:47722) 2023-10-17T01:23:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.389+0000 7f7e47fff700 1 -- 172.21.15.45:0/2462117673 learned_addr learned my addr 172.21.15.45:0/2462117673 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.390+0000 7f7e477fe700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba4e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.390+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7d90 0x5565ba4e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.391+0000 7f7e47fff700 1 -- 172.21.15.45:0/2462117673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7d90 msgr2=0x5565ba4e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.391+0000 7f7e47fff700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7d90 0x5565ba4e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.391+0000 7f7e47fff700 1 -- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 msgr2=0x5565ba4e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.391+0000 7f7e47fff700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba4e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.391+0000 7f7e47fff700 1 -- 172.21.15.45:0/2462117673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e400091b0 con 0x5565ba4d6830 2023-10-17T01:23:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.392+0000 7f7e47fff700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4e0390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7e380097c0 tx=0x7f7e38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978bbee35115ff41 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.392+0000 7f7e277fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:23:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.393+0000 7f7e277fe700 1 -- 172.21.15.45:0/2462117673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38009e70 con 0x5565ba4d6830 2023-10-17T01:23:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.393+0000 7f7e277fe700 1 -- 172.21.15.45:0/2462117673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7e3800e950 con 0x5565ba4d6830 2023-10-17T01:23:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.393+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565ba452cb0 con 0x5565ba4d6830 2023-10-17T01:23:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.393+0000 7f7e277fe700 1 -- 172.21.15.45:0/2462117673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e3800eb10 con 0x5565ba4d6830 2023-10-17T01:23:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.393+0000 7f7e620f1b80 10 client.16092 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.393+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5565ba23ec00 con 0x5565ba4d6830 2023-10-17T01:23:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e277fe700 1 -- 172.21.15.45:0/2462117673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565ba452cb0 con 0x5565ba4d6830 2023-10-17T01:23:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e277fe700 1 -- 172.21.15.45:0/2462117673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7e3801e430 con 0x5565ba4d6830 2023-10-17T01:23:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e620f1b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e620f1b80 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e620f1b80 20 client.16092 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba4ec1a0 0x5565ba4ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e620f1b80 4 client.16092 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.394+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565ba4a5cd0 con 0x5565ba4ec1a0 2023-10-17T01:23:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.395+0000 7f7e477fe700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba4ec1a0 0x5565ba4ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.396+0000 7f7e477fe700 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba4ec1a0 0x5565ba4ee5c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13330f4c2d96a550 server_cookie=4b5a586f035fb0fa in_seq=0 out_seq=0 2023-10-17T01:23:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.396+0000 7f7e277fe700 10 client.16092 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.397+0000 7f7e277fe700 1 -- 172.21.15.45:0/2462117673 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5565ba4a5cd0 con 0x5565ba4ec1a0 2023-10-17T01:23:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.397+0000 7f7e277fe700 10 client.16092 handle_command_reply: tid=0 2023-10-17T01:23:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.397+0000 7f7e620f1b80 1 client.16092 shutdown 2023-10-17T01:23:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 msgr2=0x5565ba4e0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4e0390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7e380097c0 tx=0x7f7e38004750 comp rx=0 tx=0).stop 2023-10-17T01:23:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 shutdown_connections 2023-10-17T01:23:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4d7d90 0x5565ba4e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d6830 0x5565ba4e0390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba4ec1a0 0x5565ba4ee5c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/2462117673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba4e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 >> 172.21.15.45:0/2462117673 conn(0x5565ba436070 msgr2=0x5565ba4d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e277fe700 0 client.16092 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.398+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 shutdown_connections 2023-10-17T01:23:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.399+0000 7f7e620f1b80 1 -- 172.21.15.45:0/2462117673 wait complete. 2023-10-17T01:23:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.399+0000 7f7e620f1b80 20 client.16092 trim_cache size 0 max 16384 2023-10-17T01:23:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 Processor -- start 2023-10-17T01:23:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 -- start start 2023-10-17T01:23:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7190 0x5565ba4d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba4dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba4a5cd0 con 0x5565ba4d7190 2023-10-17T01:23:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba4cb9e0 con 0x5565ba4d7d90 2023-10-17T01:23:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.423+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba4d7bf0 con 0x5565ba4de760 2023-10-17T01:23:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.424+0000 7f7e477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.424+0000 7f7e4cd9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba4dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.424+0000 7f7e477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4de220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51598/0 (socket says 172.21.15.45:51598) 2023-10-17T01:23:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.424+0000 7f7e477fe700 1 -- 172.21.15.45:0/1355674652 learned_addr learned my addr 172.21.15.45:0/1355674652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.424+0000 7f7e47fff700 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7190 0x5565ba4d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/1355674652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 msgr2=0x5565ba4de220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4de220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/1355674652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7190 msgr2=0x5565ba4d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7190 0x5565ba4d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e4cd9c700 1 -- 172.21.15.45:0/1355674652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565ba27c1f0 con 0x5565ba4de760 2023-10-17T01:23:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e477fe700 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4de220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:23:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e47fff700 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7190 0x5565ba4d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:23:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba4dad50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e40007b70 tx=0x7f7e40005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740102568c7eafbd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.425+0000 7f7e2effd700 1 -- 172.21.15.45:0/1355674652 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000e040 con 0x5565ba4de760 2023-10-17T01:23:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.426+0000 7f7e2effd700 1 -- 172.21.15.45:0/1355674652 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e40003a00 con 0x5565ba4de760 2023-10-17T01:23:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.426+0000 7f7e2effd700 1 -- 172.21.15.45:0/1355674652 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40005460 con 0x5565ba4de760 2023-10-17T01:23:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 -- 172.21.15.45:0/1355674652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 msgr2=0x5565ba4dad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba4dad50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e40007b70 tx=0x7f7e40005b30 comp rx=0 tx=0).stop 2023-10-17T01:23:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 -- 172.21.15.45:0/1355674652 shutdown_connections 2023-10-17T01:23:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba4dad50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7190 0x5565ba4d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/1355674652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7d90 0x5565ba4de220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 -- 172.21.15.45:0/1355674652 >> 172.21.15.45:0/1355674652 conn(0x5565ba436070 msgr2=0x5565ba4df030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 -- 172.21.15.45:0/1355674652 shutdown_connections 2023-10-17T01:23:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.427+0000 7f7e620f1b80 1 -- 172.21.15.45:0/1355674652 wait complete. 2023-10-17T01:23:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.428+0000 7f7e620f1b80 1 -- start start 2023-10-17T01:23:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.428+0000 7f7e620f1b80 1 Processor -- start 2023-10-17T01:23:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba535d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7d90 0x5565ba536280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba53d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba53d750 con 0x5565ba4d7d90 2023-10-17T01:23:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba53d8c0 con 0x5565ba4d7190 2023-10-17T01:23:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba53da80 con 0x5565ba4de760 2023-10-17T01:23:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e620f1b80 10 client.0 _ll_register_callbacks cb 0x7f7e3800f6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.429+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba535d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba535d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51600/0 (socket says 172.21.15.45:51600) 2023-10-17T01:23:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 -- 172.21.15.45:0/4089755913 learned_addr learned my addr 172.21.15.45:0/4089755913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e477fe700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7d90 0x5565ba536280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e4cd9c700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba53d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 -- 172.21.15.45:0/4089755913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 msgr2=0x5565ba53d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba53d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 -- 172.21.15.45:0/4089755913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7d90 msgr2=0x5565ba536280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7d90 0x5565ba536280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 -- 172.21.15.45:0/4089755913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565ba255d80 con 0x5565ba4d7190 2023-10-17T01:23:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e477fe700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7d90 0x5565ba536280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:23:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e47fff700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba535d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e38006010 tx=0x7f7e3801ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445689cc098885aa server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.430+0000 7f7e277fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:23:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.431+0000 7f7e277fe700 1 -- 172.21.15.45:0/4089755913 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800e950 con 0x5565ba4d7190 2023-10-17T01:23:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.431+0000 7f7e277fe700 1 -- 172.21.15.45:0/4089755913 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7e3800eab0 con 0x5565ba4d7190 2023-10-17T01:23:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.431+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e4000f040 con 0x5565ba4d7190 2023-10-17T01:23:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.432+0000 7f7e277fe700 1 -- 172.21.15.45:0/4089755913 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e3800ec70 con 0x5565ba4d7190 2023-10-17T01:23:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.433+0000 7f7e620f1b80 10 client.26378 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.433+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5565ba254250 con 0x5565ba4d7190 2023-10-17T01:23:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e277fe700 1 -- 172.21.15.45:0/4089755913 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e4000f040 con 0x5565ba4d7190 2023-10-17T01:23:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e277fe700 1 -- 172.21.15.45:0/4089755913 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7e38004b30 con 0x5565ba4d7190 2023-10-17T01:23:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e620f1b80 10 client.26378 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e620f1b80 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e620f1b80 20 client.26378 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba53f3f0 0x5565ba541810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e620f1b80 4 client.26378 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:23:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.434+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565ba420ce0 con 0x5565ba53f3f0 2023-10-17T01:23:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.435+0000 7f7e477fe700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba53f3f0 0x5565ba541810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.435+0000 7f7e277fe700 10 client.26378 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.435+0000 7f7e477fe700 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba53f3f0 0x5565ba541810 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd454d152c86951 server_cookie=e4b7c702a7f98af7 in_seq=0 out_seq=0 2023-10-17T01:23:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.450+0000 7f7e277fe700 1 -- 172.21.15.45:0/4089755913 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565ba420ce0 con 0x5565ba53f3f0 2023-10-17T01:23:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.450+0000 7f7e277fe700 10 client.26378 handle_command_reply: tid=0 2023-10-17T01:23:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.450+0000 7f7e620f1b80 1 client.26378 shutdown 2023-10-17T01:23:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 msgr2=0x5565ba535d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba535d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e38006010 tx=0x7f7e3801ec60 comp rx=0 tx=0).stop 2023-10-17T01:23:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 shutdown_connections 2023-10-17T01:23:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565ba4de760 0x5565ba53d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565ba53f3f0 0x5565ba541810 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565ba4d7190 0x5565ba535d40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 --2- 172.21.15.45:0/4089755913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565ba4d7d90 0x5565ba536280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 >> 172.21.15.45:0/4089755913 conn(0x5565ba436070 msgr2=0x5565ba4cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e277fe700 0 client.26378 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 shutdown_connections 2023-10-17T01:23:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.451+0000 7f7e620f1b80 1 -- 172.21.15.45:0/4089755913 wait complete. 2023-10-17T01:23:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e620f1b80 20 client.26378 trim_cache size 0 max 16384 2023-10-17T01:23:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:23:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:23:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:23:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:23:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7e300a0ba0 msgr2=0x7f7e300a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7e300a0ba0 0x7f7e300a3060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e48191db0 tx=0x7f7e38011720 comp rx=0 tx=0).stop 2023-10-17T01:23:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 msgr2=0x7f7e48192580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48192580 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009230 tx=0x7f7e3c00b840 comp rx=0 tx=0).stop 2023-10-17T01:23:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 shutdown_connections 2023-10-17T01:23:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f7e300a0ba0 0x7f7e300a3060 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.452+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e48106ad0 0x7f7e48193450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.453+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48101870 0x7f7e48192040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.453+0000 7f7e4e7ff700 1 --2- 172.21.15.45:0/3011914293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e481041a0 0x7f7e48192580 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.453+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 >> 172.21.15.45:0/3011914293 conn(0x7f7e480fb1a0 msgr2=0x7f7e481024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.453+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 shutdown_connections 2023-10-17T01:23:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:18.453+0000 7f7e4e7ff700 1 -- 172.21.15.45:0/3011914293 wait complete. 2023-10-17T01:23:19.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:18 smithi045 ceph-mon[98474]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 801 op/s 2023-10-17T01:23:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:18 smithi060 ceph-mon[134136]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 801 op/s 2023-10-17T01:23:19.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:18 smithi181 ceph-mon[134520]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 801 op/s 2023-10-17T01:23:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 490.0 s, 1289.7 tps, lat 80.331 ms stddev 137.156 2023-10-17T01:23:21.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:20 smithi045 ceph-mon[98474]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 1.60k op/s 2023-10-17T01:23:21.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:20 smithi060 ceph-mon[134136]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 1.60k op/s 2023-10-17T01:23:21.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:20 smithi181 ceph-mon[134520]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 MiB/s wr, 1.60k op/s 2023-10-17T01:23:23.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:22 smithi045 ceph-mon[98474]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-10-17T01:23:23.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:22 smithi060 ceph-mon[134136]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-10-17T01:23:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:22 smithi181 ceph-mon[134520]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-10-17T01:23:25.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:24 smithi045 ceph-mon[98474]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-10-17T01:23:25.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:24 smithi060 ceph-mon[134136]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-10-17T01:23:25.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:24 smithi181 ceph-mon[134520]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-10-17T01:23:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 495.0 s, 1565.1 tps, lat 63.453 ms stddev 75.115 2023-10-17T01:23:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:26 smithi045 ceph-mon[98474]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:23:27.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:26 smithi060 ceph-mon[134136]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:23:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:26 smithi181 ceph-mon[134520]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-10-17T01:23:28.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:28 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:23:28.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:28 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:23:28.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:28 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:23:29.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:29 smithi045 ceph-mon[98474]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.82k op/s 2023-10-17T01:23:29.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:29 smithi060 ceph-mon[134136]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.82k op/s 2023-10-17T01:23:29.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:29 smithi181 ceph-mon[134520]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.82k op/s 2023-10-17T01:23:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 500.0 s, 1705.8 tps, lat 59.179 ms stddev 64.238 2023-10-17T01:23:31.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:23:31.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:23:31.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:23:31.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:30 smithi045 ceph-mon[98474]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2023-10-17T01:23:31.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:23:31.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:23:31.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:23:31.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:30 smithi060 ceph-mon[134136]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2023-10-17T01:23:31.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:23:31.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:23:31.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:23:31.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:30 smithi181 ceph-mon[134520]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.73k op/s 2023-10-17T01:23:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.661+0000 7f8e31fff700 1 -- 172.21.15.45:0/3181634482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c101f70 msgr2=0x7f8e2c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.661+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3181634482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c101f70 0x7f8e2c1023d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009990 tx=0x7f8e20009ca0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.662+0000 7f8e31fff700 1 -- 172.21.15.45:0/3181634482 shutdown_connections 2023-10-17T01:23:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.662+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3181634482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c102910 0x7f8e2c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.662+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3181634482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c100d80 0x7f8e2c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.662+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3181634482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c101f70 0x7f8e2c1023d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.662+0000 7f8e31fff700 1 -- 172.21.15.45:0/3181634482 >> 172.21.15.45:0/3181634482 conn(0x7f8e2c0fc2d0 msgr2=0x7f8e2c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.663+0000 7f8e31fff700 1 -- 172.21.15.45:0/3181634482 shutdown_connections 2023-10-17T01:23:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.663+0000 7f8e31fff700 1 -- 172.21.15.45:0/3181634482 wait complete. 2023-10-17T01:23:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 Processor -- start 2023-10-17T01:23:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 -- start start 2023-10-17T01:23:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c100d80 0x7f8e2c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c101f70 0x7f8e2c073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c102910 0x7f8e2c073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1a0ae0 con 0x7f8e2c101f70 2023-10-17T01:23:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1a0c20 con 0x7f8e2c100d80 2023-10-17T01:23:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e31fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1a0d60 con 0x7f8e2c102910 2023-10-17T01:23:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e2affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c101f70 0x7f8e2c073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c100d80 0x7f8e2c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.665+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c100d80 0x7f8e2c073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53320/0 (socket says 172.21.15.45:53320) 2023-10-17T01:23:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/3316483149 learned_addr learned my addr 172.21.15.45:0/3316483149 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2bfff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c102910 0x7f8e2c073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2affd700 1 -- 172.21.15.45:0/3316483149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c102910 msgr2=0x7f8e2c073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2affd700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c102910 0x7f8e2c073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2affd700 1 -- 172.21.15.45:0/3316483149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c100d80 msgr2=0x7f8e2c073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2affd700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c100d80 0x7f8e2c073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2affd700 1 -- 172.21.15.45:0/3316483149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e200095f0 con 0x7f8e2c101f70 2023-10-17T01:23:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.666+0000 7f8e2affd700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c101f70 0x7f8e2c073680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b680 tx=0x7f8e20009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.667+0000 7f8e28ff9700 1 -- 172.21.15.45:0/3316483149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2001d070 con 0x7f8e2c101f70 2023-10-17T01:23:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.667+0000 7f8e28ff9700 1 -- 172.21.15.45:0/3316483149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e20021410 con 0x7f8e2c101f70 2023-10-17T01:23:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.667+0000 7f8e28ff9700 1 -- 172.21.15.45:0/3316483149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20017650 con 0x7f8e2c101f70 2023-10-17T01:23:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.668+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c1a0f90 con 0x7f8e2c101f70 2023-10-17T01:23:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.668+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c1a1300 con 0x7f8e2c101f70 2023-10-17T01:23:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.672+0000 7f8e28ff9700 1 -- 172.21.15.45:0/3316483149 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8e20021810 con 0x7f8e2c101f70 2023-10-17T01:23:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.673+0000 7f8e28ff9700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8e140a0c70 0x7f8e140a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.674+0000 7f8e28ff9700 1 -- 172.21.15.45:0/3316483149 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8e200cd420 con 0x7f8e2c101f70 2023-10-17T01:23:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.676+0000 7f8e45892b80 1 Processor -- start 2023-10-17T01:23:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 -- start start 2023-10-17T01:23:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8e140a0c70 0x7f8e140a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a856b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a856049b0 con 0x556a856b59a0 2023-10-17T01:23:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85602370 con 0x556a856b5040 2023-10-17T01:23:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.677+0000 7f8e45892b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a855fdc60 con 0x556a856b65a0 2023-10-17T01:23:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.682+0000 7f8e2bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.682+0000 7f8e2bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42030/0 (socket says 172.21.15.45:42030) 2023-10-17T01:23:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.682+0000 7f8e2bfff700 1 -- 172.21.15.45:0/1035177208 learned_addr learned my addr 172.21.15.45:0/1035177208 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.684+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8e140a0c70 0x7f8e140a3130 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c074a50 tx=0x7f8e1c0099a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.684+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a856b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.685+0000 7f8e2affd700 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.686+0000 7f8e2affd700 1 -- 172.21.15.45:0/1035177208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 msgr2=0x556a856b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.686+0000 7f8e2affd700 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.686+0000 7f8e2affd700 1 -- 172.21.15.45:0/1035177208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 msgr2=0x556a856b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.686+0000 7f8e2affd700 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a856b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.686+0000 7f8e2affd700 1 -- 172.21.15.45:0/1035177208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a8542bd80 con 0x556a856b59a0 2023-10-17T01:23:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.686+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a856b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.687+0000 7f8e2affd700 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b5fa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8e200cb2c0 tx=0x7f8e2009aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def6beb136eb6901 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.693+0000 7f8e11ffb700 1 -- 172.21.15.45:0/1035177208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2009ab30 con 0x556a856b59a0 2023-10-17T01:23:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.693+0000 7f8e11ffb700 1 -- 172.21.15.45:0/1035177208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e200b8e90 con 0x556a856b59a0 2023-10-17T01:23:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.693+0000 7f8e11ffb700 1 -- 172.21.15.45:0/1035177208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e200cd0f0 con 0x556a856b59a0 2023-10-17T01:23:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.697+0000 7f8e45892b80 1 -- 172.21.15.45:0/1035177208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 msgr2=0x556a856b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.697+0000 7f8e45892b80 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b5fa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8e200cb2c0 tx=0x7f8e2009aa20 comp rx=0 tx=0).stop 2023-10-17T01:23:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 -- 172.21.15.45:0/1035177208 shutdown_connections 2023-10-17T01:23:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a856b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 --2- 172.21.15.45:0/1035177208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b5fa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 -- 172.21.15.45:0/1035177208 >> 172.21.15.45:0/1035177208 conn(0x556a8560c070 msgr2=0x556a856a72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 -- 172.21.15.45:0/1035177208 shutdown_connections 2023-10-17T01:23:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.698+0000 7f8e45892b80 1 -- 172.21.15.45:0/1035177208 wait complete. 2023-10-17T01:23:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.699+0000 7f8e45892b80 1 -- start start 2023-10-17T01:23:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.706+0000 7f8e45892b80 1 Processor -- start 2023-10-17T01:23:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 0x556a856ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b59a0 0x556a856adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8559a4b0 con 0x556a856b5040 2023-10-17T01:23:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8559a150 con 0x556a856b59a0 2023-10-17T01:23:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a854818c0 con 0x556a856b65a0 2023-10-17T01:23:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.707+0000 7f8e45892b80 10 client.0 _ll_register_callbacks cb 0x556a854e7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.710+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 0x556a856ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.710+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 0x556a856ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38208/0 (socket says 172.21.15.45:38208) 2023-10-17T01:23:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.710+0000 7f8e2bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.710+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/480481665 learned_addr learned my addr 172.21.15.45:0/480481665 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.711+0000 7f8e2affd700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b59a0 0x556a856adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.712+0000 7f8e2bfff700 1 -- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b59a0 msgr2=0x556a856adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.712+0000 7f8e2bfff700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b59a0 0x556a856adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.712+0000 7f8e2bfff700 1 -- 172.21.15.45:0/480481665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 msgr2=0x556a856ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.712+0000 7f8e2bfff700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 0x556a856ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.712+0000 7f8e2bfff700 1 -- 172.21.15.45:0/480481665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c019050 con 0x556a856b65a0 2023-10-17T01:23:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.712+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 0x556a856ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.715+0000 7f8e0affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:23:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.716+0000 7f8e2bfff700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b2fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8e24007bd0 tx=0x7f8e24007cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f971e1026113243a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.717+0000 7f8e0affd700 1 -- 172.21.15.45:0/480481665 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400e040 con 0x556a856b65a0 2023-10-17T01:23:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.717+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a85628cb0 con 0x556a856b65a0 2023-10-17T01:23:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.717+0000 7f8e0affd700 1 -- 172.21.15.45:0/480481665 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8e24003c10 con 0x556a856b65a0 2023-10-17T01:23:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.718+0000 7f8e0affd700 1 -- 172.21.15.45:0/480481665 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e24007e90 con 0x556a856b65a0 2023-10-17T01:23:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.718+0000 7f8e45892b80 10 client.26356 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.718+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f8e2c1a1300 con 0x556a856b65a0 2023-10-17T01:23:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.726+0000 7f8e0affd700 1 -- 172.21.15.45:0/480481665 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a85628cb0 con 0x556a856b65a0 2023-10-17T01:23:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.726+0000 7f8e0affd700 1 -- 172.21.15.45:0/480481665 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8e24004270 con 0x556a856b65a0 2023-10-17T01:23:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.727+0000 7f8e45892b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.727+0000 7f8e45892b80 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.727+0000 7f8e45892b80 20 client.26356 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.727+0000 7f8e45892b80 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a856d2d80 0x556a856d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.727+0000 7f8e45892b80 4 client.26356 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.727+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a8567bcd0 con 0x556a856d2d80 2023-10-17T01:23:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.731+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a856d2d80 0x556a856d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.739+0000 7f8e0affd700 10 client.26356 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.739+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a856d2d80 0x556a856d51a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f22fec7a8992bb server_cookie=cd0b1ea1f5e723a4 in_seq=0 out_seq=0 2023-10-17T01:23:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.741+0000 7f8e0affd700 1 -- 172.21.15.45:0/480481665 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x556a8567bcd0 con 0x556a856d2d80 2023-10-17T01:23:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.741+0000 7f8e0affd700 10 client.26356 handle_command_reply: tid=0 2023-10-17T01:23:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.741+0000 7f8e45892b80 1 client.26356 shutdown 2023-10-17T01:23:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 msgr2=0x556a856b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b2fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8e24007bd0 tx=0x7f8e24007cb0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 shutdown_connections 2023-10-17T01:23:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b65a0 0x556a856b2fd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b5040 0x556a856ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a856d2d80 0x556a856d51a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 --2- 172.21.15.45:0/480481665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b59a0 0x556a856adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e0affd700 0 client.26356 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 >> 172.21.15.45:0/480481665 conn(0x556a8560c070 msgr2=0x556a856a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 shutdown_connections 2023-10-17T01:23:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.742+0000 7f8e45892b80 1 -- 172.21.15.45:0/480481665 wait complete. 2023-10-17T01:23:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.743+0000 7f8e45892b80 20 client.26356 trim_cache size 0 max 16384 2023-10-17T01:23:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.768+0000 7f8e45892b80 1 Processor -- start 2023-10-17T01:23:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 -- start start 2023-10-17T01:23:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 0x556a856d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b65a0 0x556a856bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a856a7de0 con 0x556a856b59a0 2023-10-17T01:23:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a856acf40 con 0x556a856b65a0 2023-10-17T01:23:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e45892b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a856ad5d0 con 0x556a856b5040 2023-10-17T01:23:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 0x556a856d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 0x556a856d51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42048/0 (socket says 172.21.15.45:42048) 2023-10-17T01:23:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/920861925 learned_addr learned my addr 172.21.15.45:0/920861925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.769+0000 7f8e2affd700 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2bfff700 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b65a0 0x556a856bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/920861925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b65a0 msgr2=0x556a856bc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b65a0 0x556a856bc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/920861925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 msgr2=0x556a856b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/920861925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a85414c00 con 0x556a856b5040 2023-10-17T01:23:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 0x556a856d51a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556a856d5ab0 tx=0x7f8e1c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c638439016db1d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e127fc700 1 -- 172.21.15.45:0/920861925 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c023040 con 0x556a856b5040 2023-10-17T01:23:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e127fc700 1 -- 172.21.15.45:0/920861925 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1c0122d0 con 0x556a856b5040 2023-10-17T01:23:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.770+0000 7f8e127fc700 1 -- 172.21.15.45:0/920861925 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c010690 con 0x556a856b5040 2023-10-17T01:23:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 -- 172.21.15.45:0/920861925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 msgr2=0x556a856d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 0x556a856d51a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556a856d5ab0 tx=0x7f8e1c016000 comp rx=0 tx=0).stop 2023-10-17T01:23:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 -- 172.21.15.45:0/920861925 shutdown_connections 2023-10-17T01:23:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b65a0 0x556a856bc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b5040 0x556a856d51a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 --2- 172.21.15.45:0/920861925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b59a0 0x556a856b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 -- 172.21.15.45:0/920861925 >> 172.21.15.45:0/920861925 conn(0x556a8560c070 msgr2=0x556a856ac760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 -- 172.21.15.45:0/920861925 shutdown_connections 2023-10-17T01:23:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.771+0000 7f8e45892b80 1 -- 172.21.15.45:0/920861925 wait complete. 2023-10-17T01:23:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 -- start start 2023-10-17T01:23:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 Processor -- start 2023-10-17T01:23:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a8571e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b59a0 0x556a85720c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b65a0 0x556a857231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a857238f0 con 0x556a856b65a0 2023-10-17T01:23:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85723a60 con 0x556a856b5040 2023-10-17T01:23:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.772+0000 7f8e45892b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a85723c20 con 0x556a856b59a0 2023-10-17T01:23:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e45892b80 10 client.0 _ll_register_callbacks cb 0x7f8e240093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a8571e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a8571e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53352/0 (socket says 172.21.15.45:53352) 2023-10-17T01:23:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2b7fe700 1 -- 172.21.15.45:0/2154208695 learned_addr learned my addr 172.21.15.45:0/2154208695 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b59a0 0x556a85720c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2bfff700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b65a0 0x556a857231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 -- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 msgr2=0x556a8571e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a8571e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 -- 172.21.15.45:0/2154208695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b65a0 msgr2=0x556a857231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b65a0 0x556a857231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 -- 172.21.15.45:0/2154208695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c004460 con 0x556a856b59a0 2023-10-17T01:23:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e2affd700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b59a0 0x556a85720c30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8e2009eb20 tx=0x7f8e20009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa7817f2bb06cd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.773+0000 7f8e0affd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:23:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.774+0000 7f8e0affd700 1 -- 172.21.15.45:0/2154208695 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2009e960 con 0x556a856b59a0 2023-10-17T01:23:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.774+0000 7f8e0affd700 1 -- 172.21.15.45:0/2154208695 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8e2009ab30 con 0x556a856b59a0 2023-10-17T01:23:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.774+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a855f6ce0 con 0x556a856b59a0 2023-10-17T01:23:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.774+0000 7f8e0affd700 1 -- 172.21.15.45:0/2154208695 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e200a2b10 con 0x556a856b59a0 2023-10-17T01:23:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e45892b80 10 client.26362 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556a854521f0 con 0x556a856b59a0 2023-10-17T01:23:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e0affd700 1 -- 172.21.15.45:0/2154208695 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a855f6ce0 con 0x556a856b59a0 2023-10-17T01:23:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e0affd700 1 -- 172.21.15.45:0/2154208695 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8e200a2680 con 0x556a856b59a0 2023-10-17T01:23:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e45892b80 10 client.26362 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e45892b80 10 client.26362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.775+0000 7f8e45892b80 20 client.26362 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.776+0000 7f8e45892b80 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a857251f0 0x556a85727610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.776+0000 7f8e45892b80 4 client.26362 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:23:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.776+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a855f6a00 con 0x556a857251f0 2023-10-17T01:23:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.776+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a857251f0 0x556a85727610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.776+0000 7f8e2b7fe700 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a857251f0 0x556a85727610 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1267b2aee269881e server_cookie=39c37e4dbc252b19 in_seq=0 out_seq=0 2023-10-17T01:23:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.776+0000 7f8e0affd700 10 client.26362 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e0affd700 1 -- 172.21.15.45:0/2154208695 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a855f6a00 con 0x556a857251f0 2023-10-17T01:23:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e0affd700 10 client.26362 handle_command_reply: tid=0 2023-10-17T01:23:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 client.26362 shutdown 2023-10-17T01:23:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b59a0 msgr2=0x556a85720c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b59a0 0x556a85720c30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8e2009eb20 tx=0x7f8e20009ca0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 shutdown_connections 2023-10-17T01:23:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556a856b65a0 0x556a857231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x556a857251f0 0x556a85727610 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a856b5040 0x556a8571e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 --2- 172.21.15.45:0/2154208695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a856b59a0 0x556a85720c30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 >> 172.21.15.45:0/2154208695 conn(0x556a8560c070 msgr2=0x556a856bc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e0affd700 0 client.26362 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 shutdown_connections 2023-10-17T01:23:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.791+0000 7f8e45892b80 1 -- 172.21.15.45:0/2154208695 wait complete. 2023-10-17T01:23:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e45892b80 20 client.26362 trim_cache size 0 max 16384 2023-10-17T01:23:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:23:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:23:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:23:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:23:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8e140a0c70 msgr2=0x7f8e140a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8e140a0c70 0x7f8e140a3130 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c074a50 tx=0x7f8e1c0099a0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c101f70 msgr2=0x7f8e2c073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c101f70 0x7f8e2c073680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b680 tx=0x7f8e20009580 comp rx=0 tx=0).stop 2023-10-17T01:23:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 shutdown_connections 2023-10-17T01:23:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8e140a0c70 0x7f8e140a3130 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e2c102910 0x7f8e2c073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c100d80 0x7f8e2c073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 --2- 172.21.15.45:0/3316483149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2c101f70 0x7f8e2c073680 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.792+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 >> 172.21.15.45:0/3316483149 conn(0x7f8e2c0fc2d0 msgr2=0x7f8e2c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.793+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 shutdown_connections 2023-10-17T01:23:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:31.794+0000 7f8e31fff700 1 -- 172.21.15.45:0/3316483149 wait complete. 2023-10-17T01:23:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:32 smithi045 ceph-mon[98474]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 948 op/s 2023-10-17T01:23:33.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:32 smithi060 ceph-mon[134136]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 948 op/s 2023-10-17T01:23:33.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:32 smithi181 ceph-mon[134520]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 948 op/s 2023-10-17T01:23:34.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:23:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 -- 172.21.15.45:0/1611827782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa708101f20 msgr2=0x7fa7081043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/1611827782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa708101f20 0x7fa7081043b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009990 tx=0x7fa6f0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 -- 172.21.15.45:0/1611827782 shutdown_connections 2023-10-17T01:23:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/1611827782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7081048f0 0x7fa708104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/1611827782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7080ff5b0 0x7fa7081019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/1611827782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa708101f20 0x7fa7081043b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.643+0000 7fa70d5ff700 1 -- 172.21.15.45:0/1611827782 >> 172.21.15.45:0/1611827782 conn(0x7fa7080fb1b0 msgr2=0x7fa7080fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.644+0000 7fa70d5ff700 1 -- 172.21.15.45:0/1611827782 shutdown_connections 2023-10-17T01:23:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.644+0000 7fa70d5ff700 1 -- 172.21.15.45:0/1611827782 wait complete. 2023-10-17T01:23:34.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.645+0000 7fa70d5ff700 1 Processor -- start 2023-10-17T01:23:34.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 -- start start 2023-10-17T01:23:34.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7080ff5b0 0x7fa70819d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 0x7fa70819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7081048f0 0x7fa7081a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7081a4c20 con 0x7fa7080ff5b0 2023-10-17T01:23:34.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7081a4d60 con 0x7fa708101f20 2023-10-17T01:23:34.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa70d5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7081a4ef0 con 0x7fa7081048f0 2023-10-17T01:23:34.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 0x7fa70819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.646+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 0x7fa70819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53370/0 (socket says 172.21.15.45:53370) 2023-10-17T01:23:34.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/4214525293 learned_addr learned my addr 172.21.15.45:0/4214525293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:34.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa706ffd700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7080ff5b0 0x7fa70819d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/4214525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7081048f0 msgr2=0x7fa7081a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-17T01:23:34.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7081048f0 0x7fa7081a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7080ff5b0 msgr2=0x7fa70819d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7080ff5b0 0x7fa70819d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/4214525293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00095f0 con 0x7fa708101f20 2023-10-17T01:23:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.647+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 0x7fa70819d5e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0004730 tx=0x7fa6f0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.648+0000 7fa704ff9700 1 -- 172.21.15.45:0/4214525293 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f001d070 con 0x7fa708101f20 2023-10-17T01:23:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.648+0000 7fa704ff9700 1 -- 172.21.15.45:0/4214525293 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6f0004cd0 con 0x7fa708101f20 2023-10-17T01:23:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.648+0000 7fa704ff9700 1 -- 172.21.15.45:0/4214525293 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00055d0 con 0x7fa708101f20 2023-10-17T01:23:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.649+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7081a5170 con 0x7fa708101f20 2023-10-17T01:23:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.650+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7081a54e0 con 0x7fa708101f20 2023-10-17T01:23:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.656+0000 7fa704ff9700 1 -- 172.21.15.45:0/4214525293 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa6f000fc70 con 0x7fa708101f20 2023-10-17T01:23:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.662+0000 7fa720edfb80 1 Processor -- start 2023-10-17T01:23:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.662+0000 7fa720edfb80 1 -- start start 2023-10-17T01:23:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.663+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c416ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.663+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4170d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.663+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4171730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.663+0000 7fa720edfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c40c7a10 con 0x5565c4170500 2023-10-17T01:23:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.663+0000 7fa720edfb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c40c53d0 con 0x5565c416fba0 2023-10-17T01:23:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.663+0000 7fa720edfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c40c0cc0 con 0x5565c41712b0 2023-10-17T01:23:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4171730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4171730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42070/0 (socket says 172.21.15.45:42070) 2023-10-17T01:23:34.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/3845061548 learned_addr learned my addr 172.21.15.45:0/3845061548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa7077fe700 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c416ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa706ffd700 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4170d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa7077fe700 1 -- 172.21.15.45:0/3845061548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 msgr2=0x5565c4171730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa7077fe700 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4171730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa7077fe700 1 -- 172.21.15.45:0/3845061548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 msgr2=0x5565c4170d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa7077fe700 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4170d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.664+0000 7fa7077fe700 1 -- 172.21.15.45:0/3845061548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565c3eeede0 con 0x5565c416fba0 2023-10-17T01:23:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa7077fe700 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c416ffc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b60 tx=0x7fa700009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8a102da4314dd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa704ff9700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f40a0ba0 0x7fa6f40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa6fd7fa700 1 -- 172.21.15.45:0/3845061548 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700004230 con 0x5565c416fba0 2023-10-17T01:23:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa704ff9700 1 -- 172.21.15.45:0/4214525293 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa6f00cd640 con 0x7fa708101f20 2023-10-17T01:23:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa6fd7fa700 1 -- 172.21.15.45:0/3845061548 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70000ed40 con 0x5565c416fba0 2023-10-17T01:23:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa6fd7fa700 1 -- 172.21.15.45:0/3845061548 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700005020 con 0x5565c416fba0 2023-10-17T01:23:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.665+0000 7fa706ffd700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f40a0ba0 0x7fa6f40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.667+0000 7fa720edfb80 1 -- 172.21.15.45:0/3845061548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 msgr2=0x5565c416ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.667+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c416ffc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b60 tx=0x7fa700009e70 comp rx=0 tx=0).stop 2023-10-17T01:23:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.668+0000 7fa706ffd700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f40a0ba0 0x7fa6f40a3060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5565c4170900 tx=0x7fa6f800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.668+0000 7fa720edfb80 1 -- 172.21.15.45:0/3845061548 shutdown_connections 2023-10-17T01:23:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.668+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4171730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.668+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c416ffc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.668+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3845061548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4170d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.668+0000 7fa720edfb80 1 -- 172.21.15.45:0/3845061548 >> 172.21.15.45:0/3845061548 conn(0x5565c40cf0d0 msgr2=0x5565c416a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.669+0000 7fa720edfb80 1 -- 172.21.15.45:0/3845061548 shutdown_connections 2023-10-17T01:23:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.669+0000 7fa720edfb80 1 -- 172.21.15.45:0/3845061548 wait complete. 2023-10-17T01:23:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.670+0000 7fa720edfb80 1 -- start start 2023-10-17T01:23:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.671+0000 7fa720edfb80 1 Processor -- start 2023-10-17T01:23:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.671+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c4179960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4179ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4180ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa706ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c4179960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa706ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c4179960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53386/0 (socket says 172.21.15.45:53386) 2023-10-17T01:23:34.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa720edfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c405d510 con 0x5565c4170500 2023-10-17T01:23:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa720edfb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c405d1b0 con 0x5565c416fba0 2023-10-17T01:23:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4179ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa706ffd700 1 -- 172.21.15.45:0/3274085088 learned_addr learned my addr 172.21.15.45:0/3274085088 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c3f44920 con 0x5565c41712b0 2023-10-17T01:23:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa720edfb80 10 client.0 _ll_register_callbacks cb 0x5565c3faaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.672+0000 7fa7077fe700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4180ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/3274085088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 msgr2=0x5565c4180ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4180ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 msgr2=0x5565c4179960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c4179960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/3274085088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa700009810 con 0x5565c4170500 2023-10-17T01:23:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa7077fe700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4180ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4179ea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00a27a0 tx=0x7fa6f0005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9f8365125b76d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.673+0000 7fa6ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:23:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/3274085088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f009aac0 con 0x5565c4170500 2023-10-17T01:23:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/3274085088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa6f009b3f0 con 0x5565c4170500 2023-10-17T01:23:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565c40ebd10 con 0x5565c4170500 2023-10-17T01:23:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/3274085088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f009a830 con 0x5565c4170500 2023-10-17T01:23:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa720edfb80 10 client.16125 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5565c3ed7c60 con 0x5565c4170500 2023-10-17T01:23:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.674+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/3274085088 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6f00b2070 con 0x5565c4170500 2023-10-17T01:23:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/3274085088 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa6f00a2840 con 0x5565c4170500 2023-10-17T01:23:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa720edfb80 10 client.16125 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa720edfb80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa720edfb80 20 client.16125 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c4185380 0x5565c41877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa720edfb80 4 client.16125 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565c413ed30 con 0x5565c4185380 2023-10-17T01:23:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.675+0000 7fa706ffd700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c4185380 0x5565c41877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.676+0000 7fa6ee7fc700 10 client.16125 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.676+0000 7fa706ffd700 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c4185380 0x5565c41877a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751ff110b399224d server_cookie=dc75e7d78fbbff11 in_seq=0 out_seq=0 2023-10-17T01:23:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.677+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/3274085088 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5565c413ed30 con 0x5565c4185380 2023-10-17T01:23:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.677+0000 7fa6ee7fc700 10 client.16125 handle_command_reply: tid=0 2023-10-17T01:23:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.677+0000 7fa720edfb80 1 client.16125 shutdown 2023-10-17T01:23:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.678+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 msgr2=0x5565c4179ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.678+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4179ea0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00a27a0 tx=0x7fa6f0005a90 comp rx=0 tx=0).stop 2023-10-17T01:23:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 shutdown_connections 2023-10-17T01:23:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c41712b0 0x5565c4180ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c4185380 0x5565c41877a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c416fba0 0x5565c4179960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 --2- 172.21.15.45:0/3274085088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c4170500 0x5565c4179ea0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa6ee7fc700 0 client.16125 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 >> 172.21.15.45:0/3274085088 conn(0x5565c40cf0d0 msgr2=0x5565c416a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.679+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 shutdown_connections 2023-10-17T01:23:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.680+0000 7fa720edfb80 1 -- 172.21.15.45:0/3274085088 wait complete. 2023-10-17T01:23:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.680+0000 7fa720edfb80 20 client.16125 trim_cache size 0 max 16384 2023-10-17T01:23:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 Processor -- start 2023-10-17T01:23:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 -- start start 2023-10-17T01:23:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c4178620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c416a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c4177760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c41794d0 con 0x5565c416fba0 2023-10-17T01:23:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c417a5e0 con 0x5565c41712b0 2023-10-17T01:23:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.710+0000 7fa720edfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4167420 con 0x5565c417edf0 2023-10-17T01:23:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.711+0000 7fa7077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c4177760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.711+0000 7fa7077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c4177760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42080/0 (socket says 172.21.15.45:42080) 2023-10-17T01:23:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.711+0000 7fa7077fe700 1 -- 172.21.15.45:0/1123040450 learned_addr learned my addr 172.21.15.45:0/1123040450 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c416a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa7077fe700 1 -- 172.21.15.45:0/1123040450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 msgr2=0x5565c416a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa7077fe700 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c416a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa7077fe700 1 -- 172.21.15.45:0/1123040450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 msgr2=0x5565c4178620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa706ffd700 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c4178620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa7077fe700 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c4178620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa7077fe700 1 -- 172.21.15.45:0/1123040450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565c3f15250 con 0x5565c417edf0 2023-10-17T01:23:34.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.712+0000 7fa706ffd700 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c4178620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:23:34.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.713+0000 7fa7077fe700 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c4177760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b00 tx=0x7fa700005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dabb4f0d13af6ef server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:34.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.713+0000 7fa6fdffb700 1 -- 172.21.15.45:0/1123040450 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700004710 con 0x5565c417edf0 2023-10-17T01:23:34.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.713+0000 7fa6fdffb700 1 -- 172.21.15.45:0/1123040450 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa700004870 con 0x5565c417edf0 2023-10-17T01:23:34.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.713+0000 7fa6fdffb700 1 -- 172.21.15.45:0/1123040450 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700018470 con 0x5565c417edf0 2023-10-17T01:23:34.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 -- 172.21.15.45:0/1123040450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 msgr2=0x5565c4177760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c4177760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b00 tx=0x7fa700005dc0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 -- 172.21.15.45:0/1123040450 shutdown_connections 2023-10-17T01:23:34.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c4177760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c4178620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 --2- 172.21.15.45:0/1123040450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c416a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 -- 172.21.15.45:0/1123040450 >> 172.21.15.45:0/1123040450 conn(0x5565c40cf0d0 msgr2=0x5565c416a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:34.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 -- 172.21.15.45:0/1123040450 shutdown_connections 2023-10-17T01:23:34.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 -- 172.21.15.45:0/1123040450 wait complete. 2023-10-17T01:23:34.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.714+0000 7fa720edfb80 1 -- start start 2023-10-17T01:23:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 Processor -- start 2023-10-17T01:23:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c41cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c41cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c41d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c41d68f0 con 0x5565c416fba0 2023-10-17T01:23:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c41d6a30 con 0x5565c41712b0 2023-10-17T01:23:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c41d6bc0 con 0x5565c417edf0 2023-10-17T01:23:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa720edfb80 10 client.0 _ll_register_callbacks cb 0x5565c3faaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c41cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.715+0000 7fa6fe7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c41cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:53406/0 (socket says 172.21.15.45:53406) 2023-10-17T01:23:34.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/654337810 learned_addr learned my addr 172.21.15.45:0/654337810 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:34.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa706ffd700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c41cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa7077fe700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c41d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/654337810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 msgr2=0x5565c41d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c41d63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/654337810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 msgr2=0x5565c41cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c41cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.716+0000 7fa6fe7fc700 1 -- 172.21.15.45:0/654337810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70001f040 con 0x5565c41712b0 2023-10-17T01:23:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.717+0000 7fa6fe7fc700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c41cf460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000b2e0 tx=0x7fa6f009aac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17812f6b669c1156 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.717+0000 7fa6ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:23:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.717+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/654337810 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00d0c00 con 0x5565c41712b0 2023-10-17T01:23:34.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.717+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/654337810 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa6f009b3f0 con 0x5565c41712b0 2023-10-17T01:23:34.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.717+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa70000ed40 con 0x5565c41712b0 2023-10-17T01:23:34.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.718+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/654337810 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f00a2cd0 con 0x5565c41712b0 2023-10-17T01:23:34.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.718+0000 7fa720edfb80 10 client.26414 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:34.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.718+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5565c3eed2b0 con 0x5565c41712b0 2023-10-17T01:23:34.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.718+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/654337810 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70000ed40 con 0x5565c41712b0 2023-10-17T01:23:34.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.718+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/654337810 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa6f00cd2f0 con 0x5565c41712b0 2023-10-17T01:23:34.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.718+0000 7fa720edfb80 10 client.26414 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:34.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.719+0000 7fa720edfb80 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:34.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.719+0000 7fa720edfb80 20 client.26414 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:34.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.719+0000 7fa720edfb80 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c41d7f50 0x5565c41da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:34.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.719+0000 7fa720edfb80 4 client.26414 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:23:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.719+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5565c40b9d40 con 0x5565c41d7f50 2023-10-17T01:23:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.719+0000 7fa706ffd700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c41d7f50 0x5565c41da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa706ffd700 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c41d7f50 0x5565c41da370 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965934a49bbbcad0 server_cookie=1d49ce80bc0213fc in_seq=0 out_seq=0 2023-10-17T01:23:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa6ee7fc700 10 client.26414 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa6ee7fc700 1 -- 172.21.15.45:0/654337810 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5565c40b9d40 con 0x5565c41d7f50 2023-10-17T01:23:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa6ee7fc700 10 client.26414 handle_command_reply: tid=0 2023-10-17T01:23:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 client.26414 shutdown 2023-10-17T01:23:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 msgr2=0x5565c41cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c41cf460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000b2e0 tx=0x7fa6f009aac0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 shutdown_connections 2023-10-17T01:23:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5565c417edf0 0x5565c41d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5565c416fba0 0x5565c41cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.720+0000 7fa720edfb80 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5565c41d7f50 0x5565c41da370 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.721+0000 7fa6ee7fc700 0 client.26414 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.721+0000 7fa720edfb80 1 --2- 172.21.15.45:0/654337810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5565c41712b0 0x5565c41cf460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.721+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 >> 172.21.15.45:0/654337810 conn(0x5565c40cf0d0 msgr2=0x5565c4177760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:34.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.721+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 shutdown_connections 2023-10-17T01:23:34.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.721+0000 7fa720edfb80 1 -- 172.21.15.45:0/654337810 wait complete. 2023-10-17T01:23:34.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.721+0000 7fa720edfb80 20 client.26414 trim_cache size 0 max 16384 2023-10-17T01:23:34.731 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:23:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:23:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:23:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:23:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:23:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:23:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:23:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:23:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:23:34.733 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:23:34.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f40a0ba0 msgr2=0x7fa6f40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f40a0ba0 0x7fa6f40a3060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5565c4170900 tx=0x7fa6f800b040 comp rx=0 tx=0).stop 2023-10-17T01:23:34.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 msgr2=0x7fa70819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:34.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 0x7fa70819d5e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0004730 tx=0x7fa6f0005ef0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 shutdown_connections 2023-10-17T01:23:34.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa6f40a0ba0 0x7fa6f40a3060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7081048f0 0x7fa7081a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7080ff5b0 0x7fa70819d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 --2- 172.21.15.45:0/4214525293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa708101f20 0x7fa70819d5e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:34.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.723+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 >> 172.21.15.45:0/4214525293 conn(0x7fa7080fb1b0 msgr2=0x7fa708100260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:34.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.724+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 shutdown_connections 2023-10-17T01:23:34.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:23:34.724+0000 7fa70d5ff700 1 -- 172.21.15.45:0/4214525293 wait complete. 2023-10-17T01:23:34.751 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:23:35.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:34 smithi045 ceph-mon[98474]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 632 op/s 2023-10-17T01:23:35.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:34 smithi060 ceph-mon[134136]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 632 op/s 2023-10-17T01:23:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:34 smithi181 ceph-mon[134520]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 632 op/s 2023-10-17T01:23:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 505.0 s, 1769.8 tps, lat 56.197 ms stddev 73.354 2023-10-17T01:23:37.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:36 smithi045 ceph-mon[98474]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 587 op/s 2023-10-17T01:23:37.086 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:36 smithi060 ceph-mon[134136]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 587 op/s 2023-10-17T01:23:37.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:36 smithi181 ceph-mon[134520]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 587 op/s 2023-10-17T01:23:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:38 smithi045 ceph-mon[98474]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 633 op/s 2023-10-17T01:23:39.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:38 smithi060 ceph-mon[134136]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 633 op/s 2023-10-17T01:23:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:38 smithi181 ceph-mon[134520]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 633 op/s 2023-10-17T01:23:40.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 510.0 s, 1652.9 tps, lat 60.632 ms stddev 77.246 2023-10-17T01:23:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:40 smithi045 ceph-mon[98474]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 684 op/s 2023-10-17T01:23:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:40 smithi060 ceph-mon[134136]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 684 op/s 2023-10-17T01:23:41.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:40 smithi181 ceph-mon[134520]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 684 op/s 2023-10-17T01:23:43.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:42 smithi045 ceph-mon[98474]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 647 op/s 2023-10-17T01:23:43.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:42 smithi060 ceph-mon[134136]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 647 op/s 2023-10-17T01:23:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:42 smithi181 ceph-mon[134520]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 647 op/s 2023-10-17T01:23:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.030+0000 7fef0f5ff700 1 -- 172.21.15.45:0/1377827407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef08108f40 msgr2=0x7fef0810f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.030+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/1377827407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef08108f40 0x7fef0810f7f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef0000b0a0 tx=0x7fef0000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 -- 172.21.15.45:0/1377827407 shutdown_connections 2023-10-17T01:23:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/1377827407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef08108f40 0x7fef0810f7f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/1377827407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef081073b0 0x7fef081077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/1377827407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef081085a0 0x7fef08108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 -- 172.21.15.45:0/1377827407 >> 172.21.15.45:0/1377827407 conn(0x7fef08076330 msgr2=0x7fef08078790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 -- 172.21.15.45:0/1377827407 shutdown_connections 2023-10-17T01:23:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.031+0000 7fef0f5ff700 1 -- 172.21.15.45:0/1377827407 wait complete. 2023-10-17T01:23:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.032+0000 7fef0f5ff700 1 Processor -- start 2023-10-17T01:23:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.032+0000 7fef0f5ff700 1 -- start start 2023-10-17T01:23:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0f5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef081073b0 0x7fef0819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0f5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef081085a0 0x7fef0819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0f5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 0x7fef0819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0f5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0819ed10 con 0x7fef081073b0 2023-10-17T01:23:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0f5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0819ee80 con 0x7fef08108f40 2023-10-17T01:23:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0f5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0819f040 con 0x7fef081085a0 2023-10-17T01:23:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.033+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef081073b0 0x7fef0819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0cb9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef081085a0 0x7fef0819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 0x7fef0819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 0x7fef0819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41976/0 (socket says 172.21.15.45:41976) 2023-10-17T01:23:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 -- 172.21.15.45:0/3186272255 learned_addr learned my addr 172.21.15.45:0/3186272255 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 -- 172.21.15.45:0/3186272255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef081085a0 msgr2=0x7fef0819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef081085a0 0x7fef0819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 -- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef081073b0 msgr2=0x7fef0819d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef081073b0 0x7fef0819d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.034+0000 7fef0db9c700 1 -- 172.21.15.45:0/3186272255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00009d00 con 0x7fef08108f40 2023-10-17T01:23:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.035+0000 7fef0db9c700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 0x7fef0819e6b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fef00009850 tx=0x7fef0000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.036+0000 7feefa7fc700 1 -- 172.21.15.45:0/3186272255 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000e040 con 0x7fef08108f40 2023-10-17T01:23:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.036+0000 7feefa7fc700 1 -- 172.21.15.45:0/3186272255 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef00003e10 con 0x7fef08108f40 2023-10-17T01:23:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.036+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0819f2c0 con 0x7fef08108f40 2023-10-17T01:23:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.036+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef081a5ae0 con 0x7fef08108f40 2023-10-17T01:23:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.036+0000 7feefa7fc700 1 -- 172.21.15.45:0/3186272255 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0001b750 con 0x7fef08108f40 2023-10-17T01:23:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.044+0000 7feefa7fc700 1 -- 172.21.15.45:0/3186272255 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fef0001da30 con 0x7fef08108f40 2023-10-17T01:23:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.049+0000 7feefa7fc700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feef40a0cf0 0x7feef40a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:44 smithi045 ceph-mon[98474]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 653 op/s 2023-10-17T01:23:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.049+0000 7fef22fcbb80 1 Processor -- start 2023-10-17T01:23:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.050+0000 7feefa7fc700 1 -- 172.21.15.45:0/3186272255 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fef000cd510 con 0x7fef08108f40 2023-10-17T01:23:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.050+0000 7fef22fcbb80 1 -- start start 2023-10-17T01:23:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400e94c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400e95790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400e96210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400dec9b0 con 0x562400e95d90 2023-10-17T01:23:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400dea370 con 0x562400e95190 2023-10-17T01:23:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400de5c60 con 0x562400e94830 2023-10-17T01:23:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.051+0000 7fef0d39b700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feef40a0cf0 0x7feef40a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0db9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400e96210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0db9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400e96210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60702/0 (socket says 172.21.15.45:60702) 2023-10-17T01:23:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0db9c700 1 -- 172.21.15.45:0/3608177423 learned_addr learned my addr 172.21.15.45:0/3608177423 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400e95790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0d39b700 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400e94c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0cb9a700 1 -- 172.21.15.45:0/3608177423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 msgr2=0x562400e94c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400e94c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.052+0000 7fef0cb9a700 1 -- 172.21.15.45:0/3608177423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 msgr2=0x562400e96210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.053+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400e96210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.053+0000 7fef0cb9a700 1 -- 172.21.15.45:0/3608177423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562400c13d80 con 0x562400e95190 2023-10-17T01:23:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.053+0000 7fef0db9c700 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400e96210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.054+0000 7fef0d39b700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feef40a0cf0 0x7feef40a31b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef08108400 tx=0x7fef04009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.054+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400e95790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feefc009a20 tx=0x7feefc009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6b6c26d96794e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.054+0000 7feef37fe700 1 -- 172.21.15.45:0/3608177423 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0094f0 con 0x562400e95190 2023-10-17T01:23:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.054+0000 7feef37fe700 1 -- 172.21.15.45:0/3608177423 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feefc00ed90 con 0x562400e95190 2023-10-17T01:23:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.054+0000 7feef37fe700 1 -- 172.21.15.45:0/3608177423 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc013e80 con 0x562400e95190 2023-10-17T01:23:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 -- 172.21.15.45:0/3608177423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 msgr2=0x562400e95790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400e95790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feefc009a20 tx=0x7feefc009d30 comp rx=0 tx=0).stop 2023-10-17T01:23:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 -- 172.21.15.45:0/3608177423 shutdown_connections 2023-10-17T01:23:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400e96210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400e94c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/3608177423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400e95790 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.056+0000 7fef22fcbb80 1 -- 172.21.15.45:0/3608177423 >> 172.21.15.45:0/3608177423 conn(0x562400df4070 msgr2=0x562400e8f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.057+0000 7fef22fcbb80 1 -- 172.21.15.45:0/3608177423 shutdown_connections 2023-10-17T01:23:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.057+0000 7fef22fcbb80 1 -- 172.21.15.45:0/3608177423 wait complete. 2023-10-17T01:23:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.058+0000 7fef22fcbb80 1 -- start start 2023-10-17T01:23:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.060+0000 7fef22fcbb80 1 Processor -- start 2023-10-17T01:23:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400eaf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95190 0x562400eaf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eb6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400d824b0 con 0x562400e95d90 2023-10-17T01:23:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400d82150 con 0x562400e94830 2023-10-17T01:23:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400c698c0 con 0x562400e95190 2023-10-17T01:23:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.061+0000 7fef22fcbb80 10 client.0 _ll_register_callbacks cb 0x562400ccfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400eaf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400eaf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41998/0 (socket says 172.21.15.45:41998) 2023-10-17T01:23:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 -- 172.21.15.45:0/1474690924 learned_addr learned my addr 172.21.15.45:0/1474690924 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0db9c700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eb6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95190 0x562400eaf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 -- 172.21.15.45:0/1474690924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95190 msgr2=0x562400eaf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95190 0x562400eaf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 -- 172.21.15.45:0/1474690924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 msgr2=0x562400eb6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eb6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.063+0000 7fef0d39b700 1 -- 172.21.15.45:0/1474690924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef081a5ae0 con 0x562400e94830 2023-10-17T01:23:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.064+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400eaf3e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef04005e90 tx=0x7fef040047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9d9b1e23ef770dd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.064+0000 7feef0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:23:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.064+0000 7feef0ff9700 1 -- 172.21.15.45:0/1474690924 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0401a070 con 0x562400e94830 2023-10-17T01:23:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.064+0000 7feef0ff9700 1 -- 172.21.15.45:0/1474690924 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fef04009da0 con 0x562400e94830 2023-10-17T01:23:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.065+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562400e10cb0 con 0x562400e94830 2023-10-17T01:23:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.065+0000 7feef0ff9700 1 -- 172.21.15.45:0/1474690924 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef04021650 con 0x562400e94830 2023-10-17T01:23:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.065+0000 7fef22fcbb80 10 client.26426 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.065+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7feefc009650 con 0x562400e94830 2023-10-17T01:23:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.066+0000 7feef0ff9700 1 -- 172.21.15.45:0/1474690924 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562400e10cb0 con 0x562400e94830 2023-10-17T01:23:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.067+0000 7feef0ff9700 1 -- 172.21.15.45:0/1474690924 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fef0402a850 con 0x562400e94830 2023-10-17T01:23:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.067+0000 7fef22fcbb80 10 client.26426 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.067+0000 7fef22fcbb80 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.067+0000 7fef22fcbb80 20 client.26426 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.067+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400ebb200 0x562400ebd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.068+0000 7fef22fcbb80 4 client.26426 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.068+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562400e63cd0 con 0x562400ebb200 2023-10-17T01:23:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.068+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400ebb200 0x562400ebd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.069+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400ebb200 0x562400ebd620 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f83dae4954ebe4 server_cookie=2f02efb79784e510 in_seq=0 out_seq=0 2023-10-17T01:23:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.069+0000 7feef0ff9700 10 client.26426 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7feef0ff9700 1 -- 172.21.15.45:0/1474690924 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x562400e63cd0 con 0x562400ebb200 2023-10-17T01:23:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7feef0ff9700 10 client.26426 handle_command_reply: tid=0 2023-10-17T01:23:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 client.26426 shutdown 2023-10-17T01:23:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 msgr2=0x562400eaf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400eaf3e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef04005e90 tx=0x7fef040047c0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 shutdown_connections 2023-10-17T01:23:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eb6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400ebb200 0x562400ebd620 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400eaf3e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1474690924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95190 0x562400eaf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 >> 172.21.15.45:0/1474690924 conn(0x562400df4070 msgr2=0x562400e8ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7feef0ff9700 0 client.26426 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.070+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 shutdown_connections 2023-10-17T01:23:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.071+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1474690924 wait complete. 2023-10-17T01:23:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.071+0000 7fef22fcbb80 20 client.26426 trim_cache size 0 max 16384 2023-10-17T01:23:45.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:44 smithi060 ceph-mon[134136]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 653 op/s 2023-10-17T01:23:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.095+0000 7fef22fcbb80 1 Processor -- start 2023-10-17T01:23:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.095+0000 7fef22fcbb80 1 -- start start 2023-10-17T01:23:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.095+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400ebd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400eaea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400ebd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400ebd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34354/0 (socket says 172.21.15.45:34354) 2023-10-17T01:23:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef0cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400eaea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef0cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400eaea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42008/0 (socket says 172.21.15.45:42008) 2023-10-17T01:23:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400e95bf0 con 0x562400e95d90 2023-10-17T01:23:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef0d39b700 1 -- 172.21.15.45:0/1026226608 learned_addr learned my addr 172.21.15.45:0/1026226608 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400e90df0 con 0x562400e95190 2023-10-17T01:23:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.096+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400e90180 con 0x562400e94830 2023-10-17T01:23:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7fef0d39b700 1 -- 172.21.15.45:0/1026226608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 msgr2=0x562400eaea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400eaea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7fef0d39b700 1 -- 172.21.15.45:0/1026226608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 msgr2=0x562400eae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7fef0d39b700 1 -- 172.21.15.45:0/1026226608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562400bfcc00 con 0x562400e94830 2023-10-17T01:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400ebd5f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef0402abe0 tx=0x7fef04015d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e9279206a6e451 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7feef3fff700 1 -- 172.21.15.45:0/1026226608 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04003dc0 con 0x562400e94830 2023-10-17T01:23:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7feef3fff700 1 -- 172.21.15.45:0/1026226608 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef0402a440 con 0x562400e94830 2023-10-17T01:23:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.097+0000 7feef3fff700 1 -- 172.21.15.45:0/1026226608 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040218b0 con 0x562400e94830 2023-10-17T01:23:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 msgr2=0x562400ebd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400ebd5f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef0402abe0 tx=0x7fef04015d70 comp rx=0 tx=0).stop 2023-10-17T01:23:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 shutdown_connections 2023-10-17T01:23:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95d90 0x562400eae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e94830 0x562400ebd5f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1026226608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e95190 0x562400eaea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 >> 172.21.15.45:0/1026226608 conn(0x562400df4070 msgr2=0x562400e8f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 shutdown_connections 2023-10-17T01:23:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.098+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1026226608 wait complete. 2023-10-17T01:23:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 -- start start 2023-10-17T01:23:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 Processor -- start 2023-10-17T01:23:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400f06b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95190 0x562400f090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95d90 0x562400f0b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400f0bcc0 con 0x562400e95190 2023-10-17T01:23:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400f0be30 con 0x562400e94830 2023-10-17T01:23:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562400f0bff0 con 0x562400e95d90 2023-10-17T01:23:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.099+0000 7fef22fcbb80 10 client.0 _ll_register_callbacks cb 0x7fef04004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400f06b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400f06b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42024/0 (socket says 172.21.15.45:42024) 2023-10-17T01:23:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 -- 172.21.15.45:0/1798488884 learned_addr learned my addr 172.21.15.45:0/1798488884 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0db9c700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95d90 0x562400f0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95190 0x562400f090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 -- 172.21.15.45:0/1798488884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95d90 msgr2=0x562400f0b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95d90 0x562400f0b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 -- 172.21.15.45:0/1798488884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95190 msgr2=0x562400f090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95190 0x562400f090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 -- 172.21.15.45:0/1798488884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc009650 con 0x562400e94830 2023-10-17T01:23:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0db9c700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95d90 0x562400f0b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7fef0d39b700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400f06b60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fef0402ac60 tx=0x7fef04004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167f890243fa3af4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.100+0000 7feef0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:23:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7feef0ff9700 1 -- 172.21.15.45:0/1798488884 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0401a070 con 0x562400e94830 2023-10-17T01:23:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7feef0ff9700 1 -- 172.21.15.45:0/1798488884 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fef0402a9c0 con 0x562400e94830 2023-10-17T01:23:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feefc00ed90 con 0x562400e94830 2023-10-17T01:23:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7feef0ff9700 1 -- 172.21.15.45:0/1798488884 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef04021650 con 0x562400e94830 2023-10-17T01:23:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 10 client.26432 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562400c3a1f0 con 0x562400e94830 2023-10-17T01:23:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7feef0ff9700 1 -- 172.21.15.45:0/1798488884 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef0404d020 con 0x562400e94830 2023-10-17T01:23:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7feef0ff9700 1 -- 172.21.15.45:0/1798488884 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fef04011980 con 0x562400e94830 2023-10-17T01:23:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 10 client.26432 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 10 client.26432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 20 client.26432 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400f0d4b0 0x562400f0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 4 client.26432 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:23:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.101+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562400ddece0 con 0x562400f0d4b0 2023-10-17T01:23:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.102+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400f0d4b0 0x562400f0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.105+0000 7feef0ff9700 10 client.26432 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.105+0000 7fef0cb9a700 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400f0d4b0 0x562400f0f8a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850f8697e85ec252 server_cookie=82f652d9679b4f9a in_seq=0 out_seq=0 2023-10-17T01:23:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.118+0000 7feef0ff9700 1 -- 172.21.15.45:0/1798488884 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562400ddece0 con 0x562400f0d4b0 2023-10-17T01:23:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.118+0000 7feef0ff9700 10 client.26432 handle_command_reply: tid=0 2023-10-17T01:23:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.118+0000 7fef22fcbb80 1 client.26432 shutdown 2023-10-17T01:23:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 msgr2=0x562400f06b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400f06b60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fef0402ac60 tx=0x7fef04004730 comp rx=0 tx=0).stop 2023-10-17T01:23:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 shutdown_connections 2023-10-17T01:23:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562400e95d90 0x562400f0b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562400f0d4b0 0x562400f0f8a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562400e94830 0x562400f06b60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 --2- 172.21.15.45:0/1798488884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562400e95190 0x562400f090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7feef0ff9700 0 client.26432 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 >> 172.21.15.45:0/1798488884 conn(0x562400df4070 msgr2=0x562400eb7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 shutdown_connections 2023-10-17T01:23:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 1 -- 172.21.15.45:0/1798488884 wait complete. 2023-10-17T01:23:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.119+0000 7fef22fcbb80 20 client.26432 trim_cache size 0 max 16384 2023-10-17T01:23:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:23:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:23:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:23:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:23:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feef40a0cf0 msgr2=0x7feef40a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feef40a0cf0 0x7feef40a31b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef08108400 tx=0x7fef04009760 comp rx=0 tx=0).stop 2023-10-17T01:23:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 msgr2=0x7fef0819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 0x7fef0819e6b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fef00009850 tx=0x7fef0000bce0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 shutdown_connections 2023-10-17T01:23:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7feef40a0cf0 0x7feef40a31b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef08108f40 0x7fef0819e6b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef081073b0 0x7fef0819d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 --2- 172.21.15.45:0/3186272255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef081085a0 0x7fef0819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 >> 172.21.15.45:0/3186272255 conn(0x7fef08076330 msgr2=0x7fef08078530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 shutdown_connections 2023-10-17T01:23:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:45.120+0000 7fef0f5ff700 1 -- 172.21.15.45:0/3186272255 wait complete. 2023-10-17T01:23:45.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:44 smithi181 ceph-mon[134520]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 653 op/s 2023-10-17T01:23:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 515.0 s, 1577.9 tps, lat 63.236 ms stddev 71.501 2023-10-17T01:23:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:46 smithi045 ceph-mon[98474]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 651 op/s 2023-10-17T01:23:47.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:46 smithi060 ceph-mon[134136]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 651 op/s 2023-10-17T01:23:47.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:46 smithi181 ceph-mon[134520]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 651 op/s 2023-10-17T01:23:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:48 smithi045 ceph-mon[98474]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 668 op/s 2023-10-17T01:23:49.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:48 smithi060 ceph-mon[134136]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 668 op/s 2023-10-17T01:23:49.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:48 smithi181 ceph-mon[134520]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 668 op/s 2023-10-17T01:23:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 520.0 s, 1796.6 tps, lat 55.581 ms stddev 61.909 2023-10-17T01:23:51.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:50 smithi045 ceph-mon[98474]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 708 op/s 2023-10-17T01:23:51.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:50 smithi060 ceph-mon[134136]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 708 op/s 2023-10-17T01:23:51.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:50 smithi181 ceph-mon[134520]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 708 op/s 2023-10-17T01:23:53.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:52 smithi060 ceph-mon[134136]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-10-17T01:23:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:52 smithi181 ceph-mon[134520]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-10-17T01:23:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:52 smithi045 ceph-mon[98474]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 664 op/s 2023-10-17T01:23:55.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:54 smithi060 ceph-mon[134136]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 MiB/s wr, 645 op/s 2023-10-17T01:23:55.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:54 smithi181 ceph-mon[134520]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 MiB/s wr, 645 op/s 2023-10-17T01:23:55.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:54 smithi045 ceph-mon[98474]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 MiB/s wr, 645 op/s 2023-10-17T01:23:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 525.0 s, 1306.6 tps, lat 76.216 ms stddev 99.839 2023-10-17T01:23:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:56 smithi045 ceph-mon[98474]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 675 op/s 2023-10-17T01:23:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:56 smithi060 ceph-mon[134136]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 675 op/s 2023-10-17T01:23:57.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:56 smithi181 ceph-mon[134520]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 675 op/s 2023-10-17T01:23:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 -- 172.21.15.45:0/281047719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d81017d0 msgr2=0x7f76d8103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 --2- 172.21.15.45:0/281047719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d81017d0 0x7f76d8103bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009990 tx=0x7f76d4009ca0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 -- 172.21.15.45:0/281047719 shutdown_connections 2023-10-17T01:23:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 --2- 172.21.15.45:0/281047719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d8106a30 0x7f76d8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 --2- 172.21.15.45:0/281047719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d81017d0 0x7f76d8103bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 --2- 172.21.15.45:0/281047719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d8104100 0x7f76d81064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.347+0000 7f76df7ff700 1 -- 172.21.15.45:0/281047719 >> 172.21.15.45:0/281047719 conn(0x7f76d80fb1a0 msgr2=0x7f76d80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.348+0000 7f76df7ff700 1 -- 172.21.15.45:0/281047719 shutdown_connections 2023-10-17T01:23:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.348+0000 7f76df7ff700 1 -- 172.21.15.45:0/281047719 wait complete. 2023-10-17T01:23:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 Processor -- start 2023-10-17T01:23:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 -- start start 2023-10-17T01:23:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d81017d0 0x7f76d8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 0x7f76d8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d8106a30 0x7f76d819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d819a7e0 con 0x7f76d81017d0 2023-10-17T01:23:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d819a950 con 0x7f76d8104100 2023-10-17T01:23:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76df7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d819ab10 con 0x7f76d8106a30 2023-10-17T01:23:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 0x7f76d8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76dd59b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d81017d0 0x7f76d8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 0x7f76d8199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:36934/0 (socket says 172.21.15.45:36934) 2023-10-17T01:23:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.349+0000 7f76dcd9a700 1 -- 172.21.15.45:0/2228479190 learned_addr learned my addr 172.21.15.45:0/2228479190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d8106a30 0x7f76d819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76dcd9a700 1 -- 172.21.15.45:0/2228479190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d8106a30 msgr2=0x7f76d819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d8106a30 0x7f76d819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76dcd9a700 1 -- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d81017d0 msgr2=0x7f76d8198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d81017d0 0x7f76d8198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76dcd9a700 1 -- 172.21.15.45:0/2228479190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40095f0 con 0x7f76d8104100 2023-10-17T01:23:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.350+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 0x7f76d8199330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f76c800b530 tx=0x7f76c800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.351+0000 7f76ce7fc700 1 -- 172.21.15.45:0/2228479190 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011830 con 0x7f76d8104100 2023-10-17T01:23:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.351+0000 7f76ce7fc700 1 -- 172.21.15.45:0/2228479190 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c8011e50 con 0x7f76d8104100 2023-10-17T01:23:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.351+0000 7f76ce7fc700 1 -- 172.21.15.45:0/2228479190 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c800f2e0 con 0x7f76d8104100 2023-10-17T01:23:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.351+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d819adf0 con 0x7f76d8104100 2023-10-17T01:23:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.351+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d81a1530 con 0x7f76d8104100 2023-10-17T01:23:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.359+0000 7f76f3036b80 1 Processor -- start 2023-10-17T01:23:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.359+0000 7f76f3036b80 1 -- start start 2023-10-17T01:23:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.360+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efe3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.360+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662efe4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.360+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4d90 0x55662efe5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.360+0000 7f76f3036b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662ef3b9b0 con 0x55662efe3830 2023-10-17T01:23:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.360+0000 7f76f3036b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662ef39370 con 0x55662efe4190 2023-10-17T01:23:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.360+0000 7f76f3036b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662ef34c60 con 0x55662efe4d90 2023-10-17T01:23:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.363+0000 7f76ce7fc700 1 -- 172.21.15.45:0/2228479190 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f76c8011990 con 0x7f76d8104100 2023-10-17T01:23:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.365+0000 7f76ce7fc700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f76c40a0d20 0x7f76c40a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.365+0000 7f76ce7fc700 1 -- 172.21.15.45:0/2228479190 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f76c80cb990 con 0x7f76d8104100 2023-10-17T01:23:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.366+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f76c40a0d20 0x7f76c40a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.366+0000 7f76ddd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efe3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.366+0000 7f76ddd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efe3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58276/0 (socket says 172.21.15.45:58276) 2023-10-17T01:23:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.367+0000 7f76ddd9c700 1 -- 172.21.15.45:0/1839933305 learned_addr learned my addr 172.21.15.45:0/1839933305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.366+0000 7f76dd59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662efe4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.370+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f76c40a0d20 0x7f76c40a31e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009fd0 tx=0x7f76d0007b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.370+0000 7f76ddd9c700 1 -- 172.21.15.45:0/1839933305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4d90 msgr2=0x55662efe5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.370+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4d90 0x55662efe5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.370+0000 7f76ddd9c700 1 -- 172.21.15.45:0/1839933305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 msgr2=0x55662efe4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.370+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662efe4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.370+0000 7f76ddd9c700 1 -- 172.21.15.45:0/1839933305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55662ed62d80 con 0x55662efe3830 2023-10-17T01:23:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.371+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efe3c50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76d0007f00 tx=0x7f76d001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9f070926924442 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.372+0000 7f76c37fe700 1 -- 172.21.15.45:0/1839933305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0026070 con 0x55662efe3830 2023-10-17T01:23:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.372+0000 7f76c37fe700 1 -- 172.21.15.45:0/1839933305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d0017cb0 con 0x55662efe3830 2023-10-17T01:23:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.372+0000 7f76c37fe700 1 -- 172.21.15.45:0/1839933305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d001ad50 con 0x55662efe3830 2023-10-17T01:23:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 -- 172.21.15.45:0/1839933305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 msgr2=0x55662efe3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efe3c50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76d0007f00 tx=0x7f76d001c000 comp rx=0 tx=0).stop 2023-10-17T01:23:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 -- 172.21.15.45:0/1839933305 shutdown_connections 2023-10-17T01:23:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4d90 0x55662efe5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efe3c50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 --2- 172.21.15.45:0/1839933305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662efe4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 -- 172.21.15.45:0/1839933305 >> 172.21.15.45:0/1839933305 conn(0x55662ef43070 msgr2=0x55662efde2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 -- 172.21.15.45:0/1839933305 shutdown_connections 2023-10-17T01:23:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.373+0000 7f76f3036b80 1 -- 172.21.15.45:0/1839933305 wait complete. 2023-10-17T01:23:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.375+0000 7f76f3036b80 1 -- start start 2023-10-17T01:23:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.376+0000 7f76f3036b80 1 Processor -- start 2023-10-17T01:23:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.376+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.376+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4190 0x55662efed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.376+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662eff4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.376+0000 7f76f3036b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662eed14b0 con 0x55662efe3830 2023-10-17T01:23:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.376+0000 7f76f3036b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662eed1150 con 0x55662efe4d90 2023-10-17T01:23:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.377+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4190 0x55662efed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.377+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4190 0x55662efed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59790/0 (socket says 172.21.15.45:59790) 2023-10-17T01:23:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.377+0000 7f76dcd9a700 1 -- 172.21.15.45:0/875781435 learned_addr learned my addr 172.21.15.45:0/875781435 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.377+0000 7f76f3036b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662edb88c0 con 0x55662efe4190 2023-10-17T01:23:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.377+0000 7f76f3036b80 10 client.0 _ll_register_callbacks cb 0x55662ee1ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.377+0000 7f76dd59b700 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.378+0000 7f76dd59b700 1 -- 172.21.15.45:0/875781435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4190 msgr2=0x55662efed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.378+0000 7f76dd59b700 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4190 0x55662efed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.378+0000 7f76dd59b700 1 -- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 msgr2=0x55662eff4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.378+0000 7f76dd59b700 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662eff4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.378+0000 7f76dd59b700 1 -- 172.21.15.45:0/875781435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d0011040 con 0x55662efe3830 2023-10-17T01:23:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.379+0000 7f76dd59b700 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efed3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76d8199b90 tx=0x7f76d4009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef30501edd02c09 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.379+0000 7f76c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:23:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.379+0000 7f76c0ff9700 1 -- 172.21.15.45:0/875781435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d401d070 con 0x55662efe3830 2023-10-17T01:23:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.380+0000 7f76c0ff9700 1 -- 172.21.15.45:0/875781435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f76d40056c0 con 0x55662efe3830 2023-10-17T01:23:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.382+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55662ef5fcb0 con 0x55662efe3830 2023-10-17T01:23:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.382+0000 7f76c0ff9700 1 -- 172.21.15.45:0/875781435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d4004710 con 0x55662efe3830 2023-10-17T01:23:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.382+0000 7f76f3036b80 10 client.16149 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.382+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55662ed4bc00 con 0x55662efe3830 2023-10-17T01:23:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76c0ff9700 1 -- 172.21.15.45:0/875781435 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55662ef5fcb0 con 0x55662efe3830 2023-10-17T01:23:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76c0ff9700 1 -- 172.21.15.45:0/875781435 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f76d400faf0 con 0x55662efe3830 2023-10-17T01:23:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76f3036b80 10 client.16149 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76f3036b80 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76f3036b80 20 client.16149 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76f3036b80 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f00a1a0 0x55662f00c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76f3036b80 4 client.16149 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:23:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.383+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55662efb2cd0 con 0x55662f00a1a0 2023-10-17T01:23:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.392+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f00a1a0 0x55662f00c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.393+0000 7f76c0ff9700 10 client.16149 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.393+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f00a1a0 0x55662f00c5c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5184cdcc85f1c6d server_cookie=4a4bed8954811768 in_seq=0 out_seq=0 2023-10-17T01:23:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.394+0000 7f76c0ff9700 1 -- 172.21.15.45:0/875781435 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55662efb2cd0 con 0x55662f00a1a0 2023-10-17T01:23:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.394+0000 7f76c0ff9700 10 client.16149 handle_command_reply: tid=0 2023-10-17T01:23:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.394+0000 7f76f3036b80 1 client.16149 shutdown 2023-10-17T01:23:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 msgr2=0x55662efed3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efed3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76d8199b90 tx=0x7f76d4009910 comp rx=0 tx=0).stop 2023-10-17T01:23:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 shutdown_connections 2023-10-17T01:23:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f00a1a0 0x55662f00c5c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662eff4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe3830 0x55662efed3e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 --2- 172.21.15.45:0/875781435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe4190 0x55662efed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 >> 172.21.15.45:0/875781435 conn(0x55662ef43070 msgr2=0x55662efdda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.395+0000 7f76c0ff9700 0 client.16149 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.396+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 shutdown_connections 2023-10-17T01:23:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.396+0000 7f76f3036b80 1 -- 172.21.15.45:0/875781435 wait complete. 2023-10-17T01:23:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.396+0000 7f76f3036b80 20 client.16149 trim_cache size 0 max 16384 2023-10-17T01:23:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.423+0000 7f76f3036b80 1 Processor -- start 2023-10-17T01:23:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 -- start start 2023-10-17T01:23:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f00c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662f0061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4d90 0x55662efe7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662efdf180 con 0x55662efe4d90 2023-10-17T01:23:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662efd6240 con 0x55662efe4190 2023-10-17T01:23:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76f3036b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662efd8990 con 0x55662efe3830 2023-10-17T01:23:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662f0061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662f0061a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:36968/0 (socket says 172.21.15.45:36968) 2023-10-17T01:23:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76dcd9a700 1 -- 172.21.15.45:0/163995381 learned_addr learned my addr 172.21.15.45:0/163995381 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76ddd9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4d90 0x55662efe7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.424+0000 7f76dd59b700 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f00c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76ddd9c700 1 -- 172.21.15.45:0/163995381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 msgr2=0x55662f00c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f00c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76ddd9c700 1 -- 172.21.15.45:0/163995381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 msgr2=0x55662f0061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662f0061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76ddd9c700 1 -- 172.21.15.45:0/163995381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55662ed891f0 con 0x55662efe4d90 2023-10-17T01:23:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76dd59b700 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f00c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.425+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4d90 0x55662efe7d50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f76d0025000 tx=0x7f76d0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b19cb420878eb1f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.426+0000 7f76c3fff700 1 -- 172.21.15.45:0/163995381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0026070 con 0x55662efe4d90 2023-10-17T01:23:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.426+0000 7f76c3fff700 1 -- 172.21.15.45:0/163995381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d0004a90 con 0x55662efe4d90 2023-10-17T01:23:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.426+0000 7f76c3fff700 1 -- 172.21.15.45:0/163995381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0022690 con 0x55662efe4d90 2023-10-17T01:23:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 -- 172.21.15.45:0/163995381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4d90 msgr2=0x55662efe7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4d90 0x55662efe7d50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f76d0025000 tx=0x7f76d0009d50 comp rx=0 tx=0).stop 2023-10-17T01:23:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 -- 172.21.15.45:0/163995381 shutdown_connections 2023-10-17T01:23:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4d90 0x55662efe7d50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f00c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 --2- 172.21.15.45:0/163995381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4190 0x55662f0061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 -- 172.21.15.45:0/163995381 >> 172.21.15.45:0/163995381 conn(0x55662ef43070 msgr2=0x55662efec030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 -- 172.21.15.45:0/163995381 shutdown_connections 2023-10-17T01:23:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.427+0000 7f76f3036b80 1 -- 172.21.15.45:0/163995381 wait complete. 2023-10-17T01:23:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 -- start start 2023-10-17T01:23:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 Processor -- start 2023-10-17T01:23:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f055cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4190 0x55662f058240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662f05a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662f05af30 con 0x55662efe4190 2023-10-17T01:23:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662f05b0a0 con 0x55662efe4d90 2023-10-17T01:23:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662f05b260 con 0x55662efe3830 2023-10-17T01:23:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.428+0000 7f76f3036b80 10 client.0 _ll_register_callbacks cb 0x55662ee1ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:23:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f055cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dcd9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4190 0x55662f058240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f055cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59802/0 (socket says 172.21.15.45:59802) 2023-10-17T01:23:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76ddd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662f05a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 -- 172.21.15.45:0/4028965035 learned_addr learned my addr 172.21.15.45:0/4028965035 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:23:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 -- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 msgr2=0x55662f05a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662f05a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 -- 172.21.15.45:0/4028965035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4190 msgr2=0x55662f058240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:23:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4190 0x55662f058240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76dd59b700 1 -- 172.21.15.45:0/4028965035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55662ed4bc00 con 0x55662efe3830 2023-10-17T01:23:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.429+0000 7f76ddd9c700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662f05a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.430+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4190 0x55662f058240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:23:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.430+0000 7f76dd59b700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f055cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f76d4001eb0 tx=0x7f76d40056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe881368b5482e93 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:23:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.430+0000 7f76c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:23:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.430+0000 7f76c0ff9700 1 -- 172.21.15.45:0/4028965035 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d401d040 con 0x55662efe3830 2023-10-17T01:23:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.430+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76d0017cb0 con 0x55662efe3830 2023-10-17T01:23:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.430+0000 7f76c0ff9700 1 -- 172.21.15.45:0/4028965035 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f76d4000e40 con 0x55662efe3830 2023-10-17T01:23:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.431+0000 7f76c0ff9700 1 -- 172.21.15.45:0/4028965035 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d40245f0 con 0x55662efe3830 2023-10-17T01:23:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.431+0000 7f76f3036b80 10 client.26407 fetch_fsmap learned FSMap version 40 2023-10-17T01:23:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.431+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55662ed62d80 con 0x55662efe3830 2023-10-17T01:23:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.431+0000 7f76c0ff9700 1 -- 172.21.15.45:0/4028965035 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d0017cb0 con 0x55662efe3830 2023-10-17T01:23:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.431+0000 7f76c0ff9700 1 -- 172.21.15.45:0/4028965035 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f76d4004710 con 0x55662efe3830 2023-10-17T01:23:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76f3036b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:23:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76f3036b80 10 client.26407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:23:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76f3036b80 20 client.26407 populate_metadata read hostname 'smithi045' 2023-10-17T01:23:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76f3036b80 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f05c790 0x55662f05eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:23:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76f3036b80 4 client.26407 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:23:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55662ef2dce0 con 0x55662f05c790 2023-10-17T01:23:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.432+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f05c790 0x55662f05eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:23:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.433+0000 7f76dcd9a700 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f05c790 0x55662f05eb80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ccfa37b74afd02e server_cookie=e4764dd6140abe57 in_seq=0 out_seq=0 2023-10-17T01:23:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.433+0000 7f76c0ff9700 10 client.26407 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76c0ff9700 1 -- 172.21.15.45:0/4028965035 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55662ef2dce0 con 0x55662f05c790 2023-10-17T01:23:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76c0ff9700 10 client.26407 handle_command_reply: tid=0 2023-10-17T01:23:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 client.26407 shutdown 2023-10-17T01:23:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 msgr2=0x55662f055cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f055cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f76d4001eb0 tx=0x7f76d40056f0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 shutdown_connections 2023-10-17T01:23:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55662f05c790 0x55662f05eb80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55662efe4d90 0x55662f05a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55662efe3830 0x55662f055cf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 --2- 172.21.15.45:0/4028965035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55662efe4190 0x55662f058240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 >> 172.21.15.45:0/4028965035 conn(0x55662ef43070 msgr2=0x55662f00c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76c0ff9700 0 client.26407 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:23:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 shutdown_connections 2023-10-17T01:23:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.460+0000 7f76f3036b80 1 -- 172.21.15.45:0/4028965035 wait complete. 2023-10-17T01:23:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76f3036b80 20 client.26407 trim_cache size 0 max 16384 2023-10-17T01:23:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:23:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:23:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:23:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:23:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f76c40a0d20 msgr2=0x7f76c40a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f76c40a0d20 0x7f76c40a31e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009fd0 tx=0x7f76d0007b00 comp rx=0 tx=0).stop 2023-10-17T01:23:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 msgr2=0x7f76d8199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:23:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 0x7f76d8199330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f76c800b530 tx=0x7f76c800b840 comp rx=0 tx=0).stop 2023-10-17T01:23:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 shutdown_connections 2023-10-17T01:23:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f76c40a0d20 0x7f76c40a31e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d8106a30 0x7f76d819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76d81017d0 0x7f76d8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 --2- 172.21.15.45:0/2228479190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d8104100 0x7f76d8199330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:23:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.461+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 >> 172.21.15.45:0/2228479190 conn(0x7f76d80fb1a0 msgr2=0x7f76d80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:23:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.462+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 shutdown_connections 2023-10-17T01:23:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:23:58.462+0000 7f76df7ff700 1 -- 172.21.15.45:0/2228479190 wait complete. 2023-10-17T01:23:59.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:23:58 smithi045 ceph-mon[98474]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-17T01:23:59.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:23:58 smithi060 ceph-mon[134136]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-17T01:23:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:23:58 smithi181 ceph-mon[134520]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-17T01:24:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 530.0 s, 1047.4 tps, lat 95.951 ms stddev 117.080 2023-10-17T01:24:01.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:00 smithi045 ceph-mon[98474]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 827 op/s 2023-10-17T01:24:01.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:00 smithi060 ceph-mon[134136]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 827 op/s 2023-10-17T01:24:01.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:00 smithi181 ceph-mon[134520]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 827 op/s 2023-10-17T01:24:03.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:02 smithi045 ceph-mon[98474]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 794 op/s 2023-10-17T01:24:03.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:02 smithi060 ceph-mon[134136]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 794 op/s 2023-10-17T01:24:03.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:02 smithi181 ceph-mon[134520]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 794 op/s 2023-10-17T01:24:04.752 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:24:05.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:04 smithi045 ceph-mon[98474]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 825 op/s 2023-10-17T01:24:05.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:04 smithi060 ceph-mon[134136]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 825 op/s 2023-10-17T01:24:05.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 -- 172.21.15.45:0/2825832095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 msgr2=0x7f7124100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 --2- 172.21.15.45:0/2825832095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f7124100060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f711400b970 tx=0x7f7114009260 comp rx=0 tx=0).stop 2023-10-17T01:24:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 -- 172.21.15.45:0/2825832095 shutdown_connections 2023-10-17T01:24:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 --2- 172.21.15.45:0/2825832095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71241017d0 0x7f7124108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 --2- 172.21.15.45:0/2825832095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f7124100060 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 --2- 172.21.15.45:0/2825832095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7124100e30 0x7f7124101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.110+0000 7f7128fff700 1 -- 172.21.15.45:0/2825832095 >> 172.21.15.45:0/2825832095 conn(0x7f71240fb1b0 msgr2=0x7f71240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.111+0000 7f7128fff700 1 -- 172.21.15.45:0/2825832095 shutdown_connections 2023-10-17T01:24:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.111+0000 7f7128fff700 1 -- 172.21.15.45:0/2825832095 wait complete. 2023-10-17T01:24:05.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.112+0000 7f7128fff700 1 Processor -- start 2023-10-17T01:24:05.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 -- start start 2023-10-17T01:24:05.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f712419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7124100e30 0x7f712419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71241017d0 0x7f71241a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71241a4c00 con 0x7f7124100e30 2023-10-17T01:24:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71241a4d40 con 0x7f71241017d0 2023-10-17T01:24:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.113+0000 7f7128fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71241a4ed0 con 0x7f71240ffc40 2023-10-17T01:24:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f712419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f712419d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59822/0 (socket says 172.21.15.45:59822) 2023-10-17T01:24:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f7122d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71241017d0 0x7f71241a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 -- 172.21.15.45:0/71009195 learned_addr learned my addr 172.21.15.45:0/71009195 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 -- 172.21.15.45:0/71009195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71241017d0 msgr2=0x7f71241a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71241017d0 0x7f71241a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 -- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7124100e30 msgr2=0x7f712419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f7121d9b700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7124100e30 0x7f712419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7124100e30 0x7f712419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f712259c700 1 -- 172.21.15.45:0/71009195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711400b5a0 con 0x7f71240ffc40 2023-10-17T01:24:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.114+0000 7f7121d9b700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7124100e30 0x7f712419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:24:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.115+0000 7f712259c700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f712419d080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7114000c00 tx=0x7f7114009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.115+0000 7f71137fe700 1 -- 172.21.15.45:0/71009195 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140052c0 con 0x7f71240ffc40 2023-10-17T01:24:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.115+0000 7f71137fe700 1 -- 172.21.15.45:0/71009195 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7114009c70 con 0x7f71240ffc40 2023-10-17T01:24:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.115+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71241a5150 con 0x7f71240ffc40 2023-10-17T01:24:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.115+0000 7f71137fe700 1 -- 172.21.15.45:0/71009195 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711401e680 con 0x7f71240ffc40 2023-10-17T01:24:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.116+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71241a55d0 con 0x7f71240ffc40 2023-10-17T01:24:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.120+0000 7f71137fe700 1 -- 172.21.15.45:0/71009195 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f7114003680 con 0x7f71240ffc40 2023-10-17T01:24:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.125+0000 7f71137fe700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f710c0a0bf0 0x7f710c0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.125+0000 7f71137fe700 1 -- 172.21.15.45:0/71009195 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71140d1170 con 0x7f71240ffc40 2023-10-17T01:24:05.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.126+0000 7f7121d9b700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f710c0a0bf0 0x7f710c0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.126+0000 7f713c9a2b80 1 Processor -- start 2023-10-17T01:24:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.126+0000 7f713c9a2b80 1 -- start start 2023-10-17T01:24:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.126+0000 7f7121d9b700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f710c0a0bf0 0x7f710c0a30b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f711800d5c0 tx=0x7f711800f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.127+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a38ba0 0x564236a38fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.127+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a39d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.127+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a3a2b0 0x564236a3a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.127+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236990a10 con 0x564236a39500 2023-10-17T01:24:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.127+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56423698e3d0 con 0x564236a38ba0 2023-10-17T01:24:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.127+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236989cc0 con 0x564236a3a2b0 2023-10-17T01:24:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.128+0000 7f7122d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a3a2b0 0x564236a3a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.128+0000 7f7122d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a3a2b0 0x564236a3a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59826/0 (socket says 172.21.15.45:59826) 2023-10-17T01:24:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.128+0000 7f7122d9d700 1 -- 172.21.15.45:0/3672241423 learned_addr learned my addr 172.21.15.45:0/3672241423 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.128+0000 7f712259c700 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a38ba0 0x564236a38fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f712259c700 1 -- 172.21.15.45:0/3672241423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a3a2b0 msgr2=0x564236a3a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f712259c700 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a3a2b0 0x564236a3a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f712259c700 1 -- 172.21.15.45:0/3672241423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 msgr2=0x564236a39d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f712259c700 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a39d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f712259c700 1 -- 172.21.15.45:0/3672241423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642367b7de0 con 0x564236a38ba0 2023-10-17T01:24:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f712259c700 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a38ba0 0x564236a38fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f71241a5760 tx=0x7f71140054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a4b9f5e4c9040c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.129+0000 7f7110ff9700 1 -- 172.21.15.45:0/3672241423 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140b8ea0 con 0x564236a38ba0 2023-10-17T01:24:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.130+0000 7f7110ff9700 1 -- 172.21.15.45:0/3672241423 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f711409a680 con 0x564236a38ba0 2023-10-17T01:24:05.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.130+0000 7f7110ff9700 1 -- 172.21.15.45:0/3672241423 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114003930 con 0x564236a38ba0 2023-10-17T01:24:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 -- 172.21.15.45:0/3672241423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a38ba0 msgr2=0x564236a38fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a38ba0 0x564236a38fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f71241a5760 tx=0x7f71140054a0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 -- 172.21.15.45:0/3672241423 shutdown_connections 2023-10-17T01:24:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a3a2b0 0x564236a3a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a38ba0 0x564236a38fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/3672241423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a39d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.131+0000 7f713c9a2b80 1 -- 172.21.15.45:0/3672241423 >> 172.21.15.45:0/3672241423 conn(0x5642369980d0 msgr2=0x564236a33950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.132+0000 7f713c9a2b80 1 -- 172.21.15.45:0/3672241423 shutdown_connections 2023-10-17T01:24:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.132+0000 7f713c9a2b80 1 -- 172.21.15.45:0/3672241423 wait complete. 2023-10-17T01:24:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.132+0000 7f713c9a2b80 1 -- start start 2023-10-17T01:24:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.134+0000 7f713c9a2b80 1 Processor -- start 2023-10-17T01:24:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a38ba0 0x564236a425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 0x564236a42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a49b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236926510 con 0x564236a38ba0 2023-10-17T01:24:05.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642369261b0 con 0x564236a3a2b0 2023-10-17T01:24:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56423680d920 con 0x564236a39500 2023-10-17T01:24:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f713c9a2b80 10 client.0 _ll_register_callbacks cb 0x564236873a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f7121d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 0x564236a42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f7121d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 0x564236a42af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59828/0 (socket says 172.21.15.45:59828) 2023-10-17T01:24:05.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.135+0000 7f7121d9b700 1 -- 172.21.15.45:0/1248635081 learned_addr learned my addr 172.21.15.45:0/1248635081 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:05.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.136+0000 7f7121d9b700 1 -- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 msgr2=0x564236a49b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.136+0000 7f7121d9b700 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a49b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.136+0000 7f7121d9b700 1 -- 172.21.15.45:0/1248635081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a38ba0 msgr2=0x564236a425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.136+0000 7f7121d9b700 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a38ba0 0x564236a425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.136+0000 7f7121d9b700 1 -- 172.21.15.45:0/1248635081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71241a55d0 con 0x564236a39500 2023-10-17T01:24:05.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.136+0000 7f7121d9b700 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 0x564236a42af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f711801b7b0 tx=0x7f711800d670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7fa64f0d00c512a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.137+0000 7f7109ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.137+0000 7f7109ffb700 1 -- 172.21.15.45:0/1248635081 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118012070 con 0x564236a39500 2023-10-17T01:24:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.137+0000 7f7109ffb700 1 -- 172.21.15.45:0/1248635081 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7118018600 con 0x564236a39500 2023-10-17T01:24:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.137+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642369b4d10 con 0x564236a39500 2023-10-17T01:24:05.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.138+0000 7f7109ffb700 1 -- 172.21.15.45:0/1248635081 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71180187c0 con 0x564236a39500 2023-10-17T01:24:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.138+0000 7f713c9a2b80 10 client.26419 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5642367a0c60 con 0x564236a39500 2023-10-17T01:24:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f7109ffb700 1 -- 172.21.15.45:0/1248635081 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642369b4d10 con 0x564236a39500 2023-10-17T01:24:05.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f7109ffb700 1 -- 172.21.15.45:0/1248635081 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f711802b070 con 0x564236a39500 2023-10-17T01:24:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 10 client.26419 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 20 client.26419 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236a4e310 0x564236a50730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 4 client.26419 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:24:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.139+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564236a07d30 con 0x564236a4e310 2023-10-17T01:24:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.140+0000 7f712259c700 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236a4e310 0x564236a50730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.141+0000 7f712259c700 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236a4e310 0x564236a50730 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fc502505f3735d server_cookie=920891ac9907c2c in_seq=0 out_seq=0 2023-10-17T01:24:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.141+0000 7f7109ffb700 10 client.26419 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.141+0000 7f7109ffb700 1 -- 172.21.15.45:0/1248635081 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x564236a07d30 con 0x564236a4e310 2023-10-17T01:24:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.141+0000 7f7109ffb700 10 client.26419 handle_command_reply: tid=0 2023-10-17T01:24:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.141+0000 7f713c9a2b80 1 client.26419 shutdown 2023-10-17T01:24:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.142+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 msgr2=0x564236a42af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.142+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 0x564236a42af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f711801b7b0 tx=0x7f711800d670 comp rx=0 tx=0).stop 2023-10-17T01:24:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.143+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 shutdown_connections 2023-10-17T01:24:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.143+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236a4e310 0x564236a50730 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.143+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a49b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.143+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a38ba0 0x564236a425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.143+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/1248635081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a39500 0x564236a42af0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.143+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 >> 172.21.15.45:0/1248635081 conn(0x5642369980d0 msgr2=0x564236a33020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.144+0000 7f7109ffb700 0 client.26419 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.144+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 shutdown_connections 2023-10-17T01:24:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.144+0000 7f713c9a2b80 1 -- 172.21.15.45:0/1248635081 wait complete. 2023-10-17T01:24:05.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.145+0000 7f713c9a2b80 20 client.26419 trim_cache size 0 max 16384 2023-10-17T01:24:05.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:04 smithi181 ceph-mon[134520]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 825 op/s 2023-10-17T01:24:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.174+0000 7f713c9a2b80 1 Processor -- start 2023-10-17T01:24:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.174+0000 7f713c9a2b80 1 -- start start 2023-10-17T01:24:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 0x564236a33020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236a4b850 con 0x564236a39500 2023-10-17T01:24:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236a41a90 con 0x564236a3a2b0 2023-10-17T01:24:05.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236a2b6a0 con 0x564236a38ba0 2023-10-17T01:24:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f712259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 0x564236a33020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f7122d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f712259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 0x564236a33020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59834/0 (socket says 172.21.15.45:59834) 2023-10-17T01:24:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.175+0000 7f712259c700 1 -- 172.21.15.45:0/224523503 learned_addr learned my addr 172.21.15.45:0/224523503 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f712259c700 1 -- 172.21.15.45:0/224523503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 msgr2=0x564236a40720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f712259c700 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a40720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f712259c700 1 -- 172.21.15.45:0/224523503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 msgr2=0x564236a398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f712259c700 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f712259c700 1 -- 172.21.15.45:0/224523503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642367de250 con 0x564236a38ba0 2023-10-17T01:24:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f7122d9d700 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a40720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:24:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.176+0000 7f712259c700 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 0x564236a33020 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f71140968c0 tx=0x7f7114005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98055c3bafc1c4d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:05.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.177+0000 7f71117fa700 1 -- 172.21.15.45:0/224523503 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711409e970 con 0x564236a38ba0 2023-10-17T01:24:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.177+0000 7f71117fa700 1 -- 172.21.15.45:0/224523503 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71140a2b90 con 0x564236a38ba0 2023-10-17T01:24:05.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.177+0000 7f71117fa700 1 -- 172.21.15.45:0/224523503 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140cd110 con 0x564236a38ba0 2023-10-17T01:24:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.178+0000 7f713c9a2b80 1 -- 172.21.15.45:0/224523503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 msgr2=0x564236a33020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.178+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 0x564236a33020 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f71140968c0 tx=0x7f7114005420 comp rx=0 tx=0).stop 2023-10-17T01:24:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 -- 172.21.15.45:0/224523503 shutdown_connections 2023-10-17T01:24:05.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a40720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a38ba0 0x564236a33020 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/224523503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 -- 172.21.15.45:0/224523503 >> 172.21.15.45:0/224523503 conn(0x5642369980d0 msgr2=0x564236a338c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 -- 172.21.15.45:0/224523503 shutdown_connections 2023-10-17T01:24:05.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 -- 172.21.15.45:0/224523503 wait complete. 2023-10-17T01:24:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.179+0000 7f713c9a2b80 1 -- start start 2023-10-17T01:24:05.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.180+0000 7f713c9a2b80 1 Processor -- start 2023-10-17T01:24:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a98410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a9d020 0x564236a9f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236a9f950 con 0x564236a39500 2023-10-17T01:24:05.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236a9fa90 con 0x564236a3a2b0 2023-10-17T01:24:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564236a9fc50 con 0x564236a9d020 2023-10-17T01:24:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f713c9a2b80 10 client.0 _ll_register_callbacks cb 0x7f711800cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f712259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.181+0000 7f7121d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a98410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7121d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a98410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:37036/0 (socket says 172.21.15.45:37036) 2023-10-17T01:24:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7121d9b700 1 -- 172.21.15.45:0/2772488532 learned_addr learned my addr 172.21.15.45:0/2772488532 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7122d9d700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a9d020 0x564236a9f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7122d9d700 1 -- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 msgr2=0x564236a98410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7122d9d700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a98410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7122d9d700 1 -- 172.21.15.45:0/2772488532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 msgr2=0x564236a97ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7122d9d700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a97ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f7122d9d700 1 -- 172.21.15.45:0/2772488532 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c009140 con 0x564236a9d020 2023-10-17T01:24:05.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.182+0000 7f712259c700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a97ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:24:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.184+0000 7f7122d9d700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a9d020 0x564236a9f410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5642369b4e50 tx=0x7f711c007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f2bc5ab0902a87 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:05.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.184+0000 7f7109ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.185+0000 7f7109ffb700 1 -- 172.21.15.45:0/2772488532 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c00e040 con 0x564236a9d020 2023-10-17T01:24:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.185+0000 7f7109ffb700 1 -- 172.21.15.45:0/2772488532 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f711c004590 con 0x564236a9d020 2023-10-17T01:24:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.185+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564236982d40 con 0x564236a9d020 2023-10-17T01:24:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.185+0000 7f7109ffb700 1 -- 172.21.15.45:0/2772488532 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f711c004750 con 0x564236a9d020 2023-10-17T01:24:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.185+0000 7f713c9a2b80 10 client.26425 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:05.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.185+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5642367b7de0 con 0x564236a9d020 2023-10-17T01:24:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f7109ffb700 1 -- 172.21.15.45:0/2772488532 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564236982d40 con 0x564236a9d020 2023-10-17T01:24:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f7109ffb700 1 -- 172.21.15.45:0/2772488532 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f711c0294b0 con 0x564236a9d020 2023-10-17T01:24:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f713c9a2b80 10 client.26425 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f713c9a2b80 10 client.26425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:05.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f713c9a2b80 20 client.26425 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236aa1730 0x564236aa3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f713c9a2b80 4 client.26425 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:24:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564236982a60 con 0x564236aa1730 2023-10-17T01:24:05.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.186+0000 7f712259c700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236aa1730 0x564236aa3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.188+0000 7f712259c700 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236aa1730 0x564236aa3b50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecfa1bda7ca62c6 server_cookie=1e4db8e288164a4 in_seq=0 out_seq=0 2023-10-17T01:24:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.188+0000 7f7109ffb700 10 client.26425 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:05.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.188+0000 7f7109ffb700 1 -- 172.21.15.45:0/2772488532 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564236982a60 con 0x564236aa1730 2023-10-17T01:24:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.188+0000 7f7109ffb700 10 client.26425 handle_command_reply: tid=0 2023-10-17T01:24:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.188+0000 7f713c9a2b80 1 client.26425 shutdown 2023-10-17T01:24:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a9d020 msgr2=0x564236a9f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a9d020 0x564236a9f410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5642369b4e50 tx=0x7f711c007b20 comp rx=0 tx=0).stop 2023-10-17T01:24:05.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 shutdown_connections 2023-10-17T01:24:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564236a9d020 0x564236a9f410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564236a39500 0x564236a97ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x564236aa1730 0x564236aa3b50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 --2- 172.21.15.45:0/2772488532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564236a3a2b0 0x564236a98410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 >> 172.21.15.45:0/2772488532 conn(0x5642369980d0 msgr2=0x564236a40720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.191+0000 7f7109ffb700 0 client.26425 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.192+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 shutdown_connections 2023-10-17T01:24:05.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.193+0000 7f713c9a2b80 1 -- 172.21.15.45:0/2772488532 wait complete. 2023-10-17T01:24:05.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.193+0000 7f713c9a2b80 20 client.26425 trim_cache size 0 max 16384 2023-10-17T01:24:05.197 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:24:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:24:05.197 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:24:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:24:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:24:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:24:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:24:05.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:24:05.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:24:05.199 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:24:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f710c0a0bf0 msgr2=0x7f710c0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f710c0a0bf0 0x7f710c0a30b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f711800d5c0 tx=0x7f711800f040 comp rx=0 tx=0).stop 2023-10-17T01:24:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 msgr2=0x7f712419d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f712419d080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7114000c00 tx=0x7f7114009810 comp rx=0 tx=0).stop 2023-10-17T01:24:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 shutdown_connections 2023-10-17T01:24:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f710c0a0bf0 0x7f710c0a30b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71241017d0 0x7f71241a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71240ffc40 0x7f712419d080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 --2- 172.21.15.45:0/71009195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7124100e30 0x7f712419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 >> 172.21.15.45:0/71009195 conn(0x7f71240fb1b0 msgr2=0x7f7124104360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 shutdown_connections 2023-10-17T01:24:05.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:05.195+0000 7f7128fff700 1 -- 172.21.15.45:0/71009195 wait complete. 2023-10-17T01:24:05.236 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:24:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 535.0 s, 1390.2 tps, lat 71.690 ms stddev 78.330 2023-10-17T01:24:07.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:06 smithi045 ceph-mon[98474]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 843 op/s 2023-10-17T01:24:07.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:06 smithi060 ceph-mon[134136]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 843 op/s 2023-10-17T01:24:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:06 smithi181 ceph-mon[134520]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 843 op/s 2023-10-17T01:24:09.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:08 smithi045 ceph-mon[98474]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 809 op/s 2023-10-17T01:24:09.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:08 smithi060 ceph-mon[134136]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 809 op/s 2023-10-17T01:24:09.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:08 smithi181 ceph-mon[134520]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 809 op/s 2023-10-17T01:24:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 540.0 s, 1237.2 tps, lat 80.521 ms stddev 101.700 2023-10-17T01:24:11.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:10 smithi060 ceph-mon[134136]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 799 op/s 2023-10-17T01:24:11.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:10 smithi181 ceph-mon[134520]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 799 op/s 2023-10-17T01:24:11.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:10 smithi045 ceph-mon[98474]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 799 op/s 2023-10-17T01:24:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.673+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2568677428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0100d80 msgr2=0x7fb6e01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.673+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2568677428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0100d80 0x7fb6e01011a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009990 tx=0x7fb6dc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.674+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2568677428 shutdown_connections 2023-10-17T01:24:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.674+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2568677428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0102910 0x7fb6e01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.674+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2568677428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0100d80 0x7fb6e01011a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.674+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2568677428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e0101f70 0x7fb6e01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.674+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2568677428 >> 172.21.15.45:0/2568677428 conn(0x7fb6e00fc2d0 msgr2=0x7fb6e00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.674+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2568677428 shutdown_connections 2023-10-17T01:24:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.675+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2568677428 wait complete. 2023-10-17T01:24:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.676+0000 7fb6e83ff700 1 Processor -- start 2023-10-17T01:24:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.676+0000 7fb6e83ff700 1 -- start start 2023-10-17T01:24:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e83ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e0100d80 0x7fb6e0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e83ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 0x7fb6e01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e83ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0102910 0x7fb6e019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e83ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019a7a0 con 0x7fb6e0100d80 2023-10-17T01:24:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e83ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019a910 con 0x7fb6e0101f70 2023-10-17T01:24:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e83ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019aad0 con 0x7fb6e0102910 2023-10-17T01:24:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 0x7fb6e01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 0x7fb6e01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:52308/0 (socket says 172.21.15.45:52308) 2023-10-17T01:24:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.677+0000 7fb6e599a700 1 -- 172.21.15.45:0/2812799886 learned_addr learned my addr 172.21.15.45:0/2812799886 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e699c700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0102910 0x7fb6e019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e619b700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e0100d80 0x7fb6e0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e699c700 1 -- 172.21.15.45:0/2812799886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 msgr2=0x7fb6e01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e699c700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 0x7fb6e01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e699c700 1 -- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e0100d80 msgr2=0x7fb6e0198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e699c700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e0100d80 0x7fb6e0198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e699c700 1 -- 172.21.15.45:0/2812799886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc0095f0 con 0x7fb6e0102910 2023-10-17T01:24:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.678+0000 7fb6e599a700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 0x7fb6e01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:24:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.679+0000 7fb6e699c700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0102910 0x7fb6e019a0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8007f10 tx=0x7fb6d800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.679+0000 7fb6d77fe700 1 -- 172.21.15.45:0/2812799886 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8004db0 con 0x7fb6e0102910 2023-10-17T01:24:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.679+0000 7fb6d77fe700 1 -- 172.21.15.45:0/2812799886 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d800f040 con 0x7fb6e0102910 2023-10-17T01:24:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.679+0000 7fb6d77fe700 1 -- 172.21.15.45:0/2812799886 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8005990 con 0x7fb6e0102910 2023-10-17T01:24:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.679+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e019adb0 con 0x7fb6e0102910 2023-10-17T01:24:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.680+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e01a14f0 con 0x7fb6e0102910 2023-10-17T01:24:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.688+0000 7fb6fbdd8b80 1 Processor -- start 2023-10-17T01:24:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.688+0000 7fb6d77fe700 1 -- 172.21.15.45:0/2812799886 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb6d800db20 con 0x7fb6e0102910 2023-10-17T01:24:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.688+0000 7fb6fbdd8b80 1 -- start start 2023-10-17T01:24:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833d830 0x55753833dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753833e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x55753833f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575382959b0 con 0x55753833ed90 2023-10-17T01:24:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538293370 con 0x55753833d830 2023-10-17T01:24:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753828ec60 con 0x55753833e190 2023-10-17T01:24:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6e619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753833e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833d830 0x55753833dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6e619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753833e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:39098/0 (socket says 172.21.15.45:39098) 2023-10-17T01:24:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6e619b700 1 -- 172.21.15.45:0/3450766572 learned_addr learned my addr 172.21.15.45:0/3450766572 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.689+0000 7fb6e599a700 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x55753833f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.690+0000 7fb6e619b700 1 -- 172.21.15.45:0/3450766572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833d830 msgr2=0x55753833dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.690+0000 7fb6e619b700 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833d830 0x55753833dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.690+0000 7fb6e619b700 1 -- 172.21.15.45:0/3450766572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 msgr2=0x55753833f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.690+0000 7fb6e619b700 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x55753833f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.690+0000 7fb6e619b700 1 -- 172.21.15.45:0/3450766572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575380bcd80 con 0x55753833e190 2023-10-17T01:24:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.690+0000 7fb6e619b700 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753833e790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0101dd0 tx=0x7fb6dc009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed19ce36be2952f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.691+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6cc0a0ba0 0x7fb6cc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.691+0000 7fb6d4bfe700 1 -- 172.21.15.45:0/3450766572 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc01d050 con 0x55753833e190 2023-10-17T01:24:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.691+0000 7fb6d4bfe700 1 -- 172.21.15.45:0/3450766572 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc01e040 con 0x55753833e190 2023-10-17T01:24:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.691+0000 7fb6d4bfe700 1 -- 172.21.15.45:0/3450766572 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc023e40 con 0x55753833e190 2023-10-17T01:24:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.691+0000 7fb6d77fe700 1 -- 172.21.15.45:0/2812799886 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6d80cb900 con 0x7fb6e0102910 2023-10-17T01:24:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3450766572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 msgr2=0x55753833e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753833e790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0101dd0 tx=0x7fb6dc009ee0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6e599a700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6cc0a0ba0 0x7fb6cc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3450766572 shutdown_connections 2023-10-17T01:24:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x55753833f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833d830 0x55753833dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3450766572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753833e790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3450766572 >> 172.21.15.45:0/3450766572 conn(0x55753829d070 msgr2=0x5575383382c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3450766572 shutdown_connections 2023-10-17T01:24:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3450766572 wait complete. 2023-10-17T01:24:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.693+0000 7fb6e599a700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6cc0a0ba0 0x7fb6cc0a3060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55753833f750 tx=0x7fb6d0007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.694+0000 7fb6fbdd8b80 1 -- start start 2023-10-17T01:24:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.694+0000 7fb6fbdd8b80 1 Processor -- start 2023-10-17T01:24:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.695+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x55753834fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.695+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833e190 0x5575383500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.695+0000 7fb6e619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x55753834fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.695+0000 7fb6e619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x55753834fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:39110/0 (socket says 172.21.15.45:39110) 2023-10-17T01:24:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833e190 0x5575383500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833e190 0x5575383500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35456/0 (socket says 172.21.15.45:35456) 2023-10-17T01:24:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6e619b700 1 -- 172.21.15.45:0/357927697 learned_addr learned my addr 172.21.15.45:0/357927697 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x557538348f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753822b4b0 con 0x55753833e190 2023-10-17T01:24:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753822b150 con 0x55753833ed90 2023-10-17T01:24:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575381128c0 con 0x55753833d830 2023-10-17T01:24:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.696+0000 7fb6fbdd8b80 10 client.0 _ll_register_callbacks cb 0x557538178a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e699c700 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x557538348f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e619b700 1 -- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 msgr2=0x557538348f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e619b700 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x557538348f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e619b700 1 -- 172.21.15.45:0/357927697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833e190 msgr2=0x5575383500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e619b700 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833e190 0x5575383500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e619b700 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e0073140 con 0x55753833d830 2023-10-17T01:24:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6e619b700 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x55753834fb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009990 tx=0x7fb6dc009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc94ca21a675a90d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.697+0000 7fb6c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.698+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/357927697 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc023d80 con 0x55753833d830 2023-10-17T01:24:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.698+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/357927697 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6dc003890 con 0x55753833d830 2023-10-17T01:24:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.698+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575382b9cb0 con 0x55753833d830 2023-10-17T01:24:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.698+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/357927697 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6dc01d020 con 0x55753833d830 2023-10-17T01:24:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.698+0000 7fb6fbdd8b80 10 client.26437 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.698+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5575380a5c00 con 0x55753833d830 2023-10-17T01:24:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/357927697 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc03b020 con 0x55753833d830 2023-10-17T01:24:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/357927697 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb6dc00f460 con 0x55753833d830 2023-10-17T01:24:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6fbdd8b80 10 client.26437 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6fbdd8b80 10 client.26437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6fbdd8b80 20 client.26437 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55753834d2e0 0x5575383550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6fbdd8b80 4 client.26437 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:24:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.703+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55753830ccd0 con 0x55753834d2e0 2023-10-17T01:24:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.704+0000 7fb6e599a700 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55753834d2e0 0x5575383550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.705+0000 7fb6e599a700 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55753834d2e0 0x5575383550e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f4622dbae4b52e server_cookie=42cdb63393b45082 in_seq=0 out_seq=0 2023-10-17T01:24:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.705+0000 7fb6c9ffb700 10 client.26437 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.706+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/357927697 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55753830ccd0 con 0x55753834d2e0 2023-10-17T01:24:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.706+0000 7fb6c9ffb700 10 client.26437 handle_command_reply: tid=0 2023-10-17T01:24:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.706+0000 7fb6fbdd8b80 1 client.26437 shutdown 2023-10-17T01:24:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 msgr2=0x55753834fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x55753834fb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009990 tx=0x7fb6dc009fd0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 shutdown_connections 2023-10-17T01:24:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55753834d2e0 0x5575383550e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x557538348f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x55753834fb70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/357927697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833e190 0x5575383500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 >> 172.21.15.45:0/357927697 conn(0x55753829d070 msgr2=0x557538337a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6c9ffb700 0 client.26437 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 shutdown_connections 2023-10-17T01:24:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/357927697 wait complete. 2023-10-17T01:24:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.707+0000 7fb6fbdd8b80 20 client.26437 trim_cache size 0 max 16384 2023-10-17T01:24:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.731+0000 7fb6fbdd8b80 1 Processor -- start 2023-10-17T01:24:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 -- start start 2023-10-17T01:24:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x5575383550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 0x5575383451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x557538348a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538338db0 con 0x55753833ed90 2023-10-17T01:24:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753833ebf0 con 0x55753833e190 2023-10-17T01:24:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538339df0 con 0x55753833d830 2023-10-17T01:24:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6e619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x5575383550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 0x5575383451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6e599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 0x5575383451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:52338/0 (socket says 172.21.15.45:52338) 2023-10-17T01:24:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.732+0000 7fb6e599a700 1 -- 172.21.15.45:0/2421044179 learned_addr learned my addr 172.21.15.45:0/2421044179 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.733+0000 7fb6e599a700 1 -- 172.21.15.45:0/2421044179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 msgr2=0x5575383550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.733+0000 7fb6e599a700 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x5575383550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.733+0000 7fb6e699c700 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x557538348a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.733+0000 7fb6e599a700 1 -- 172.21.15.45:0/2421044179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 msgr2=0x557538348a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.734+0000 7fb6e599a700 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x557538348a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.734+0000 7fb6e599a700 1 -- 172.21.15.45:0/2421044179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575380e31f0 con 0x55753833e190 2023-10-17T01:24:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.734+0000 7fb6e699c700 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x557538348a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:24:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.734+0000 7fb6e599a700 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 0x5575383451f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0007fd0 tx=0x7fb6d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ffe9a67c89da599 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.735+0000 7fb6d53ff700 1 -- 172.21.15.45:0/2421044179 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0036040 con 0x55753833e190 2023-10-17T01:24:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.735+0000 7fb6d53ff700 1 -- 172.21.15.45:0/2421044179 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d0032d40 con 0x55753833e190 2023-10-17T01:24:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.735+0000 7fb6d53ff700 1 -- 172.21.15.45:0/2421044179 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d003a4f0 con 0x55753833e190 2023-10-17T01:24:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/2421044179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 msgr2=0x5575383451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 0x5575383451f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0007fd0 tx=0x7fb6d0004750 comp rx=0 tx=0).stop 2023-10-17T01:24:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/2421044179 shutdown_connections 2023-10-17T01:24:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833ed90 0x557538348a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833d830 0x5575383550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/2421044179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833e190 0x5575383451f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/2421044179 >> 172.21.15.45:0/2421044179 conn(0x55753829d070 msgr2=0x557538346030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/2421044179 shutdown_connections 2023-10-17T01:24:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.736+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/2421044179 wait complete. 2023-10-17T01:24:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.737+0000 7fb6fbdd8b80 1 -- start start 2023-10-17T01:24:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.737+0000 7fb6fbdd8b80 1 Processor -- start 2023-10-17T01:24:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833d830 0x55753839c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753839cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x5575383a3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575383a3ba0 con 0x55753833d830 2023-10-17T01:24:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575383a3d10 con 0x55753833ed90 2023-10-17T01:24:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575383a3ed0 con 0x55753833e190 2023-10-17T01:24:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6fbdd8b80 10 client.0 _ll_register_callbacks cb 0x7fb6d0007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6e699c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x5575383a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6e699c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x5575383a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:52352/0 (socket says 172.21.15.45:52352) 2023-10-17T01:24:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6e699c700 1 -- 172.21.15.45:0/3457547792 learned_addr learned my addr 172.21.15.45:0/3457547792 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6e599a700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753839cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.738+0000 7fb6e619b700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833d830 0x55753839c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6e699c700 1 -- 172.21.15.45:0/3457547792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 msgr2=0x55753839cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6e699c700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753839cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6e699c700 1 -- 172.21.15.45:0/3457547792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833d830 msgr2=0x55753839c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6e699c700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833d830 0x55753839c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6e699c700 1 -- 172.21.15.45:0/3457547792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d0012780 con 0x55753833ed90 2023-10-17T01:24:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6e699c700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x5575383a3630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8098f10 tx=0x7fb6d8098ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3c4222d9432935c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:24:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/3457547792 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8099680 con 0x55753833ed90 2023-10-17T01:24:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/3457547792 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6d8098f80 con 0x55753833ed90 2023-10-17T01:24:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.739+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6dc01e040 con 0x55753833ed90 2023-10-17T01:24:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.740+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/3457547792 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6d8077ea0 con 0x55753833ed90 2023-10-17T01:24:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.741+0000 7fb6fbdd8b80 10 client.26480 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.741+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5575380bb250 con 0x55753833ed90 2023-10-17T01:24:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.742+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/3457547792 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d807b070 con 0x55753833ed90 2023-10-17T01:24:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.742+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/3457547792 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb6d80a0b60 con 0x55753833ed90 2023-10-17T01:24:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.743+0000 7fb6fbdd8b80 10 client.26480 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.743+0000 7fb6fbdd8b80 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.743+0000 7fb6fbdd8b80 20 client.26480 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.743+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5575383a5810 0x5575383a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.743+0000 7fb6fbdd8b80 4 client.26480 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:24:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.743+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557538287ce0 con 0x5575383a5810 2023-10-17T01:24:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.744+0000 7fb6e619b700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5575383a5810 0x5575383a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.745+0000 7fb6e619b700 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5575383a5810 0x5575383a7c00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9ea4c3e82d43e6 server_cookie=d9840a4b71730bee in_seq=0 out_seq=0 2023-10-17T01:24:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.746+0000 7fb6c9ffb700 10 client.26480 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.759+0000 7fb6c9ffb700 1 -- 172.21.15.45:0/3457547792 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557538287ce0 con 0x5575383a5810 2023-10-17T01:24:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.759+0000 7fb6c9ffb700 10 client.26480 handle_command_reply: tid=0 2023-10-17T01:24:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 client.26480 shutdown 2023-10-17T01:24:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 msgr2=0x5575383a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x5575383a3630 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8098f10 tx=0x7fb6d8098ac0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 shutdown_connections 2023-10-17T01:24:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5575383a5810 0x5575383a7c00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55753833ed90 0x5575383a3630 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55753833d830 0x55753839c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 --2- 172.21.15.45:0/3457547792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55753833e190 0x55753839cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 >> 172.21.15.45:0/3457547792 conn(0x55753829d070 msgr2=0x5575383451f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.760+0000 7fb6c9ffb700 0 client.26480 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.761+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 shutdown_connections 2023-10-17T01:24:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.761+0000 7fb6fbdd8b80 1 -- 172.21.15.45:0/3457547792 wait complete. 2023-10-17T01:24:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.761+0000 7fb6fbdd8b80 20 client.26480 trim_cache size 0 max 16384 2023-10-17T01:24:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:24:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:24:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:24:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:24:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6cc0a0ba0 msgr2=0x7fb6cc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6cc0a0ba0 0x7fb6cc0a3060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55753833f750 tx=0x7fb6d0007480 comp rx=0 tx=0).stop 2023-10-17T01:24:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0102910 msgr2=0x7fb6e019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0102910 0x7fb6e019a0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8007f10 tx=0x7fb6d800d3b0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 shutdown_connections 2023-10-17T01:24:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6cc0a0ba0 0x7fb6cc0a3060 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6e0102910 0x7fb6e019a0c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e0100d80 0x7fb6e0198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 --2- 172.21.15.45:0/2812799886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e0101f70 0x7fb6e01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.762+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 >> 172.21.15.45:0/2812799886 conn(0x7fb6e00fc2d0 msgr2=0x7fb6e0105500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.763+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 shutdown_connections 2023-10-17T01:24:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:11.763+0000 7fb6e83ff700 1 -- 172.21.15.45:0/2812799886 wait complete. 2023-10-17T01:24:13.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:12 smithi045 ceph-mon[98474]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 683 op/s 2023-10-17T01:24:13.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:12 smithi060 ceph-mon[134136]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 683 op/s 2023-10-17T01:24:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:12 smithi181 ceph-mon[134520]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 683 op/s 2023-10-17T01:24:15.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:14 smithi060 ceph-mon[134136]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 724 op/s 2023-10-17T01:24:15.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:14 smithi181 ceph-mon[134520]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 724 op/s 2023-10-17T01:24:15.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:14 smithi045 ceph-mon[98474]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 724 op/s 2023-10-17T01:24:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 545.0 s, 1066.0 tps, lat 88.332 ms stddev 100.188 2023-10-17T01:24:17.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:16 smithi060 ceph-mon[134136]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 830 op/s 2023-10-17T01:24:17.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:16 smithi181 ceph-mon[134520]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 830 op/s 2023-10-17T01:24:17.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:16 smithi045 ceph-mon[98474]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 830 op/s 2023-10-17T01:24:19.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:18 smithi045 ceph-mon[98474]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.38k op/s 2023-10-17T01:24:19.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:18 smithi060 ceph-mon[134136]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.38k op/s 2023-10-17T01:24:19.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:18 smithi181 ceph-mon[134520]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.38k op/s 2023-10-17T01:24:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 550.0 s, 1293.4 tps, lat 82.752 ms stddev 150.645 2023-10-17T01:24:21.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:20 smithi045 ceph-mon[98474]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-10-17T01:24:21.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:20 smithi060 ceph-mon[134136]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-10-17T01:24:21.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:20 smithi181 ceph-mon[134520]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-10-17T01:24:23.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:22 smithi045 ceph-mon[98474]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 1.82k op/s 2023-10-17T01:24:23.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:22 smithi060 ceph-mon[134136]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 1.82k op/s 2023-10-17T01:24:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:22 smithi181 ceph-mon[134520]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 1.82k op/s 2023-10-17T01:24:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 -- 172.21.15.45:0/4110681926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840fff20 msgr2=0x7fb8841003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/4110681926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840fff20 0x7fb8841003a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb870009990 tx=0x7fb870009ca0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 -- 172.21.15.45:0/4110681926 shutdown_connections 2023-10-17T01:24:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/4110681926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8841008e0 0x7fb884100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/4110681926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8840ff5c0 0x7fb8840ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/4110681926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840fff20 0x7fb8841003a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.004+0000 7fb88a9ff700 1 -- 172.21.15.45:0/4110681926 >> 172.21.15.45:0/4110681926 conn(0x7fb8840fb1a0 msgr2=0x7fb8840fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.005+0000 7fb88a9ff700 1 -- 172.21.15.45:0/4110681926 shutdown_connections 2023-10-17T01:24:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.005+0000 7fb88a9ff700 1 -- 172.21.15.45:0/4110681926 wait complete. 2023-10-17T01:24:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.006+0000 7fb88a9ff700 1 Processor -- start 2023-10-17T01:24:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 -- start start 2023-10-17T01:24:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8840ff5c0 0x7fb884198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8840fff20 0x7fb8841992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 0x7fb88419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88419a780 con 0x7fb8840ff5c0 2023-10-17T01:24:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88419a8f0 con 0x7fb8841008e0 2023-10-17T01:24:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.007+0000 7fb88a9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88419aab0 con 0x7fb8840fff20 2023-10-17T01:24:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 0x7fb88419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8840fff20 0x7fb8841992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 0x7fb88419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50498/0 (socket says 172.21.15.45:50498) 2023-10-17T01:24:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 -- 172.21.15.45:0/1819546744 learned_addr learned my addr 172.21.15.45:0/1819546744 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 -- 172.21.15.45:0/1819546744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8840fff20 msgr2=0x7fb8841992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8840fff20 0x7fb8841992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 -- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8840ff5c0 msgr2=0x7fb884198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8840ff5c0 0x7fb884198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.008+0000 7fb888f9c700 1 -- 172.21.15.45:0/1819546744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700095f0 con 0x7fb8841008e0 2023-10-17T01:24:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.009+0000 7fb8837fe700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8840fff20 0x7fb8841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:24:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.009+0000 7fb888f9c700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 0x7fb88419a070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007ee0 tx=0x7fb87c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.010+0000 7fb8817fa700 1 -- 172.21.15.45:0/1819546744 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c017070 con 0x7fb8841008e0 2023-10-17T01:24:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.010+0000 7fb8817fa700 1 -- 172.21.15.45:0/1819546744 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb87c00f040 con 0x7fb8841008e0 2023-10-17T01:24:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.010+0000 7fb8817fa700 1 -- 172.21.15.45:0/1819546744 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c005520 con 0x7fb8841008e0 2023-10-17T01:24:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.010+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88419ad90 con 0x7fb8841008e0 2023-10-17T01:24:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.010+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8841a14a0 con 0x7fb8841008e0 2023-10-17T01:24:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.017+0000 7fb8817fa700 1 -- 172.21.15.45:0/1819546744 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb87c005000 con 0x7fb8841008e0 2023-10-17T01:24:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.019+0000 7fb89e398b80 1 Processor -- start 2023-10-17T01:24:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 -- start start 2023-10-17T01:24:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb64c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb65790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65d90 0x55641bb66210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641babc9b0 con 0x55641bb65d90 2023-10-17T01:24:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641baba370 con 0x55641bb65190 2023-10-17T01:24:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb89e398b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bab5c60 con 0x55641bb64830 2023-10-17T01:24:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb8817fa700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8680a0d20 0x7fb8680a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb883fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb65790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb64c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb883fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb65790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50502/0 (socket says 172.21.15.45:50502) 2023-10-17T01:24:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.020+0000 7fb883fff700 1 -- 172.21.15.45:0/3118777504 learned_addr learned my addr 172.21.15.45:0/3118777504 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65d90 0x55641bb66210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8817fa700 1 -- 172.21.15.45:0/1819546744 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb87c0caea0 con 0x7fb8841008e0 2023-10-17T01:24:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 -- 172.21.15.45:0/3118777504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 msgr2=0x55641bb65790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb65790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 -- 172.21.15.45:0/3118777504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65d90 msgr2=0x55641bb66210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65d90 0x55641bb66210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb888f9c700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8680a0d20 0x7fb8680a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 -- 172.21.15.45:0/3118777504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55641b8e3d80 con 0x55641bb64830 2023-10-17T01:24:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb64c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb870000c00 tx=0x7fb87001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a27dba4f4eb492d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb87a7fc700 1 -- 172.21.15.45:0/3118777504 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870003950 con 0x55641bb64830 2023-10-17T01:24:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb87a7fc700 1 -- 172.21.15.45:0/3118777504 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8700050d0 con 0x55641bb64830 2023-10-17T01:24:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.021+0000 7fb87a7fc700 1 -- 172.21.15.45:0/3118777504 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87000f700 con 0x55641bb64830 2023-10-17T01:24:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.022+0000 7fb89e398b80 1 -- 172.21.15.45:0/3118777504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 msgr2=0x55641bb64c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.022+0000 7fb888f9c700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8680a0d20 0x7fb8680a31e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0981c0 tx=0x7fb87c004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.022+0000 7fb89e398b80 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb64c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb870000c00 tx=0x7fb87001a040 comp rx=0 tx=0).stop 2023-10-17T01:24:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 -- 172.21.15.45:0/3118777504 shutdown_connections 2023-10-17T01:24:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65d90 0x55641bb66210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb64c50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 --2- 172.21.15.45:0/3118777504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb65790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 -- 172.21.15.45:0/3118777504 >> 172.21.15.45:0/3118777504 conn(0x55641bac4070 msgr2=0x55641bb5f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 -- 172.21.15.45:0/3118777504 shutdown_connections 2023-10-17T01:24:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.023+0000 7fb89e398b80 1 -- 172.21.15.45:0/3118777504 wait complete. 2023-10-17T01:24:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.024+0000 7fb89e398b80 1 -- start start 2023-10-17T01:24:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.024+0000 7fb89e398b80 1 Processor -- start 2023-10-17T01:24:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb64830 0x55641bb76aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb76fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb7e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641ba524b0 con 0x55641bb64830 2023-10-17T01:24:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641ba52150 con 0x55641bb65190 2023-10-17T01:24:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641b9398c0 con 0x55641bb65d90 2023-10-17T01:24:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb89e398b80 10 client.0 _ll_register_callbacks cb 0x55641b99fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb883fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb76fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.025+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb7e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb7e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45762/0 (socket says 172.21.15.45:45762) 2023-10-17T01:24:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 -- 172.21.15.45:0/924409457 learned_addr learned my addr 172.21.15.45:0/924409457 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb64830 0x55641bb76aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 -- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 msgr2=0x55641bb76fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb76fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 -- 172.21.15.45:0/924409457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb64830 msgr2=0x55641bb76aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb64830 0x55641bb76aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.026+0000 7fb8837fe700 1 -- 172.21.15.45:0/924409457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb870020040 con 0x55641bb65d90 2023-10-17T01:24:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.027+0000 7fb8837fe700 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb7e020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb870005480 tx=0x7fb870005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c9a6e81cb4a7b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.027+0000 7fb8677fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.027+0000 7fb8677fe700 1 -- 172.21.15.45:0/924409457 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870003c20 con 0x55641bb65d90 2023-10-17T01:24:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.027+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55641bae0cb0 con 0x55641bb65d90 2023-10-17T01:24:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.027+0000 7fb8677fe700 1 -- 172.21.15.45:0/924409457 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb870003d80 con 0x55641bb65d90 2023-10-17T01:24:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.027+0000 7fb8677fe700 1 -- 172.21.15.45:0/924409457 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb87000f460 con 0x55641bb65d90 2023-10-17T01:24:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 10 client.26452 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55641b8ccc00 con 0x55641bb65d90 2023-10-17T01:24:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb8677fe700 1 -- 172.21.15.45:0/924409457 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55641bae0cb0 con 0x55641bb65d90 2023-10-17T01:24:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb8677fe700 1 -- 172.21.15.45:0/924409457 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb8700285f0 con 0x55641bb65d90 2023-10-17T01:24:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 10 client.26452 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 10 client.26452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 20 client.26452 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bb82410 0x55641bb84830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 4 client.26452 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:24:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.028+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55641bb33cd0 con 0x55641bb82410 2023-10-17T01:24:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.029+0000 7fb883fff700 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bb82410 0x55641bb84830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.029+0000 7fb883fff700 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bb82410 0x55641bb84830 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df13d84ca36f6292 server_cookie=27b7ea3f18089f40 in_seq=0 out_seq=0 2023-10-17T01:24:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.029+0000 7fb8677fe700 10 client.26452 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.030+0000 7fb8677fe700 1 -- 172.21.15.45:0/924409457 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55641bb33cd0 con 0x55641bb82410 2023-10-17T01:24:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.030+0000 7fb8677fe700 10 client.26452 handle_command_reply: tid=0 2023-10-17T01:24:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.030+0000 7fb89e398b80 1 client.26452 shutdown 2023-10-17T01:24:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 msgr2=0x55641bb7e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb7e020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb870005480 tx=0x7fb870005a00 comp rx=0 tx=0).stop 2023-10-17T01:24:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 shutdown_connections 2023-10-17T01:24:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb7e020 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb64830 0x55641bb76aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bb82410 0x55641bb84830 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 --2- 172.21.15.45:0/924409457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65190 0x55641bb76fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb8677fe700 0 client.26452 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.032+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 >> 172.21.15.45:0/924409457 conn(0x55641bac4070 msgr2=0x55641bb5ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.035+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 shutdown_connections 2023-10-17T01:24:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.035+0000 7fb89e398b80 1 -- 172.21.15.45:0/924409457 wait complete. 2023-10-17T01:24:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.036+0000 7fb89e398b80 20 client.26452 trim_cache size 0 max 16384 2023-10-17T01:24:25.060 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:24 smithi045 ceph-mon[98474]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2023-10-17T01:24:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.068+0000 7fb89e398b80 1 Processor -- start 2023-10-17T01:24:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.068+0000 7fb89e398b80 1 -- start start 2023-10-17T01:24:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.068+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb60360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.068+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65d90 0x55641bb5f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb89e398b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bb59990 con 0x55641bb65190 2023-10-17T01:24:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb89e398b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bb60150 con 0x55641bb65d90 2023-10-17T01:24:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb89e398b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bb77720 con 0x55641bb64830 2023-10-17T01:24:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb883fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb883fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb5b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50758/0 (socket says 172.21.15.45:50758) 2023-10-17T01:24:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.069+0000 7fb883fff700 1 -- 172.21.15.45:0/708250302 learned_addr learned my addr 172.21.15.45:0/708250302 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.071+0000 7fb883fff700 1 -- 172.21.15.45:0/708250302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 msgr2=0x55641bb60360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.071+0000 7fb883fff700 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb60360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.071+0000 7fb883fff700 1 -- 172.21.15.45:0/708250302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65d90 msgr2=0x55641bb5f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.071+0000 7fb883fff700 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65d90 0x55641bb5f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.071+0000 7fb883fff700 1 -- 172.21.15.45:0/708250302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55641b90a1f0 con 0x55641bb65190 2023-10-17T01:24:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.075+0000 7fb883fff700 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb5b460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55641bb85140 tx=0x7fb87401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87b1d95b3bcc7b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.076+0000 7fb87affd700 1 -- 172.21.15.45:0/708250302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874015040 con 0x55641bb65190 2023-10-17T01:24:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.076+0000 7fb87affd700 1 -- 172.21.15.45:0/708250302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb874017040 con 0x55641bb65190 2023-10-17T01:24:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.076+0000 7fb87affd700 1 -- 172.21.15.45:0/708250302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400f680 con 0x55641bb65190 2023-10-17T01:24:25.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:24 smithi060 ceph-mon[134136]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2023-10-17T01:24:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.080+0000 7fb89e398b80 1 -- 172.21.15.45:0/708250302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 msgr2=0x55641bb5b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.080+0000 7fb89e398b80 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb5b460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55641bb85140 tx=0x7fb87401c9b0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 -- 172.21.15.45:0/708250302 shutdown_connections 2023-10-17T01:24:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb65d90 0x55641bb5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb64830 0x55641bb60360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 --2- 172.21.15.45:0/708250302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb5b460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 -- 172.21.15.45:0/708250302 >> 172.21.15.45:0/708250302 conn(0x55641bac4070 msgr2=0x55641bb5ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 -- 172.21.15.45:0/708250302 shutdown_connections 2023-10-17T01:24:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.081+0000 7fb89e398b80 1 -- 172.21.15.45:0/708250302 wait complete. 2023-10-17T01:24:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.082+0000 7fb89e398b80 1 -- start start 2023-10-17T01:24:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.082+0000 7fb89e398b80 1 Processor -- start 2023-10-17T01:24:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb64830 0x55641bb7b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb7bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bb7a5a0 con 0x55641bb65190 2023-10-17T01:24:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bb7a710 con 0x55641bb64830 2023-10-17T01:24:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641bb7a8d0 con 0x55641bb65d90 2023-10-17T01:24:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb89e398b80 10 client.0 _ll_register_callbacks cb 0x7fb87400a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.083+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb64830 0x55641bb7b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb64830 0x55641bb7b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50524/0 (socket says 172.21.15.45:50524) 2023-10-17T01:24:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 -- 172.21.15.45:0/1893724675 learned_addr learned my addr 172.21.15.45:0/1893724675 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 -- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb64830 msgr2=0x55641bb7b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb64830 0x55641bb7b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 -- 172.21.15.45:0/1893724675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 msgr2=0x55641bb7bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb7bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.084+0000 7fb8837fe700 1 -- 172.21.15.45:0/1893724675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87400a930 con 0x55641bb65d90 2023-10-17T01:24:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.085+0000 7fb8837fe700 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb79f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55641bb7b570 tx=0x7fb870005240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2376a8247857efba server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.085+0000 7fb8677fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.086+0000 7fb8677fe700 1 -- 172.21.15.45:0/1893724675 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870005bc0 con 0x55641bb65d90 2023-10-17T01:24:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.086+0000 7fb8677fe700 1 -- 172.21.15.45:0/1893724675 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb870000e40 con 0x55641bb65d90 2023-10-17T01:24:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.087+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8700050d0 con 0x55641bb65d90 2023-10-17T01:24:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.087+0000 7fb8677fe700 1 -- 172.21.15.45:0/1893724675 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb870004e60 con 0x55641bb65d90 2023-10-17T01:24:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.087+0000 7fb89e398b80 10 client.26455 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.087+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55641b8e2250 con 0x55641bb65d90 2023-10-17T01:24:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb8677fe700 1 -- 172.21.15.45:0/1893724675 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8700050d0 con 0x55641bb65d90 2023-10-17T01:24:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb8677fe700 1 -- 172.21.15.45:0/1893724675 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb870049070 con 0x55641bb65d90 2023-10-17T01:24:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb89e398b80 10 client.26455 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb89e398b80 10 client.26455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb89e398b80 20 client.26455 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb89e398b80 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bbd4ab0 0x55641bbd6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb89e398b80 4 client.26455 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:24:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.089+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55641baaece0 con 0x55641bbd4ab0 2023-10-17T01:24:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.094+0000 7fb883fff700 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bbd4ab0 0x55641bbd6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.097+0000 7fb883fff700 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bbd4ab0 0x55641bbd6ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c047d2d2d5a7fe11 server_cookie=8e0b7e7ef10feaf5 in_seq=0 out_seq=0 2023-10-17T01:24:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.097+0000 7fb8677fe700 10 client.26455 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.113+0000 7fb8677fe700 1 -- 172.21.15.45:0/1893724675 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55641baaece0 con 0x55641bbd4ab0 2023-10-17T01:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.113+0000 7fb8677fe700 10 client.26455 handle_command_reply: tid=0 2023-10-17T01:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.113+0000 7fb89e398b80 1 client.26455 shutdown 2023-10-17T01:24:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 msgr2=0x55641bb79f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb79f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55641bb7b570 tx=0x7fb870005240 comp rx=0 tx=0).stop 2023-10-17T01:24:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 shutdown_connections 2023-10-17T01:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55641bb65d90 0x55641bb79f20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55641bbd4ab0 0x55641bbd6ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55641bb64830 0x55641bb7b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 --2- 172.21.15.45:0/1893724675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641bb65190 0x55641bb7bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 >> 172.21.15.45:0/1893724675 conn(0x55641bac4070 msgr2=0x55641bb6c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb8677fe700 0 client.26455 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 shutdown_connections 2023-10-17T01:24:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 1 -- 172.21.15.45:0/1893724675 wait complete. 2023-10-17T01:24:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.114+0000 7fb89e398b80 20 client.26455 trim_cache size 0 max 16384 2023-10-17T01:24:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:24:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:24:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:24:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:24:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8680a0d20 msgr2=0x7fb8680a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8680a0d20 0x7fb8680a31e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0981c0 tx=0x7fb87c004ad0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 msgr2=0x7fb88419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 0x7fb88419a070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007ee0 tx=0x7fb87c00d3b0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 shutdown_connections 2023-10-17T01:24:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb8680a0d20 0x7fb8680a31e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8841008e0 0x7fb88419a070 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8840ff5c0 0x7fb884198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 --2- 172.21.15.45:0/1819546744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8840fff20 0x7fb8841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.117+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 >> 172.21.15.45:0/1819546744 conn(0x7fb8840fb1a0 msgr2=0x7fb8840fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.118+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 shutdown_connections 2023-10-17T01:24:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:25.118+0000 7fb88a9ff700 1 -- 172.21.15.45:0/1819546744 wait complete. 2023-10-17T01:24:25.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:24 smithi181 ceph-mon[134520]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2023-10-17T01:24:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 555.0 s, 1665.1 tps, lat 59.782 ms stddev 69.270 2023-10-17T01:24:27.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:26 smithi045 ceph-mon[98474]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 1.75k op/s 2023-10-17T01:24:27.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:26 smithi060 ceph-mon[134136]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 1.75k op/s 2023-10-17T01:24:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:26 smithi181 ceph-mon[134520]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 1.75k op/s 2023-10-17T01:24:29.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:28 smithi060 ceph-mon[134136]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 MiB/s wr, 1.69k op/s 2023-10-17T01:24:29.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:28 smithi181 ceph-mon[134520]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 MiB/s wr, 1.69k op/s 2023-10-17T01:24:29.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:28 smithi045 ceph-mon[98474]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 MiB/s wr, 1.69k op/s 2023-10-17T01:24:30.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 560.0 s, 1458.8 tps, lat 68.082 ms stddev 74.338 2023-10-17T01:24:31.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:30 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:24:31.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:30 smithi045 ceph-mon[98474]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 1.16k op/s 2023-10-17T01:24:31.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:30 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:24:31.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:30 smithi060 ceph-mon[134136]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 1.16k op/s 2023-10-17T01:24:31.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:30 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:24:31.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:30 smithi181 ceph-mon[134520]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 1.16k op/s 2023-10-17T01:24:32.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:24:32.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:24:32.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:24:32.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:24:32.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:24:32.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:24:32.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:24:32.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:24:32.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:24:33.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:32 smithi045 ceph-mon[98474]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 674 op/s 2023-10-17T01:24:33.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:32 smithi060 ceph-mon[134136]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 674 op/s 2023-10-17T01:24:33.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:32 smithi181 ceph-mon[134520]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 674 op/s 2023-10-17T01:24:35.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:34 smithi060 ceph-mon[134136]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 732 op/s 2023-10-17T01:24:35.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:34 smithi181 ceph-mon[134520]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 732 op/s 2023-10-17T01:24:35.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:24:35.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:34 smithi045 ceph-mon[98474]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 732 op/s 2023-10-17T01:24:35.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 -- 172.21.15.45:0/3613280542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44108f00 msgr2=0x7fcb4410f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 --2- 172.21.15.45:0/3613280542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44108f00 0x7fcb4410f7b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009fc0 tx=0x7fcb3c00b330 comp rx=0 tx=0).stop 2023-10-17T01:24:35.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 -- 172.21.15.45:0/3613280542 shutdown_connections 2023-10-17T01:24:35.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 --2- 172.21.15.45:0/3613280542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44108f00 0x7fcb4410f7b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 --2- 172.21.15.45:0/3613280542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb44107370 0x7fcb44107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 --2- 172.21.15.45:0/3613280542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb441089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.600+0000 7fcb495ff700 1 -- 172.21.15.45:0/3613280542 >> 172.21.15.45:0/3613280542 conn(0x7fcb440762f0 msgr2=0x7fcb44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.601+0000 7fcb495ff700 1 -- 172.21.15.45:0/3613280542 shutdown_connections 2023-10-17T01:24:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.601+0000 7fcb495ff700 1 -- 172.21.15.45:0/3613280542 wait complete. 2023-10-17T01:24:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.604+0000 7fcb495ff700 1 Processor -- start 2023-10-17T01:24:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.604+0000 7fcb495ff700 1 -- start start 2023-10-17T01:24:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.604+0000 7fcb495ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44107370 0x7fcb4419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.604+0000 7fcb495ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb4419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.604+0000 7fcb495ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb44108f00 0x7fcb441a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.604+0000 7fcb495ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb441a4ba0 con 0x7fcb44107370 2023-10-17T01:24:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.605+0000 7fcb495ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440795a0 con 0x7fcb44108560 2023-10-17T01:24:35.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.605+0000 7fcb495ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44079730 con 0x7fcb44108f00 2023-10-17T01:24:35.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb4419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb4419d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58732/0 (socket says 172.21.15.45:58732) 2023-10-17T01:24:35.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 -- 172.21.15.45:0/4216108764 learned_addr learned my addr 172.21.15.45:0/4216108764 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:35.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 -- 172.21.15.45:0/4216108764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb44108f00 msgr2=0x7fcb441a45d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:24:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb44108f00 0x7fcb441a45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 -- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44107370 msgr2=0x7fcb4419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44107370 0x7fcb4419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.606+0000 7fcb427fc700 1 -- 172.21.15.45:0/4216108764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c009c40 con 0x7fcb44108560 2023-10-17T01:24:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.607+0000 7fcb427fc700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb4419d560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c9b0 tx=0x7fcb3400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.607+0000 7fcb3bfff700 1 -- 172.21.15.45:0/4216108764 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34007b00 con 0x7fcb44108560 2023-10-17T01:24:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.608+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb44079a10 con 0x7fcb44108560 2023-10-17T01:24:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.608+0000 7fcb3bfff700 1 -- 172.21.15.45:0/4216108764 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb34007c60 con 0x7fcb44108560 2023-10-17T01:24:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.608+0000 7fcb3bfff700 1 -- 172.21.15.45:0/4216108764 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34007b00 con 0x7fcb44108560 2023-10-17T01:24:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.609+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb44079f70 con 0x7fcb44108560 2023-10-17T01:24:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.612+0000 7fcb3bfff700 1 -- 172.21.15.45:0/4216108764 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fcb34018400 con 0x7fcb44108560 2023-10-17T01:24:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.614+0000 7fcb3bfff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb300a0c30 0x7fcb300a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.615+0000 7fcb3bfff700 1 -- 172.21.15.45:0/4216108764 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcb340cc1a0 con 0x7fcb44108560 2023-10-17T01:24:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.616+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb300a0c30 0x7fcb300a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.618+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb300a0c30 0x7fcb300a30f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcb441083c0 tx=0x7fcb2c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.621+0000 7fcb5cf68b80 1 Processor -- start 2023-10-17T01:24:35.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.622+0000 7fcb5cf68b80 1 -- start start 2023-10-17T01:24:35.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.623+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf571fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.624+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf572500 0x5633cf572d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.624+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf573730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.624+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf4c9a10 con 0x5633cf571ba0 2023-10-17T01:24:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.624+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf4c73d0 con 0x5633cf5732b0 2023-10-17T01:24:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.624+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf4c2cc0 con 0x5633cf572500 2023-10-17T01:24:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf571fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf571fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60750/0 (socket says 172.21.15.45:60750) 2023-10-17T01:24:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 -- 172.21.15.45:0/741477615 learned_addr learned my addr 172.21.15.45:0/741477615 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb437fe700 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf573730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb427fc700 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf572500 0x5633cf572d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 -- 172.21.15.45:0/741477615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf572500 msgr2=0x5633cf572d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf572500 0x5633cf572d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 -- 172.21.15.45:0/741477615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 msgr2=0x5633cf573730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf573730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb42ffd700 1 -- 172.21.15.45:0/741477615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633cf2f0de0 con 0x5633cf571ba0 2023-10-17T01:24:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.625+0000 7fcb427fc700 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf572500 0x5633cf572d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:24:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.626+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf571fc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00cc20 tx=0x7fcb2c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8fd924f3b614fd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:35.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.627+0000 7fcb397fa700 1 -- 172.21.15.45:0/741477615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00fba0 con 0x5633cf571ba0 2023-10-17T01:24:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.627+0000 7fcb397fa700 1 -- 172.21.15.45:0/741477615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2c00fd00 con 0x5633cf571ba0 2023-10-17T01:24:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.627+0000 7fcb397fa700 1 -- 172.21.15.45:0/741477615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00e5f0 con 0x5633cf571ba0 2023-10-17T01:24:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.628+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/741477615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 msgr2=0x5633cf571fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.628+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf571fc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00cc20 tx=0x7fcb2c00f000 comp rx=0 tx=0).stop 2023-10-17T01:24:35.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.629+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/741477615 shutdown_connections 2023-10-17T01:24:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.629+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf573730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.629+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf571fc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.629+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/741477615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf572500 0x5633cf572d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.629+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/741477615 >> 172.21.15.45:0/741477615 conn(0x5633cf4d10d0 msgr2=0x5633cf56c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:35.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.630+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/741477615 shutdown_connections 2023-10-17T01:24:35.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.630+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/741477615 wait complete. 2023-10-17T01:24:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.631+0000 7fcb5cf68b80 1 -- start start 2023-10-17T01:24:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.631+0000 7fcb5cf68b80 1 Processor -- start 2023-10-17T01:24:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf571ba0 0x5633cf57b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf57be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf582e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf45f510 con 0x5633cf572500 2023-10-17T01:24:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf45f1b0 con 0x5633cf571ba0 2023-10-17T01:24:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf346920 con 0x5633cf5732b0 2023-10-17T01:24:35.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb5cf68b80 10 client.0 _ll_register_callbacks cb 0x5633cf3aca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf571ba0 0x5633cf57b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf571ba0 0x5633cf57b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58750/0 (socket says 172.21.15.45:58750) 2023-10-17T01:24:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb42ffd700 1 -- 172.21.15.45:0/1441539541 learned_addr learned my addr 172.21.15.45:0/1441539541 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:35.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.632+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf582e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.633+0000 7fcb437fe700 1 -- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf571ba0 msgr2=0x5633cf57b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.633+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf571ba0 0x5633cf57b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.633+0000 7fcb437fe700 1 -- 172.21.15.45:0/1441539541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 msgr2=0x5633cf57be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.633+0000 7fcb427fc700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf57be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.634+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf57be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.634+0000 7fcb437fe700 1 -- 172.21.15.45:0/1441539541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c00c900 con 0x5633cf5732b0 2023-10-17T01:24:35.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.634+0000 7fcb427fc700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf57be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:24:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.635+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf582e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5633cf567320 tx=0x7fcb3c00bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e6818890eaf93c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.635+0000 7fcb2a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:35.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.635+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/1441539541 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c00e070 con 0x5633cf5732b0 2023-10-17T01:24:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.635+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/1441539541 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcb3c003a00 con 0x5633cf5732b0 2023-10-17T01:24:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.635+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633cf4edd10 con 0x5633cf5732b0 2023-10-17T01:24:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/1441539541 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb3c0077f0 con 0x5633cf5732b0 2023-10-17T01:24:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 10 client.26461 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:35.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5633cf2d9c60 con 0x5633cf5732b0 2023-10-17T01:24:35.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/1441539541 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633cf4edd10 con 0x5633cf5732b0 2023-10-17T01:24:35.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/1441539541 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fcb3c00b050 con 0x5633cf5732b0 2023-10-17T01:24:35.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 10 client.26461 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:35.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 10 client.26461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:35.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 20 client.26461 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf587740 0x5633cf589b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 4 client.26461 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.636+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633cf540d30 con 0x5633cf587740 2023-10-17T01:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.637+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf587740 0x5633cf589b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.639+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf587740 0x5633cf589b60 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701ed6d0b0d88505 server_cookie=6b5bddbd0b078600 in_seq=0 out_seq=0 2023-10-17T01:24:35.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.639+0000 7fcb2a7fc700 10 client.26461 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:35.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/1441539541 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5633cf540d30 con 0x5633cf587740 2023-10-17T01:24:35.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb2a7fc700 10 client.26461 handle_command_reply: tid=0 2023-10-17T01:24:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb5cf68b80 1 client.26461 shutdown 2023-10-17T01:24:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 msgr2=0x5633cf582e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf582e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5633cf567320 tx=0x7fcb3c00bd20 comp rx=0 tx=0).stop 2023-10-17T01:24:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 shutdown_connections 2023-10-17T01:24:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.640+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf582e70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf587740 0x5633cf589b60 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf571ba0 0x5633cf57b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1441539541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf57be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 >> 172.21.15.45:0/1441539541 conn(0x5633cf4d10d0 msgr2=0x5633cf56c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb2a7fc700 0 client.26461 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 shutdown_connections 2023-10-17T01:24:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.641+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1441539541 wait complete. 2023-10-17T01:24:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.642+0000 7fcb5cf68b80 20 client.26461 trim_cache size 0 max 16384 2023-10-17T01:24:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 565.0 s, 1358.6 tps, lat 74.349 ms stddev 85.799 2023-10-17T01:24:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 Processor -- start 2023-10-17T01:24:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 -- start start 2023-10-17T01:24:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf571ba0 0x5633cf579730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf589750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf57b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf4edd10 con 0x5633cf572500 2023-10-17T01:24:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf57cae0 con 0x5633cf5732b0 2023-10-17T01:24:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.671+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf5646a0 con 0x5633cf571ba0 2023-10-17T01:24:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf571ba0 0x5633cf579730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf571ba0 0x5633cf579730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41034/0 (socket says 172.21.15.45:41034) 2023-10-17T01:24:35.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb42ffd700 1 -- 172.21.15.45:0/1767324644 learned_addr learned my addr 172.21.15.45:0/1767324644 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf57b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb427fc700 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf589750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 -- 172.21.15.45:0/1767324644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf571ba0 msgr2=0x5633cf579730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf571ba0 0x5633cf579730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 -- 172.21.15.45:0/1767324644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 msgr2=0x5633cf589750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf589750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 -- 172.21.15.45:0/1767324644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633cf317250 con 0x5633cf5732b0 2023-10-17T01:24:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.672+0000 7fcb437fe700 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf57b8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b770 tx=0x7fcb3c009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b198779e5fb7b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb39ffb700 1 -- 172.21.15.45:0/1767324644 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c00e070 con 0x5633cf5732b0 2023-10-17T01:24:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb39ffb700 1 -- 172.21.15.45:0/1767324644 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb3c006000 con 0x5633cf5732b0 2023-10-17T01:24:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb39ffb700 1 -- 172.21.15.45:0/1767324644 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0246c0 con 0x5633cf5732b0 2023-10-17T01:24:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1767324644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 msgr2=0x5633cf57b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf57b8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b770 tx=0x7fcb3c009f10 comp rx=0 tx=0).stop 2023-10-17T01:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1767324644 shutdown_connections 2023-10-17T01:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf5732b0 0x5633cf57b8f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf571ba0 0x5633cf579730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/1767324644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf572500 0x5633cf589750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.673+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1767324644 >> 172.21.15.45:0/1767324644 conn(0x5633cf4d10d0 msgr2=0x5633cf57a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.674+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1767324644 shutdown_connections 2023-10-17T01:24:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.674+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/1767324644 wait complete. 2023-10-17T01:24:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.674+0000 7fcb5cf68b80 1 -- start start 2023-10-17T01:24:35.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.674+0000 7fcb5cf68b80 1 Processor -- start 2023-10-17T01:24:35.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf5d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 0x5633cf5d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf5d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf5d8080 con 0x5633cf571ba0 2023-10-17T01:24:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf5d81c0 con 0x5633cf572500 2023-10-17T01:24:35.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633cf5d8300 con 0x5633cf5732b0 2023-10-17T01:24:35.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb5cf68b80 10 client.0 _ll_register_callbacks cb 0x7fcb3c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:35.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 0x5633cf5d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 0x5633cf5d55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58770/0 (socket says 172.21.15.45:58770) 2023-10-17T01:24:35.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb42ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf5d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 -- 172.21.15.45:0/2142591747 learned_addr learned my addr 172.21.15.45:0/2142591747 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb437fe700 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf5d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 -- 172.21.15.45:0/2142591747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 msgr2=0x5633cf5d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf5d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 -- 172.21.15.45:0/2142591747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 msgr2=0x5633cf5d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf5d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.675+0000 7fcb427fc700 1 -- 172.21.15.45:0/2142591747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633cf2d9c60 con 0x5633cf572500 2023-10-17T01:24:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb427fc700 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 0x5633cf5d55f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcb3401f000 tx=0x7fcb34099170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6bf82e9535d233c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb2a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:24:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/2142591747 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340cb040 con 0x5633cf572500 2023-10-17T01:24:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/2142591747 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcb340997c0 con 0x5633cf572500 2023-10-17T01:24:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb2c00fd00 con 0x5633cf572500 2023-10-17T01:24:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/2142591747 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb3409d5f0 con 0x5633cf572500 2023-10-17T01:24:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb5cf68b80 10 client.26510 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:35.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5633cf2ef2b0 con 0x5633cf572500 2023-10-17T01:24:35.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/2142591747 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2c00fd00 con 0x5633cf572500 2023-10-17T01:24:35.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.676+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/2142591747 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fcb34098dd0 con 0x5633cf572500 2023-10-17T01:24:35.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb5cf68b80 10 client.26510 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb5cf68b80 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb5cf68b80 20 client.26510 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf5d9c50 0x5633cf5dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb5cf68b80 4 client.26510 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:24:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633cf4bbd40 con 0x5633cf5d9c50 2023-10-17T01:24:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf5d9c50 0x5633cf5dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb42ffd700 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf5d9c50 0x5633cf5dc070 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f5b27a3b5db6f9 server_cookie=1edf15f314b85435 in_seq=0 out_seq=0 2023-10-17T01:24:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.677+0000 7fcb2a7fc700 10 client.26510 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb2a7fc700 1 -- 172.21.15.45:0/2142591747 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5633cf4bbd40 con 0x5633cf5d9c50 2023-10-17T01:24:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb2a7fc700 10 client.26510 handle_command_reply: tid=0 2023-10-17T01:24:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 client.26510 shutdown 2023-10-17T01:24:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 msgr2=0x5633cf5d55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 0x5633cf5d55f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcb3401f000 tx=0x7fcb34099170 comp rx=0 tx=0).stop 2023-10-17T01:24:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 shutdown_connections 2023-10-17T01:24:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633cf5732b0 0x5633cf5d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633cf571ba0 0x5633cf5d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633cf5d9c50 0x5633cf5dc070 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 --2- 172.21.15.45:0/2142591747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633cf572500 0x5633cf5d55f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb2a7fc700 0 client.26510 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 >> 172.21.15.45:0/2142591747 conn(0x5633cf4d10d0 msgr2=0x5633cf579730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 shutdown_connections 2023-10-17T01:24:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 1 -- 172.21.15.45:0/2142591747 wait complete. 2023-10-17T01:24:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.678+0000 7fcb5cf68b80 20 client.26510 trim_cache size 0 max 16384 2023-10-17T01:24:35.695 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:24:35.696 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:24:35.696 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:24:35.696 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:24:35.696 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:24:35.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:24:35.697 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:24:35.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:24:35.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:24:35.697 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:24:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb300a0c30 msgr2=0x7fcb300a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb300a0c30 0x7fcb300a30f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcb441083c0 tx=0x7fcb2c005d30 comp rx=0 tx=0).stop 2023-10-17T01:24:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 msgr2=0x7fcb4419d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb4419d560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c9b0 tx=0x7fcb3400cd70 comp rx=0 tx=0).stop 2023-10-17T01:24:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 shutdown_connections 2023-10-17T01:24:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fcb300a0c30 0x7fcb300a30f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb44108f00 0x7fcb441a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb44107370 0x7fcb4419d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 --2- 172.21.15.45:0/4216108764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb44108560 0x7fcb4419d560 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 >> 172.21.15.45:0/4216108764 conn(0x7fcb440762f0 msgr2=0x7fcb44078370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 shutdown_connections 2023-10-17T01:24:35.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:24:35.679+0000 7fcb495ff700 1 -- 172.21.15.45:0/4216108764 wait complete. 2023-10-17T01:24:35.704 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:24:37.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:36 smithi060 ceph-mon[134136]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 630 op/s 2023-10-17T01:24:37.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:36 smithi181 ceph-mon[134520]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 630 op/s 2023-10-17T01:24:37.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:36 smithi045 ceph-mon[98474]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 630 op/s 2023-10-17T01:24:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 -- 172.21.15.45:0/2325650652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338101f60 msgr2=0x7f93381023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 --2- 172.21.15.45:0/2325650652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338101f60 0x7f93381023c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f932c009920 tx=0x7f932c009c30 comp rx=0 tx=0).stop 2023-10-17T01:24:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 -- 172.21.15.45:0/2325650652 shutdown_connections 2023-10-17T01:24:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 --2- 172.21.15.45:0/2325650652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9338102900 0x7f93381091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 --2- 172.21.15.45:0/2325650652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338100d70 0x7f9338101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 --2- 172.21.15.45:0/2325650652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338101f60 0x7f93381023c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 -- 172.21.15.45:0/2325650652 >> 172.21.15.45:0/2325650652 conn(0x7f93380fc2e0 msgr2=0x7f93380fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 -- 172.21.15.45:0/2325650652 shutdown_connections 2023-10-17T01:24:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.331+0000 7f933cbff700 1 -- 172.21.15.45:0/2325650652 wait complete. 2023-10-17T01:24:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.332+0000 7f933cbff700 1 Processor -- start 2023-10-17T01:24:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.332+0000 7f933cbff700 1 -- start start 2023-10-17T01:24:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933cbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9338100d70 0x7f9338198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933cbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 0x7f93381993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933cbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338102900 0x7f933819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933cbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933819a870 con 0x7f9338100d70 2023-10-17T01:24:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933cbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933819a9e0 con 0x7f9338102900 2023-10-17T01:24:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933cbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933819aba0 con 0x7f9338101f60 2023-10-17T01:24:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f9335d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 0x7f93381993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f933659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9338100d70 0x7f9338198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f9336d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338102900 0x7f933819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f9335d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 0x7f93381993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51948/0 (socket says 172.21.15.45:51948) 2023-10-17T01:24:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f9335d9b700 1 -- 172.21.15.45:0/4141018147 learned_addr learned my addr 172.21.15.45:0/4141018147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.333+0000 7f9335d9b700 1 -- 172.21.15.45:0/4141018147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338102900 msgr2=0x7f933819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f9335d9b700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338102900 0x7f933819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f9335d9b700 1 -- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9338100d70 msgr2=0x7f9338198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f9335d9b700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9338100d70 0x7f9338198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f9335d9b700 1 -- 172.21.15.45:0/4141018147 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280095f0 con 0x7f9338101f60 2023-10-17T01:24:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f9335d9b700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 0x7f93381993c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b810 tx=0x7f932c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f93277fe700 1 -- 172.21.15.45:0/4141018147 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c005240 con 0x7f9338101f60 2023-10-17T01:24:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f93277fe700 1 -- 172.21.15.45:0/4141018147 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c00f7b0 con 0x7f9338101f60 2023-10-17T01:24:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c009520 con 0x7f9338101f60 2023-10-17T01:24:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f93277fe700 1 -- 172.21.15.45:0/4141018147 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0216a0 con 0x7f9338101f60 2023-10-17T01:24:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.334+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93381a13b0 con 0x7f9338101f60 2023-10-17T01:24:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.341+0000 7f93277fe700 1 -- 172.21.15.45:0/4141018147 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f932c0037f0 con 0x7f9338101f60 2023-10-17T01:24:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.342+0000 7f935045fb80 1 Processor -- start 2023-10-17T01:24:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.343+0000 7f935045fb80 1 -- start start 2023-10-17T01:24:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f935045fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f935045fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8368790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f935045fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e8369210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f935045fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e82bf9b0 con 0x55b8e8368190 2023-10-17T01:24:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f935045fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e82bd370 con 0x55b8e8368d90 2023-10-17T01:24:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f935045fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e82b8c60 con 0x55b8e8367830 2023-10-17T01:24:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f9335d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e8369210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f933659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8368790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f9335d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e8369210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:35070/0 (socket says 172.21.15.45:35070) 2023-10-17T01:24:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f933659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8368790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40160/0 (socket says 172.21.15.45:40160) 2023-10-17T01:24:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f9335d9b700 1 -- 172.21.15.45:0/821842269 learned_addr learned my addr 172.21.15.45:0/821842269 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.344+0000 7f9336d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9336d9d700 1 -- 172.21.15.45:0/821842269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 msgr2=0x55b8e8369210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9336d9d700 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e8369210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9336d9d700 1 -- 172.21.15.45:0/821842269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 msgr2=0x55b8e8368790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9336d9d700 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8368790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9336d9d700 1 -- 172.21.15.45:0/821842269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8e80e6d80 con 0x55b8e8367830 2023-10-17T01:24:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f933659c700 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8368790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:24:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9336d9d700 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9330003dc0 tx=0x7f9330009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802d147b9e856d38 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f93277fe700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93200a0b20 0x7f93200a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9324ff9700 1 -- 172.21.15.45:0/821842269 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330004450 con 0x55b8e8367830 2023-10-17T01:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9324ff9700 1 -- 172.21.15.45:0/821842269 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933000f040 con 0x55b8e8367830 2023-10-17T01:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f9324ff9700 1 -- 172.21.15.45:0/821842269 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330015480 con 0x55b8e8367830 2023-10-17T01:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.345+0000 7f93277fe700 1 -- 172.21.15.45:0/4141018147 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f932c0cd3d0 con 0x7f9338101f60 2023-10-17T01:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f933659c700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93200a0b20 0x7f93200a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 -- 172.21.15.45:0/821842269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 msgr2=0x55b8e8367c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9330003dc0 tx=0x7f9330009c20 comp rx=0 tx=0).stop 2023-10-17T01:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f933659c700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93200a0b20 0x7f93200a2fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55b8e83675d0 tx=0x7f9328009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 -- 172.21.15.45:0/821842269 shutdown_connections 2023-10-17T01:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e8369210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 --2- 172.21.15.45:0/821842269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8368790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.346+0000 7f935045fb80 1 -- 172.21.15.45:0/821842269 >> 172.21.15.45:0/821842269 conn(0x55b8e82c7070 msgr2=0x55b8e83622c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.347+0000 7f935045fb80 1 -- 172.21.15.45:0/821842269 shutdown_connections 2023-10-17T01:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.347+0000 7f935045fb80 1 -- 172.21.15.45:0/821842269 wait complete. 2023-10-17T01:24:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.347+0000 7f935045fb80 1 -- start start 2023-10-17T01:24:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.347+0000 7f935045fb80 1 Processor -- start 2023-10-17T01:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8367830 0x55b8e83713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8371910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368d90 0x55b8e8378950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e82554b0 con 0x55b8e8368190 2023-10-17T01:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e8255150 con 0x55b8e8367830 2023-10-17T01:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e813c8c0 con 0x55b8e8368d90 2023-10-17T01:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f935045fb80 10 client.0 _ll_register_callbacks cb 0x55b8e81a2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f933659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8367830 0x55b8e83713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f9336d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368d90 0x55b8e8378950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f933659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8367830 0x55b8e83713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:35076/0 (socket says 172.21.15.45:35076) 2023-10-17T01:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f933659c700 1 -- 172.21.15.45:0/417527098 learned_addr learned my addr 172.21.15.45:0/417527098 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f9335d9b700 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8371910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f9336d9d700 1 -- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8367830 msgr2=0x55b8e83713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.348+0000 7f9336d9d700 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8367830 0x55b8e83713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.349+0000 7f9336d9d700 1 -- 172.21.15.45:0/417527098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 msgr2=0x55b8e8371910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.349+0000 7f9336d9d700 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8371910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.349+0000 7f9336d9d700 1 -- 172.21.15.45:0/417527098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9328009b50 con 0x55b8e8368d90 2023-10-17T01:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.349+0000 7f9336d9d700 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368d90 0x55b8e8378950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f93300095b0 tx=0x7f9330004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5911716d2c17341d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.349+0000 7f931dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.350+0000 7f931dffb700 1 -- 172.21.15.45:0/417527098 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330004900 con 0x55b8e8368d90 2023-10-17T01:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.350+0000 7f931dffb700 1 -- 172.21.15.45:0/417527098 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9330015480 con 0x55b8e8368d90 2023-10-17T01:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.350+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8e82e3cb0 con 0x55b8e8368d90 2023-10-17T01:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.351+0000 7f931dffb700 1 -- 172.21.15.45:0/417527098 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9330015660 con 0x55b8e8368d90 2023-10-17T01:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.351+0000 7f935045fb80 10 client.26479 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.351+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f9330003a20 con 0x55b8e8368d90 2023-10-17T01:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f931dffb700 1 -- 172.21.15.45:0/417527098 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8e82e3cb0 con 0x55b8e8368d90 2023-10-17T01:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f931dffb700 1 -- 172.21.15.45:0/417527098 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f9330025020 con 0x55b8e8368d90 2023-10-17T01:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f935045fb80 10 client.26479 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f935045fb80 10 client.26479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f935045fb80 20 client.26479 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f935045fb80 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e837d090 0x55b8e837f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f935045fb80 4 client.26479 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.352+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8e8336cd0 con 0x55b8e837d090 2023-10-17T01:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.353+0000 7f933659c700 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e837d090 0x55b8e837f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.354+0000 7f933659c700 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e837d090 0x55b8e837f4b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cb44b27ee9d809 server_cookie=ba211334ba66786f in_seq=0 out_seq=0 2023-10-17T01:24:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.354+0000 7f931dffb700 10 client.26479 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.354+0000 7f931dffb700 1 -- 172.21.15.45:0/417527098 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55b8e8336cd0 con 0x55b8e837d090 2023-10-17T01:24:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.354+0000 7f931dffb700 10 client.26479 handle_command_reply: tid=0 2023-10-17T01:24:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 client.26479 shutdown 2023-10-17T01:24:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368d90 msgr2=0x55b8e8378950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368d90 0x55b8e8378950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f93300095b0 tx=0x7f9330004250 comp rx=0 tx=0).stop 2023-10-17T01:24:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 shutdown_connections 2023-10-17T01:24:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368d90 0x55b8e8378950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e837d090 0x55b8e837f4b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8367830 0x55b8e83713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 --2- 172.21.15.45:0/417527098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368190 0x55b8e8371910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f931dffb700 0 client.26479 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 >> 172.21.15.45:0/417527098 conn(0x55b8e82c7070 msgr2=0x55b8e8361a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.355+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 shutdown_connections 2023-10-17T01:24:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.356+0000 7f935045fb80 1 -- 172.21.15.45:0/417527098 wait complete. 2023-10-17T01:24:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.356+0000 7f935045fb80 20 client.26479 trim_cache size 0 max 16384 2023-10-17T01:24:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.377+0000 7f935045fb80 1 Processor -- start 2023-10-17T01:24:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.377+0000 7f935045fb80 1 -- start start 2023-10-17T01:24:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.377+0000 7f935045fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.377+0000 7f935045fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368190 0x55b8e836f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.377+0000 7f935045fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368d90 0x55b8e83713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f933659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f935045fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e836fe80 con 0x55b8e8368d90 2023-10-17T01:24:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f935045fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e8363d40 con 0x55b8e8368190 2023-10-17T01:24:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f935045fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e835a110 con 0x55b8e8367830 2023-10-17T01:24:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f9335d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368190 0x55b8e836f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f933659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:51966/0 (socket says 172.21.15.45:51966) 2023-10-17T01:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f9335d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368190 0x55b8e836f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:35078/0 (socket says 172.21.15.45:35078) 2023-10-17T01:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f933659c700 1 -- 172.21.15.45:0/1832649654 learned_addr learned my addr 172.21.15.45:0/1832649654 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.378+0000 7f9336d9d700 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368d90 0x55b8e83713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f933659c700 1 -- 172.21.15.45:0/1832649654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368190 msgr2=0x55b8e836f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f933659c700 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368190 0x55b8e836f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f933659c700 1 -- 172.21.15.45:0/1832649654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368d90 msgr2=0x55b8e83713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f933659c700 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368d90 0x55b8e83713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f933659c700 1 -- 172.21.15.45:0/1832649654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8e80cfc00 con 0x55b8e8367830 2023-10-17T01:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f9336d9d700 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368d90 0x55b8e83713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:24:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f933659c700 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b8e837fdc0 tx=0x7f932801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb1bfd2808cf917 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.379+0000 7f93257fa700 1 -- 172.21.15.45:0/1832649654 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328025040 con 0x55b8e8367830 2023-10-17T01:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f93257fa700 1 -- 172.21.15.45:0/1832649654 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93280121d0 con 0x55b8e8367830 2023-10-17T01:24:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f93257fa700 1 -- 172.21.15.45:0/1832649654 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328017070 con 0x55b8e8367830 2023-10-17T01:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 -- 172.21.15.45:0/1832649654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 msgr2=0x55b8e8367c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b8e837fdc0 tx=0x7f932801a000 comp rx=0 tx=0).stop 2023-10-17T01:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 -- 172.21.15.45:0/1832649654 shutdown_connections 2023-10-17T01:24:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8368d90 0x55b8e83713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8367830 0x55b8e8367c10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 --2- 172.21.15.45:0/1832649654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368190 0x55b8e836f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.380+0000 7f935045fb80 1 -- 172.21.15.45:0/1832649654 >> 172.21.15.45:0/1832649654 conn(0x55b8e82c7070 msgr2=0x55b8e83700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.381+0000 7f935045fb80 1 -- 172.21.15.45:0/1832649654 shutdown_connections 2023-10-17T01:24:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.381+0000 7f935045fb80 1 -- 172.21.15.45:0/1832649654 wait complete. 2023-10-17T01:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.381+0000 7f935045fb80 1 -- start start 2023-10-17T01:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.381+0000 7f935045fb80 1 Processor -- start 2023-10-17T01:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8367830 0x55b8e83d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368190 0x55b8e83d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e83d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e83d6580 con 0x55b8e8367830 2023-10-17T01:24:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e83d66f0 con 0x55b8e8368d90 2023-10-17T01:24:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e83d68b0 con 0x55b8e8368190 2023-10-17T01:24:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.382+0000 7f935045fb80 10 client.0 _ll_register_callbacks cb 0x7f932c0d0d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e83d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e83d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:35082/0 (socket says 172.21.15.45:35082) 2023-10-17T01:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 -- 172.21.15.45:0/2963470941 learned_addr learned my addr 172.21.15.45:0/2963470941 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f933659c700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8367830 0x55b8e83d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9335d9b700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368190 0x55b8e83d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 -- 172.21.15.45:0/2963470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368190 msgr2=0x55b8e83d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368190 0x55b8e83d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 -- 172.21.15.45:0/2963470941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8367830 msgr2=0x55b8e83d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8367830 0x55b8e83d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.383+0000 7f9336d9d700 1 -- 172.21.15.45:0/2963470941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932801f420 con 0x55b8e8368d90 2023-10-17T01:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f9336d9d700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e83d5e90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55b8e835a2d0 tx=0x7f9330007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e680107a2976efd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f931dffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:24:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f931dffb700 1 -- 172.21.15.45:0/2963470941 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330007880 con 0x55b8e8368d90 2023-10-17T01:24:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f931dffb700 1 -- 172.21.15.45:0/2963470941 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9330009130 con 0x55b8e8368d90 2023-10-17T01:24:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f933000f040 con 0x55b8e8368d90 2023-10-17T01:24:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f931dffb700 1 -- 172.21.15.45:0/2963470941 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9330027af0 con 0x55b8e8368d90 2023-10-17T01:24:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f935045fb80 10 client.26528 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.384+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b8e810d1f0 con 0x55b8e8368d90 2023-10-17T01:24:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f931dffb700 1 -- 172.21.15.45:0/2963470941 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933000f040 con 0x55b8e8368d90 2023-10-17T01:24:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f931dffb700 1 -- 172.21.15.45:0/2963470941 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f933003e050 con 0x55b8e8368d90 2023-10-17T01:24:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f935045fb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f935045fb80 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f935045fb80 20 client.26528 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f935045fb80 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e83d7e30 0x55b8e83da250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f935045fb80 4 client.26528 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:24:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8e82b1ce0 con 0x55b8e83d7e30 2023-10-17T01:24:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.385+0000 7f933659c700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e83d7e30 0x55b8e83da250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.386+0000 7f931dffb700 10 client.26528 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.386+0000 7f933659c700 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e83d7e30 0x55b8e83da250 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21472ef50bc3987 server_cookie=e9567c5734a437a1 in_seq=0 out_seq=0 2023-10-17T01:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f931dffb700 1 -- 172.21.15.45:0/2963470941 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8e82b1ce0 con 0x55b8e83d7e30 2023-10-17T01:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f931dffb700 10 client.26528 handle_command_reply: tid=0 2023-10-17T01:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 client.26528 shutdown 2023-10-17T01:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 msgr2=0x55b8e83d5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e83d5e90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55b8e835a2d0 tx=0x7f9330007d10 comp rx=0 tx=0).stop 2023-10-17T01:24:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 shutdown_connections 2023-10-17T01:24:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b8e83d7e30 0x55b8e83da250 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b8e8368d90 0x55b8e83d5e90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b8e8367830 0x55b8e83d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 --2- 172.21.15.45:0/2963470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b8e8368190 0x55b8e83d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f931dffb700 0 client.26528 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 >> 172.21.15.45:0/2963470941 conn(0x55b8e82c7070 msgr2=0x55b8e836f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 shutdown_connections 2023-10-17T01:24:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.400+0000 7f935045fb80 1 -- 172.21.15.45:0/2963470941 wait complete. 2023-10-17T01:24:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f935045fb80 20 client.26528 trim_cache size 0 max 16384 2023-10-17T01:24:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:24:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:24:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:24:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:24:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93200a0b20 msgr2=0x7f93200a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93200a0b20 0x7f93200a2fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55b8e83675d0 tx=0x7f9328009310 comp rx=0 tx=0).stop 2023-10-17T01:24:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 msgr2=0x7f93381993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 0x7f93381993c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b810 tx=0x7f932c00b840 comp rx=0 tx=0).stop 2023-10-17T01:24:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 shutdown_connections 2023-10-17T01:24:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93200a0b20 0x7f93200a2fe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9338102900 0x7f933819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9338100d70 0x7f9338198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 --2- 172.21.15.45:0/4141018147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9338101f60 0x7f93381993c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 >> 172.21.15.45:0/4141018147 conn(0x7f93380fc2e0 msgr2=0x7f93380fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 shutdown_connections 2023-10-17T01:24:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:38.401+0000 7f933cbff700 1 -- 172.21.15.45:0/4141018147 wait complete. 2023-10-17T01:24:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:38 smithi045 ceph-mon[98474]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 689 op/s 2023-10-17T01:24:39.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:38 smithi060 ceph-mon[134136]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 689 op/s 2023-10-17T01:24:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:38 smithi181 ceph-mon[134520]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 MiB/s wr, 689 op/s 2023-10-17T01:24:40.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 570.0 s, 1445.6 tps, lat 69.230 ms stddev 83.200 2023-10-17T01:24:41.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:40 smithi045 ceph-mon[98474]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 642 op/s 2023-10-17T01:24:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:40 smithi060 ceph-mon[134136]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 642 op/s 2023-10-17T01:24:41.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:40 smithi181 ceph-mon[134520]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 642 op/s 2023-10-17T01:24:43.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:42 smithi045 ceph-mon[98474]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 612 op/s 2023-10-17T01:24:43.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:42 smithi060 ceph-mon[134136]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 612 op/s 2023-10-17T01:24:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:42 smithi181 ceph-mon[134520]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 612 op/s 2023-10-17T01:24:45.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:44 smithi045 ceph-mon[98474]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 663 op/s 2023-10-17T01:24:45.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:44 smithi060 ceph-mon[134136]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 663 op/s 2023-10-17T01:24:45.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:44 smithi181 ceph-mon[134520]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 663 op/s 2023-10-17T01:24:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 575.0 s, 1672.8 tps, lat 59.452 ms stddev 68.165 2023-10-17T01:24:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:46 smithi045 ceph-mon[98474]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 560 op/s 2023-10-17T01:24:47.084 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:46 smithi060 ceph-mon[134136]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 560 op/s 2023-10-17T01:24:47.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:46 smithi181 ceph-mon[134520]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 560 op/s 2023-10-17T01:24:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:48 smithi045 ceph-mon[98474]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 682 op/s 2023-10-17T01:24:49.085 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:48 smithi060 ceph-mon[134136]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 682 op/s 2023-10-17T01:24:49.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:48 smithi181 ceph-mon[134520]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 682 op/s 2023-10-17T01:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 580.0 s, 1087.4 tps, lat 88.798 ms stddev 102.792 2023-10-17T01:24:51.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:50 smithi045 ceph-mon[98474]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 702 op/s 2023-10-17T01:24:51.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:50 smithi060 ceph-mon[134136]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 702 op/s 2023-10-17T01:24:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:50 smithi181 ceph-mon[134520]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 702 op/s 2023-10-17T01:24:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 -- 172.21.15.45:0/2103509184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 msgr2=0x7f58d81091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 --2- 172.21.15.45:0/2103509184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d81091f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b0a0 tx=0x7f58d000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 -- 172.21.15.45:0/2103509184 shutdown_connections 2023-10-17T01:24:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 --2- 172.21.15.45:0/2103509184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d81091f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 --2- 172.21.15.45:0/2103509184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8100db0 0x7f58d81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 --2- 172.21.15.45:0/2103509184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8101fa0 0x7f58d8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 -- 172.21.15.45:0/2103509184 >> 172.21.15.45:0/2103509184 conn(0x7f58d80fc300 msgr2=0x7f58d80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 -- 172.21.15.45:0/2103509184 shutdown_connections 2023-10-17T01:24:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.602+0000 7f58dcfff700 1 -- 172.21.15.45:0/2103509184 wait complete. 2023-10-17T01:24:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.603+0000 7f58dcfff700 1 Processor -- start 2023-10-17T01:24:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 -- start start 2023-10-17T01:24:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 0x7f58d8073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8101fa0 0x7f58d8073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d8073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8074ce0 con 0x7f58d8101fa0 2023-10-17T01:24:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8074e50 con 0x7f58d8100db0 2023-10-17T01:24:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58dcfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8075010 con 0x7f58d8102940 2023-10-17T01:24:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58d659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 0x7f58d8073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58d659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 0x7f58d8073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56124/0 (socket says 172.21.15.45:56124) 2023-10-17T01:24:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58d659c700 1 -- 172.21.15.45:0/965756622 learned_addr learned my addr 172.21.15.45:0/965756622 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d8073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.604+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8101fa0 0x7f58d8073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d659c700 1 -- 172.21.15.45:0/965756622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 msgr2=0x7f58d8073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d659c700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d8073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d659c700 1 -- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8101fa0 msgr2=0x7f58d8073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d659c700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8101fa0 0x7f58d8073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d659c700 1 -- 172.21.15.45:0/965756622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009d00 con 0x7f58d8100db0 2023-10-17T01:24:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8101fa0 0x7f58d8073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:24:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d8073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:24:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58d659c700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 0x7f58d8073140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58c800e9a0 tx=0x7f58c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58c77fe700 1 -- 172.21.15.45:0/965756622 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8009790 con 0x7f58d8100db0 2023-10-17T01:24:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d81a5530 con 0x7f58d8100db0 2023-10-17T01:24:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58c77fe700 1 -- 172.21.15.45:0/965756622 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c80044e0 con 0x7f58d8100db0 2023-10-17T01:24:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58c77fe700 1 -- 172.21.15.45:0/965756622 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8003d20 con 0x7f58d8100db0 2023-10-17T01:24:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.605+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d81a5a60 con 0x7f58d8100db0 2023-10-17T01:24:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.612+0000 7f58c77fe700 1 -- 172.21.15.45:0/965756622 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f58c8003e80 con 0x7f58d8100db0 2023-10-17T01:24:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 Processor -- start 2023-10-17T01:24:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 -- start start 2023-10-17T01:24:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979be210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572979149b0 con 0x5572979bd190 2023-10-17T01:24:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557297912370 con 0x5572979bc830 2023-10-17T01:24:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.613+0000 7f58f08edb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55729790dc60 con 0x5572979bdd90 2023-10-17T01:24:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.614+0000 7f58d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.614+0000 7f58d659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.614+0000 7f58d659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979bd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45488/0 (socket says 172.21.15.45:45488) 2023-10-17T01:24:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.614+0000 7f58d659c700 1 -- 172.21.15.45:0/162455790 learned_addr learned my addr 172.21.15.45:0/162455790 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.614+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.615+0000 7f58c77fe700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f58c00a0c20 0x7f58c00a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.616+0000 7f58c77fe700 1 -- 172.21.15.45:0/965756622 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58c8014070 con 0x7f58d8100db0 2023-10-17T01:24:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.616+0000 7f58d5d9b700 1 -- 172.21.15.45:0/162455790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 msgr2=0x5572979bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.616+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979bcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.616+0000 7f58d5d9b700 1 -- 172.21.15.45:0/162455790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 msgr2=0x5572979bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.616+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.616+0000 7f58d5d9b700 1 -- 172.21.15.45:0/162455790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55729773bd80 con 0x5572979bdd90 2023-10-17T01:24:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.617+0000 7f58d659c700 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:24:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.617+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979be210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009c90 tx=0x7f58cc0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4d2cb9ea323bdfc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.618+0000 7f58c4ff9700 1 -- 172.21.15.45:0/162455790 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc011c00 con 0x5572979bdd90 2023-10-17T01:24:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.618+0000 7f58c4ff9700 1 -- 172.21.15.45:0/162455790 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58cc011d60 con 0x5572979bdd90 2023-10-17T01:24:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.618+0000 7f58c4ff9700 1 -- 172.21.15.45:0/162455790 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00f730 con 0x5572979bdd90 2023-10-17T01:24:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.618+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f58c00a0c20 0x7f58c00a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 -- 172.21.15.45:0/162455790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 msgr2=0x5572979be210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979be210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009c90 tx=0x7f58cc0113e0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 -- 172.21.15.45:0/162455790 shutdown_connections 2023-10-17T01:24:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979be210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 --2- 172.21.15.45:0/162455790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 -- 172.21.15.45:0/162455790 >> 172.21.15.45:0/162455790 conn(0x55729791c070 msgr2=0x5572979b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 -- 172.21.15.45:0/162455790 shutdown_connections 2023-10-17T01:24:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.619+0000 7f58f08edb80 1 -- 172.21.15.45:0/162455790 wait complete. 2023-10-17T01:24:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.620+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f58c00a0c20 0x7f58c00a30e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b780 tx=0x7f58d000b3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.620+0000 7f58f08edb80 1 -- start start 2023-10-17T01:24:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.620+0000 7f58f08edb80 1 Processor -- start 2023-10-17T01:24:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979c6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572978aa4b0 con 0x5572979bd190 2023-10-17T01:24:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572978aa150 con 0x5572979bc830 2023-10-17T01:24:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572977918c0 con 0x5572979bdd90 2023-10-17T01:24:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58f08edb80 10 client.0 _ll_register_callbacks cb 0x5572977f7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54866/0 (socket says 172.21.15.45:54866) 2023-10-17T01:24:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 -- 172.21.15.45:0/3843551933 learned_addr learned my addr 172.21.15.45:0/3843551933 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d659c700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 -- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 msgr2=0x5572979c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 -- 172.21.15.45:0/3843551933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 msgr2=0x5572979c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d6d9d700 1 -- 172.21.15.45:0/3843551933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d81a5a60 con 0x5572979bdd90 2023-10-17T01:24:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.621+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979cd910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b0a0 tx=0x7f58d00099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5128446390a4091b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58bdffb700 1 -- 172.21.15.45:0/3843551933 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000e040 con 0x5572979bdd90 2023-10-17T01:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557297938cb0 con 0x5572979bdd90 2023-10-17T01:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58bdffb700 1 -- 172.21.15.45:0/3843551933 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58d0004e10 con 0x5572979bdd90 2023-10-17T01:24:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58bdffb700 1 -- 172.21.15.45:0/3843551933 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58d0007880 con 0x5572979bdd90 2023-10-17T01:24:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58f08edb80 10 client.26497 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.622+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f58cc0098f0 con 0x5572979bdd90 2023-10-17T01:24:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.623+0000 7f58bdffb700 1 -- 172.21.15.45:0/3843551933 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557297938cb0 con 0x5572979bdd90 2023-10-17T01:24:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.623+0000 7f58bdffb700 1 -- 172.21.15.45:0/3843551933 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f58d0020ac0 con 0x5572979bdd90 2023-10-17T01:24:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.623+0000 7f58f08edb80 10 client.26497 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.623+0000 7f58f08edb80 10 client.26497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.623+0000 7f58f08edb80 20 client.26497 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.624+0000 7f58f08edb80 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5572979d2110 0x5572979d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.624+0000 7f58f08edb80 4 client.26497 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:24:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.624+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55729798bcd0 con 0x5572979d2110 2023-10-17T01:24:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.624+0000 7f58d659c700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5572979d2110 0x5572979d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.624+0000 7f58bdffb700 10 client.26497 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.624+0000 7f58d659c700 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5572979d2110 0x5572979d4530 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6821f95a7c6d16b8 server_cookie=ac4cc26795ade658 in_seq=0 out_seq=0 2023-10-17T01:24:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.626+0000 7f58bdffb700 1 -- 172.21.15.45:0/3843551933 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55729798bcd0 con 0x5572979d2110 2023-10-17T01:24:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.626+0000 7f58bdffb700 10 client.26497 handle_command_reply: tid=0 2023-10-17T01:24:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.626+0000 7f58f08edb80 1 client.26497 shutdown 2023-10-17T01:24:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 msgr2=0x5572979cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979cd910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b0a0 tx=0x7f58d00099e0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 shutdown_connections 2023-10-17T01:24:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bdd90 0x5572979cd910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5572979d2110 0x5572979d4530 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bc830 0x5572979c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 --2- 172.21.15.45:0/3843551933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bd190 0x5572979c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58bdffb700 0 client.26497 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.627+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 >> 172.21.15.45:0/3843551933 conn(0x55729791c070 msgr2=0x5572979b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.628+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 shutdown_connections 2023-10-17T01:24:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.628+0000 7f58f08edb80 1 -- 172.21.15.45:0/3843551933 wait complete. 2023-10-17T01:24:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.629+0000 7f58f08edb80 20 client.26497 trim_cache size 0 max 16384 2023-10-17T01:24:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.653+0000 7f58f08edb80 1 Processor -- start 2023-10-17T01:24:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.653+0000 7f58f08edb80 1 -- start start 2023-10-17T01:24:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.653+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.654+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.654+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.654+0000 7f58f08edb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572979bdbf0 con 0x5572979bdd90 2023-10-17T01:24:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.654+0000 7f58f08edb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55729798bcd0 con 0x5572979bd190 2023-10-17T01:24:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.654+0000 7f58f08edb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572979c7580 con 0x5572979bc830 2023-10-17T01:24:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.655+0000 7f58d5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.655+0000 7f58d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.655+0000 7f58d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54868/0 (socket says 172.21.15.45:54868) 2023-10-17T01:24:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.655+0000 7f58d5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:56164/0 (socket says 172.21.15.45:56164) 2023-10-17T01:24:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.655+0000 7f58d659c700 1 -- 172.21.15.45:0/2583646963 learned_addr learned my addr 172.21.15.45:0/2583646963 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.655+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58d659c700 1 -- 172.21.15.45:0/2583646963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 msgr2=0x5572979c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58d659c700 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58d659c700 1 -- 172.21.15.45:0/2583646963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 msgr2=0x5572979c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58d659c700 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58d659c700 1 -- 172.21.15.45:0/2583646963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557297724c00 con 0x5572979bc830 2023-10-17T01:24:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58d659c700 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c5a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f58c8096460 tx=0x7f58c80c9d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63df04c57dea2047 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58c57fa700 1 -- 172.21.15.45:0/2583646963 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8099350 con 0x5572979bc830 2023-10-17T01:24:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58c57fa700 1 -- 172.21.15.45:0/2583646963 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c809a0f0 con 0x5572979bc830 2023-10-17T01:24:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.656+0000 7f58c57fa700 1 -- 172.21.15.45:0/2583646963 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8097bc0 con 0x5572979bc830 2023-10-17T01:24:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 -- 172.21.15.45:0/2583646963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 msgr2=0x5572979c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c5a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f58c8096460 tx=0x7f58c80c9d70 comp rx=0 tx=0).stop 2023-10-17T01:24:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 -- 172.21.15.45:0/2583646963 shutdown_connections 2023-10-17T01:24:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c5a40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 --2- 172.21.15.45:0/2583646963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.657+0000 7f58f08edb80 1 -- 172.21.15.45:0/2583646963 >> 172.21.15.45:0/2583646963 conn(0x55729791c070 msgr2=0x5572979c5060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.658+0000 7f58f08edb80 1 -- 172.21.15.45:0/2583646963 shutdown_connections 2023-10-17T01:24:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.658+0000 7f58f08edb80 1 -- 172.21.15.45:0/2583646963 wait complete. 2023-10-17T01:24:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.658+0000 7f58f08edb80 1 -- start start 2023-10-17T01:24:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 Processor -- start 2023-10-17T01:24:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572979cb170 con 0x5572979bdd90 2023-10-17T01:24:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572979cb2b0 con 0x5572979bd190 2023-10-17T01:24:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.659+0000 7f58f08edb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572979cb470 con 0x5572979bc830 2023-10-17T01:24:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.660+0000 7f58d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.660+0000 7f58d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54884/0 (socket says 172.21.15.45:54884) 2023-10-17T01:24:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.660+0000 7f58d659c700 1 -- 172.21.15.45:0/4153968741 learned_addr learned my addr 172.21.15.45:0/4153968741 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:24:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.660+0000 7f58f08edb80 10 client.0 _ll_register_callbacks cb 0x7f58d000cdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:24:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.660+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.661+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d659c700 1 -- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 msgr2=0x5572979c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d659c700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d659c700 1 -- 172.21.15.45:0/4153968741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 msgr2=0x5572979cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:24:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d659c700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d659c700 1 -- 172.21.15.45:0/4153968741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d81a5530 con 0x5572979bc830 2023-10-17T01:24:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:24:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.662+0000 7f58d6d9d700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:24:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.663+0000 7f58d659c700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c7a50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c5b0 tx=0x7f58c800ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82dc2e6f7f6acc81 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:24:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.663+0000 7f58bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:24:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.664+0000 7f58bdffb700 1 -- 172.21.15.45:0/4153968741 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c809d820 con 0x5572979bc830 2023-10-17T01:24:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.664+0000 7f58bdffb700 1 -- 172.21.15.45:0/4153968741 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58c80b3070 con 0x5572979bc830 2023-10-17T01:24:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.664+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58cc011d60 con 0x5572979bc830 2023-10-17T01:24:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.664+0000 7f58bdffb700 1 -- 172.21.15.45:0/4153968741 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58c8097ba0 con 0x5572979bc830 2023-10-17T01:24:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.664+0000 7f58f08edb80 10 client.26503 fetch_fsmap learned FSMap version 40 2023-10-17T01:24:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.664+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5572977621f0 con 0x5572979bc830 2023-10-17T01:24:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58bdffb700 1 -- 172.21.15.45:0/4153968741 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c807e020 con 0x5572979bc830 2023-10-17T01:24:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58bdffb700 1 -- 172.21.15.45:0/4153968741 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f58c8099c50 con 0x5572979bc830 2023-10-17T01:24:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58f08edb80 10 client.26503 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:24:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58f08edb80 10 client.26503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:24:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58f08edb80 20 client.26503 populate_metadata read hostname 'smithi045' 2023-10-17T01:24:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58f08edb80 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557297a358b0 0x557297a37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:24:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58f08edb80 4 client.26503 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:24:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557297906ce0 con 0x557297a358b0 2023-10-17T01:24:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.665+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557297a358b0 0x557297a37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:24:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.666+0000 7f58bdffb700 10 client.26503 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.666+0000 7f58d5d9b700 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557297a358b0 0x557297a37ca0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c6c71c89593d272 server_cookie=ecba575348907bf3 in_seq=0 out_seq=0 2023-10-17T01:24:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.682+0000 7f58bdffb700 1 -- 172.21.15.45:0/4153968741 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557297906ce0 con 0x557297a358b0 2023-10-17T01:24:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.682+0000 7f58bdffb700 10 client.26503 handle_command_reply: tid=0 2023-10-17T01:24:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.682+0000 7f58f08edb80 1 client.26503 shutdown 2023-10-17T01:24:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.683+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 msgr2=0x5572979c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.683+0000 7f58f08edb80 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c7a50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c5b0 tx=0x7f58c800ccd0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 shutdown_connections 2023-10-17T01:24:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572979bdd90 0x5572979cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5572979bc830 0x5572979c7a50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557297a358b0 0x557297a37ca0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 --2- 172.21.15.45:0/4153968741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5572979bd190 0x5572979c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 >> 172.21.15.45:0/4153968741 conn(0x55729791c070 msgr2=0x5572979c41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58bdffb700 0 client.26503 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:24:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 shutdown_connections 2023-10-17T01:24:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.684+0000 7f58f08edb80 1 -- 172.21.15.45:0/4153968741 wait complete. 2023-10-17T01:24:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.685+0000 7f58f08edb80 20 client.26503 trim_cache size 0 max 16384 2023-10-17T01:24:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:24:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:24:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:24:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:24:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f58c00a0c20 msgr2=0x7f58c00a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f58c00a0c20 0x7f58c00a30e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b780 tx=0x7f58d000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 msgr2=0x7f58d8073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:24:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 0x7f58d8073140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58c800e9a0 tx=0x7f58c800ecb0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 shutdown_connections 2023-10-17T01:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f58c00a0c20 0x7f58c00a30e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58d8102940 0x7f58d8073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d8100db0 0x7f58d8073140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 --2- 172.21.15.45:0/965756622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58d8101fa0 0x7f58d8073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.686+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 >> 172.21.15.45:0/965756622 conn(0x7f58d80fc300 msgr2=0x7f58d8105400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.687+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 shutdown_connections 2023-10-17T01:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:24:51.687+0000 7f58dcfff700 1 -- 172.21.15.45:0/965756622 wait complete. 2023-10-17T01:24:53.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:52 smithi060 ceph-mon[134136]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 674 op/s 2023-10-17T01:24:53.163 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:52 smithi181 ceph-mon[134520]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 674 op/s 2023-10-17T01:24:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:52 smithi045 ceph-mon[98474]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 674 op/s 2023-10-17T01:24:55.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:54 smithi045 ceph-mon[98474]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 779 op/s 2023-10-17T01:24:55.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:54 smithi060 ceph-mon[134136]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 779 op/s 2023-10-17T01:24:55.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:54 smithi181 ceph-mon[134520]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 779 op/s 2023-10-17T01:24:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 585.0 s, 917.0 tps, lat 112.345 ms stddev 155.112 2023-10-17T01:24:57.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:56 smithi045 ceph-mon[98474]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 709 op/s 2023-10-17T01:24:57.081 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:56 smithi060 ceph-mon[134136]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 709 op/s 2023-10-17T01:24:57.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:56 smithi181 ceph-mon[134520]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 709 op/s 2023-10-17T01:24:59.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:24:58 smithi045 ceph-mon[98474]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 794 op/s 2023-10-17T01:24:59.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:24:58 smithi060 ceph-mon[134136]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 794 op/s 2023-10-17T01:24:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:24:58 smithi181 ceph-mon[134520]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 794 op/s 2023-10-17T01:25:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 590.0 s, 1413.4 tps, lat 70.829 ms stddev 80.248 2023-10-17T01:25:01.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:00 smithi060 ceph-mon[134136]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 839 op/s 2023-10-17T01:25:01.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:00 smithi181 ceph-mon[134520]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 839 op/s 2023-10-17T01:25:01.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:00 smithi045 ceph-mon[98474]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 839 op/s 2023-10-17T01:25:03.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:02 smithi181 ceph-mon[134520]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 786 op/s 2023-10-17T01:25:03.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:02 smithi045 ceph-mon[98474]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 786 op/s 2023-10-17T01:25:03.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:02 smithi060 ceph-mon[134136]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 786 op/s 2023-10-17T01:25:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.928+0000 7fa96e1ff700 1 -- 172.21.15.45:0/429998597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa968106a30 msgr2=0x7fa968108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.928+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/429998597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa968106a30 0x7fa968108ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa960009990 tx=0x7fa960009ca0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.929+0000 7fa96e1ff700 1 -- 172.21.15.45:0/429998597 shutdown_connections 2023-10-17T01:25:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.929+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/429998597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa968106a30 0x7fa968108ea0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.929+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/429998597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9681017d0 0x7fa968103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.929+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/429998597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 0x7fa9681064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.929+0000 7fa96e1ff700 1 -- 172.21.15.45:0/429998597 >> 172.21.15.45:0/429998597 conn(0x7fa9680fb1a0 msgr2=0x7fa9680fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.929+0000 7fa96e1ff700 1 -- 172.21.15.45:0/429998597 shutdown_connections 2023-10-17T01:25:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.930+0000 7fa96e1ff700 1 -- 172.21.15.45:0/429998597 wait complete. 2023-10-17T01:25:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.932+0000 7fa96e1ff700 1 Processor -- start 2023-10-17T01:25:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.933+0000 7fa96e1ff700 1 -- start start 2023-10-17T01:25:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa96e1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9681017d0 0x7fa9680ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa96e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 0x7fa9680ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9681017d0 0x7fa9680ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa96e1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa968106a30 0x7fa968100200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9681017d0 0x7fa9680ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46440/0 (socket says 172.21.15.45:46440) 2023-10-17T01:25:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa96e1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a0ac0 con 0x7fa968106a30 2023-10-17T01:25:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa9677fe700 1 -- 172.21.15.45:0/1875921513 learned_addr learned my addr 172.21.15.45:0/1875921513 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a0c00 con 0x7fa968104100 2023-10-17T01:25:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.934+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a0d40 con 0x7fa9681017d0 2023-10-17T01:25:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa95effd700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 0x7fa9680ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa967fff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa968106a30 0x7fa968100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa95effd700 1 -- 172.21.15.45:0/1875921513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9681017d0 msgr2=0x7fa9680ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa95effd700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9681017d0 0x7fa9680ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa95effd700 1 -- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa968106a30 msgr2=0x7fa968100200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa95effd700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa968106a30 0x7fa968100200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa95effd700 1 -- 172.21.15.45:0/1875921513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9600095f0 con 0x7fa968104100 2023-10-17T01:25:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.935+0000 7fa967fff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa968106a30 0x7fa968100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.936+0000 7fa95effd700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 0x7fa9680ffcc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa95000b450 tx=0x7fa95000b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.936+0000 7fa9657fa700 1 -- 172.21.15.45:0/1875921513 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950011830 con 0x7fa968104100 2023-10-17T01:25:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.936+0000 7fa9657fa700 1 -- 172.21.15.45:0/1875921513 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95000be60 con 0x7fa968104100 2023-10-17T01:25:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.936+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9681a0fd0 con 0x7fa968104100 2023-10-17T01:25:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.936+0000 7fa9657fa700 1 -- 172.21.15.45:0/1875921513 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950010d20 con 0x7fa968104100 2023-10-17T01:25:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.937+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9681a14d0 con 0x7fa968104100 2023-10-17T01:25:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.941+0000 7fa981b09b80 1 Processor -- start 2023-10-17T01:25:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 -- start start 2023-10-17T01:25:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d05859dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d05859e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859ed90 0x55d05859f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0584f59b0 con 0x55d05859d830 2023-10-17T01:25:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0584f3370 con 0x55d05859e190 2023-10-17T01:25:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa981b09b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0584eec60 con 0x55d05859ed90 2023-10-17T01:25:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d05859e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa967fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d05859dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d05859e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47324/0 (socket says 172.21.15.45:47324) 2023-10-17T01:25:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa967fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d05859dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49738/0 (socket says 172.21.15.45:49738) 2023-10-17T01:25:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.942+0000 7fa9677fe700 1 -- 172.21.15.45:0/132046791 learned_addr learned my addr 172.21.15.45:0/132046791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa9677fe700 1 -- 172.21.15.45:0/132046791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859ed90 msgr2=0x55d05859f210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:25:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa9677fe700 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859ed90 0x55d05859f210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa9677fe700 1 -- 172.21.15.45:0/132046791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 msgr2=0x55d05859dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa9677fe700 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d05859dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa9657fa700 1 -- 172.21.15.45:0/1875921513 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa950010380 con 0x7fa968104100 2023-10-17T01:25:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa9677fe700 1 -- 172.21.15.45:0/132046791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d05831cd80 con 0x55d05859e190 2023-10-17T01:25:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.943+0000 7fa967fff700 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d05859dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.944+0000 7fa9677fe700 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d05859e790 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e240 tx=0x7fa958009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b690b83cd70771 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.945+0000 7fa95dffb700 1 -- 172.21.15.45:0/132046791 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958009c40 con 0x55d05859e190 2023-10-17T01:25:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.945+0000 7fa95dffb700 1 -- 172.21.15.45:0/132046791 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95800f040 con 0x55d05859e190 2023-10-17T01:25:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.945+0000 7fa95dffb700 1 -- 172.21.15.45:0/132046791 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958013940 con 0x55d05859e190 2023-10-17T01:25:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 -- 172.21.15.45:0/132046791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 msgr2=0x55d05859e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d05859e790 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e240 tx=0x7fa958009790 comp rx=0 tx=0).stop 2023-10-17T01:25:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 -- 172.21.15.45:0/132046791 shutdown_connections 2023-10-17T01:25:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859ed90 0x55d05859f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d05859dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 --2- 172.21.15.45:0/132046791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d05859e790 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 -- 172.21.15.45:0/132046791 >> 172.21.15.45:0/132046791 conn(0x55d0584fd070 msgr2=0x55d0585982c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 -- 172.21.15.45:0/132046791 shutdown_connections 2023-10-17T01:25:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.946+0000 7fa981b09b80 1 -- 172.21.15.45:0/132046791 wait complete. 2023-10-17T01:25:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.947+0000 7fa981b09b80 1 -- start start 2023-10-17T01:25:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.947+0000 7fa981b09b80 1 Processor -- start 2023-10-17T01:25:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d0585a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859ed90 0x55d0585ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d05848b4b0 con 0x55d05859ed90 2023-10-17T01:25:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d05848b150 con 0x55d05859e190 2023-10-17T01:25:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0583728c0 con 0x55d05859d830 2023-10-17T01:25:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa981b09b80 10 client.0 _ll_register_callbacks cb 0x55d0583d8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.948+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d0585a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d0585a79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47334/0 (socket says 172.21.15.45:47334) 2023-10-17T01:25:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 -- 172.21.15.45:0/2977219592 learned_addr learned my addr 172.21.15.45:0/2977219592 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa95effd700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859ed90 0x55d0585ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa967fff700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9657fa700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9540a0ae0 0x7fa9540a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 -- 172.21.15.45:0/2977219592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 msgr2=0x55d0585a7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 -- 172.21.15.45:0/2977219592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859ed90 msgr2=0x55d0585ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859ed90 0x55d0585ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 -- 172.21.15.45:0/2977219592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa968109a90 con 0x55d05859e190 2023-10-17T01:25:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.949+0000 7fa9677fe700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d0585a79a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa958013420 tx=0x7fa958014bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1236554f915cabd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa946ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa9657fa700 1 -- 172.21.15.45:0/1875921513 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9500cbb00 con 0x7fa968104100 2023-10-17T01:25:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa946ffd700 1 -- 172.21.15.45:0/2977219592 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958009450 con 0x55d05859e190 2023-10-17T01:25:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa946ffd700 1 -- 172.21.15.45:0/2977219592 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa958006e40 con 0x55d05859e190 2023-10-17T01:25:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa967fff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9540a0ae0 0x7fa9540a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d058519cb0 con 0x55d05859e190 2023-10-17T01:25:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa946ffd700 1 -- 172.21.15.45:0/2977219592 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa958009c20 con 0x55d05859e190 2023-10-17T01:25:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa981b09b80 10 client.26552 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.950+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fa958009da0 con 0x55d05859e190 2023-10-17T01:25:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa946ffd700 1 -- 172.21.15.45:0/2977219592 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d058519cb0 con 0x55d05859e190 2023-10-17T01:25:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa946ffd700 1 -- 172.21.15.45:0/2977219592 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa95802e070 con 0x55d05859e190 2023-10-17T01:25:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa981b09b80 10 client.26552 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa981b09b80 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa981b09b80 20 client.26552 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa981b09b80 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d0585bb840 0x55d0585bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa981b09b80 4 client.26552 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d05856ccd0 con 0x55d0585bb840 2023-10-17T01:25:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.951+0000 7fa95effd700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d0585bb840 0x55d0585bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.952+0000 7fa967fff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9540a0ae0 0x7fa9540a2fa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa960005cd0 tx=0x7fa960009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.953+0000 7fa95effd700 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d0585bb840 0x55d0585bdc60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962211e8d505cde5 server_cookie=63200f8e0f3bd6f8 in_seq=0 out_seq=0 2023-10-17T01:25:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.953+0000 7fa946ffd700 10 client.26552 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.954+0000 7fa946ffd700 1 -- 172.21.15.45:0/2977219592 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d05856ccd0 con 0x55d0585bb840 2023-10-17T01:25:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.954+0000 7fa946ffd700 10 client.26552 handle_command_reply: tid=0 2023-10-17T01:25:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.954+0000 7fa981b09b80 1 client.26552 shutdown 2023-10-17T01:25:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 msgr2=0x55d0585a79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d0585a79a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa958013420 tx=0x7fa958014bb0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 shutdown_connections 2023-10-17T01:25:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859ed90 0x55d0585ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d0585bb840 0x55d0585bdc60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 --2- 172.21.15.45:0/2977219592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859e190 0x55d0585a79a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 >> 172.21.15.45:0/2977219592 conn(0x55d0584fd070 msgr2=0x55d058597a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.955+0000 7fa946ffd700 0 client.26552 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.956+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 shutdown_connections 2023-10-17T01:25:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.956+0000 7fa981b09b80 1 -- 172.21.15.45:0/2977219592 wait complete. 2023-10-17T01:25:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.956+0000 7fa981b09b80 20 client.26552 trim_cache size 0 max 16384 2023-10-17T01:25:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 Processor -- start 2023-10-17T01:25:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 -- start start 2023-10-17T01:25:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859e190 0x55d0585a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d0585a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d058595010 con 0x55d05859e190 2023-10-17T01:25:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0585a5c60 con 0x55d05859ed90 2023-10-17T01:25:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.982+0000 7fa981b09b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d058590280 con 0x55d05859d830 2023-10-17T01:25:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46466/0 (socket says 172.21.15.45:46466) 2023-10-17T01:25:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 -- 172.21.15.45:0/4276201521 learned_addr learned my addr 172.21.15.45:0/4276201521 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa967fff700 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d0585a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa95effd700 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859e190 0x55d0585a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 -- 172.21.15.45:0/4276201521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 msgr2=0x55d0585a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d0585a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 -- 172.21.15.45:0/4276201521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859e190 msgr2=0x55d0585a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859e190 0x55d0585a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa9677fe700 1 -- 172.21.15.45:0/4276201521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d058305c00 con 0x55d05859d830 2023-10-17T01:25:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.983+0000 7fa95effd700 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859e190 0x55d0585a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.984+0000 7fa9677fe700 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a5280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9580097d0 tx=0x7fa95800ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d58d50eb0c08fc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.984+0000 7fa95e7fc700 1 -- 172.21.15.45:0/4276201521 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95801d620 con 0x55d05859d830 2023-10-17T01:25:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.984+0000 7fa95e7fc700 1 -- 172.21.15.45:0/4276201521 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958009c20 con 0x55d05859d830 2023-10-17T01:25:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.984+0000 7fa95e7fc700 1 -- 172.21.15.45:0/4276201521 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958024600 con 0x55d05859d830 2023-10-17T01:25:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 -- 172.21.15.45:0/4276201521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 msgr2=0x55d0585a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a5280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9580097d0 tx=0x7fa95800ef80 comp rx=0 tx=0).stop 2023-10-17T01:25:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 -- 172.21.15.45:0/4276201521 shutdown_connections 2023-10-17T01:25:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d0585a22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859d830 0x55d0585a5280 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 --2- 172.21.15.45:0/4276201521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859e190 0x55d0585a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 -- 172.21.15.45:0/4276201521 >> 172.21.15.45:0/4276201521 conn(0x55d0584fd070 msgr2=0x55d0585bd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 -- 172.21.15.45:0/4276201521 shutdown_connections 2023-10-17T01:25:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.985+0000 7fa981b09b80 1 -- 172.21.15.45:0/4276201521 wait complete. 2023-10-17T01:25:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.986+0000 7fa981b09b80 1 -- start start 2023-10-17T01:25:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.986+0000 7fa981b09b80 1 Processor -- start 2023-10-17T01:25:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d0586071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859e190 0x55d0586096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d05860bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d05860c180 con 0x55d05859d830 2023-10-17T01:25:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d05860c2f0 con 0x55d05859ed90 2023-10-17T01:25:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d05861d4c0 con 0x55d05859e190 2023-10-17T01:25:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa981b09b80 10 client.0 _ll_register_callbacks cb 0x7fa960009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa95effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859e190 0x55d0586096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa95effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859e190 0x55d0586096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46474/0 (socket says 172.21.15.45:46474) 2023-10-17T01:25:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa95effd700 1 -- 172.21.15.45:0/1625342089 learned_addr learned my addr 172.21.15.45:0/1625342089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.987+0000 7fa967fff700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d05860bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.988+0000 7fa9677fe700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d0586071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.988+0000 7fa9677fe700 1 -- 172.21.15.45:0/1625342089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859e190 msgr2=0x55d0586096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.988+0000 7fa9677fe700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859e190 0x55d0586096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.988+0000 7fa9677fe700 1 -- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 msgr2=0x55d05860bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.988+0000 7fa9677fe700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d05860bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.988+0000 7fa9677fe700 1 -- 172.21.15.45:0/1625342089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958009da0 con 0x55d05859d830 2023-10-17T01:25:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.989+0000 7fa9677fe700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d0586071a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa958000f80 tx=0x7fa958009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b7cb9c0c53ffc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.989+0000 7fa946ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:25:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.990+0000 7fa946ffd700 1 -- 172.21.15.45:0/1625342089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958021040 con 0x55d05859d830 2023-10-17T01:25:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.990+0000 7fa946ffd700 1 -- 172.21.15.45:0/1625342089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa958024cb0 con 0x55d05859d830 2023-10-17T01:25:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.990+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0584e7ce0 con 0x55d05859d830 2023-10-17T01:25:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.992+0000 7fa946ffd700 1 -- 172.21.15.45:0/1625342089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa958024e70 con 0x55d05859d830 2023-10-17T01:25:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.992+0000 7fa981b09b80 10 client.16242 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.993+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d0583431f0 con 0x55d05859d830 2023-10-17T01:25:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.993+0000 7fa946ffd700 1 -- 172.21.15.45:0/1625342089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0584e7ce0 con 0x55d05859d830 2023-10-17T01:25:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.993+0000 7fa946ffd700 1 -- 172.21.15.45:0/1625342089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa958045050 con 0x55d05859d830 2023-10-17T01:25:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa981b09b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa981b09b80 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa981b09b80 20 client.16242 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa981b09b80 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d05861e9c0 0x55d058620db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa981b09b80 4 client.16242 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:25:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0584e7a00 con 0x55d05861e9c0 2023-10-17T01:25:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.994+0000 7fa95effd700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d05861e9c0 0x55d058620db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.995+0000 7fa95effd700 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d05861e9c0 0x55d058620db0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84be3de1b8e48ff server_cookie=ebd410d3afecdbac in_seq=0 out_seq=0 2023-10-17T01:25:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:04.995+0000 7fa946ffd700 10 client.16242 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.008+0000 7fa946ffd700 1 -- 172.21.15.45:0/1625342089 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0584e7a00 con 0x55d05861e9c0 2023-10-17T01:25:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.009+0000 7fa946ffd700 10 client.16242 handle_command_reply: tid=0 2023-10-17T01:25:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.009+0000 7fa981b09b80 1 client.16242 shutdown 2023-10-17T01:25:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 msgr2=0x55d0586071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d0586071a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa958000f80 tx=0x7fa958009c90 comp rx=0 tx=0).stop 2023-10-17T01:25:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 shutdown_connections 2023-10-17T01:25:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d05861e9c0 0x55d058620db0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d05859ed90 0x55d05860bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d05859d830 0x55d0586071a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 --2- 172.21.15.45:0/1625342089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d05859e190 0x55d0586096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 >> 172.21.15.45:0/1625342089 conn(0x55d0584fd070 msgr2=0x55d0585a5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.010+0000 7fa946ffd700 0 client.16242 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.011+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 shutdown_connections 2023-10-17T01:25:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.011+0000 7fa981b09b80 1 -- 172.21.15.45:0/1625342089 wait complete. 2023-10-17T01:25:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:25:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:25:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:25:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:25:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.012+0000 7fa981b09b80 20 client.16242 trim_cache size 0 max 16384 2023-10-17T01:25:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9540a0ae0 msgr2=0x7fa9540a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9540a0ae0 0x7fa9540a2fa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa960005cd0 tx=0x7fa960009310 comp rx=0 tx=0).stop 2023-10-17T01:25:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 msgr2=0x7fa9680ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 0x7fa9680ffcc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa95000b450 tx=0x7fa95000b760 comp rx=0 tx=0).stop 2023-10-17T01:25:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 shutdown_connections 2023-10-17T01:25:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9540a0ae0 0x7fa9540a2fa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa968106a30 0x7fa968100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9681017d0 0x7fa9680ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 --2- 172.21.15.45:0/1875921513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa968104100 0x7fa9680ffcc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 >> 172.21.15.45:0/1875921513 conn(0x7fa9680fb1a0 msgr2=0x7fa968102440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 shutdown_connections 2023-10-17T01:25:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:05.013+0000 7fa96e1ff700 1 -- 172.21.15.45:0/1875921513 wait complete. 2023-10-17T01:25:05.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:04 smithi060 ceph-mon[134136]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 849 op/s 2023-10-17T01:25:05.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:04 smithi181 ceph-mon[134520]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 849 op/s 2023-10-17T01:25:05.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:04 smithi045 ceph-mon[98474]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 17 MiB/s wr, 849 op/s 2023-10-17T01:25:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 595.0 s, 1296.0 tps, lat 77.202 ms stddev 87.514 2023-10-17T01:25:05.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:25:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 -- 172.21.15.45:0/3384081437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd854104ab0 msgr2=0x7fd85410b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3384081437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd854104ab0 0x7fd85410b3d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd84800b0a0 tx=0x7fd84800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 -- 172.21.15.45:0/3384081437 shutdown_connections 2023-10-17T01:25:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3384081437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd854104ab0 0x7fd85410b3d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3384081437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854102f20 0x7fd854103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3384081437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8541040f0 0x7fd854104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 -- 172.21.15.45:0/3384081437 >> 172.21.15.45:0/3384081437 conn(0x7fd8540fe500 msgr2=0x7fd854100920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 -- 172.21.15.45:0/3384081437 shutdown_connections 2023-10-17T01:25:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.061+0000 7fd8593ff700 1 -- 172.21.15.45:0/3384081437 wait complete. 2023-10-17T01:25:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.063+0000 7fd8593ff700 1 Processor -- start 2023-10-17T01:25:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.063+0000 7fd8593ff700 1 -- start start 2023-10-17T01:25:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.064+0000 7fd8593ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 0x7fd85419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.064+0000 7fd8593ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8541040f0 0x7fd85419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd852ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 0x7fd85419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd852ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 0x7fd85419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49788/0 (socket says 172.21.15.45:49788) 2023-10-17T01:25:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd852ffd700 1 -- 172.21.15.45:0/3256003432 learned_addr learned my addr 172.21.15.45:0/3256003432 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854104ab0 0x7fd8541a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541a4d40 con 0x7fd854102f20 2023-10-17T01:25:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541a4e80 con 0x7fd8541040f0 2023-10-17T01:25:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541a5010 con 0x7fd854104ab0 2023-10-17T01:25:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.065+0000 7fd8527fc700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8541040f0 0x7fd85419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8537fe700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854104ab0 0x7fd8541a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8527fc700 1 -- 172.21.15.45:0/3256003432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854104ab0 msgr2=0x7fd8541a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8527fc700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854104ab0 0x7fd8541a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8527fc700 1 -- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 msgr2=0x7fd85419d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8527fc700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 0x7fd85419d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8527fc700 1 -- 172.21.15.45:0/3256003432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009d00 con 0x7fd8541040f0 2023-10-17T01:25:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8537fe700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854104ab0 0x7fd8541a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:25:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd852ffd700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 0x7fd85419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.066+0000 7fd8527fc700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8541040f0 0x7fd85419d700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd84400e9f0 tx=0x7fd84400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.067+0000 7fd83bfff700 1 -- 172.21.15.45:0/3256003432 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440096d0 con 0x7fd8541040f0 2023-10-17T01:25:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.067+0000 7fd83bfff700 1 -- 172.21.15.45:0/3256003432 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84400cbd0 con 0x7fd8541040f0 2023-10-17T01:25:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.067+0000 7fd83bfff700 1 -- 172.21.15.45:0/3256003432 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844018680 con 0x7fd8541040f0 2023-10-17T01:25:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.067+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8541a52f0 con 0x7fd8541040f0 2023-10-17T01:25:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.067+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8541a57b0 con 0x7fd8541040f0 2023-10-17T01:25:06.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.073+0000 7fd83bfff700 1 -- 172.21.15.45:0/3256003432 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fd84400cd40 con 0x7fd8541040f0 2023-10-17T01:25:06.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.075+0000 7fd86cd0ab80 1 Processor -- start 2023-10-17T01:25:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.075+0000 7fd83bfff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd83c0a0ba0 0x7fd83c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.076+0000 7fd83bfff700 1 -- 172.21.15.45:0/3256003432 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd844014070 con 0x7fd8541040f0 2023-10-17T01:25:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.076+0000 7fd86cd0ab80 1 -- start start 2023-10-17T01:25:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59ecc6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecc7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ec1da10 con 0x55e59ecc6500 2023-10-17T01:25:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ec1b3d0 con 0x55e59ecc5ba0 2023-10-17T01:25:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ec16cc0 con 0x55e59ecc72b0 2023-10-17T01:25:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd8537fe700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd83c0a0ba0 0x7fd83c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd852ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.077+0000 7fd8527fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59ecc6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59ecc6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49794/0 (socket says 172.21.15.45:49794) 2023-10-17T01:25:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 -- 172.21.15.45:0/2610009974 learned_addr learned my addr 172.21.15.45:0/2610009974 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd852ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:47398/0 (socket says 172.21.15.45:47398) 2023-10-17T01:25:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8537fe700 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecc7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 -- 172.21.15.45:0/2610009974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 msgr2=0x55e59ecc7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecc7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 -- 172.21.15.45:0/2610009974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 msgr2=0x55e59ecc5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 -- 172.21.15.45:0/2610009974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59ea44de0 con 0x55e59ecc6500 2023-10-17T01:25:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.078+0000 7fd8527fc700 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59ecc6d20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd844099900 tx=0x7fd8440992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e5e31597aa5c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.079+0000 7fd8537fe700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd83c0a0ba0 0x7fd83c0a3060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd84800b070 tx=0x7fd8480092e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.079+0000 7fd8397fa700 1 -- 172.21.15.45:0/2610009974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84409d7a0 con 0x55e59ecc6500 2023-10-17T01:25:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.079+0000 7fd8397fa700 1 -- 172.21.15.45:0/2610009974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8440a19e0 con 0x55e59ecc6500 2023-10-17T01:25:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.079+0000 7fd8397fa700 1 -- 172.21.15.45:0/2610009974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844099bd0 con 0x55e59ecc6500 2023-10-17T01:25:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/2610009974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 msgr2=0x55e59ecc6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59ecc6d20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd844099900 tx=0x7fd8440992d0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/2610009974 shutdown_connections 2023-10-17T01:25:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecc7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/2610009974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59ecc6d20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/2610009974 >> 172.21.15.45:0/2610009974 conn(0x55e59ec250d0 msgr2=0x55e59ecc0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/2610009974 shutdown_connections 2023-10-17T01:25:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.080+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/2610009974 wait complete. 2023-10-17T01:25:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.081+0000 7fd86cd0ab80 1 -- start start 2023-10-17T01:25:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.081+0000 7fd86cd0ab80 1 Processor -- start 2023-10-17T01:25:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59eccf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59eccfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecd6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ebb3510 con 0x55e59ecc6500 2023-10-17T01:25:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ebb31b0 con 0x55e59ecc5ba0 2023-10-17T01:25:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ea9a920 con 0x55e59ecc72b0 2023-10-17T01:25:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd86cd0ab80 10 client.0 _ll_register_callbacks cb 0x55e59eb00a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd8527fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59eccfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd8537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecd6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd8537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecd6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46506/0 (socket says 172.21.15.45:46506) 2023-10-17T01:25:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd8527fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59eccfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49804/0 (socket says 172.21.15.45:49804) 2023-10-17T01:25:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd8537fe700 1 -- 172.21.15.45:0/1627890942 learned_addr learned my addr 172.21.15.45:0/1627890942 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.082+0000 7fd852ffd700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59eccf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.083+0000 7fd8537fe700 1 -- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 msgr2=0x55e59eccf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.083+0000 7fd8537fe700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59eccf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.083+0000 7fd8537fe700 1 -- 172.21.15.45:0/1627890942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 msgr2=0x55e59eccfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.083+0000 7fd8537fe700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59eccfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.083+0000 7fd8537fe700 1 -- 172.21.15.45:0/1627890942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8541a57b0 con 0x55e59ecc72b0 2023-10-17T01:25:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.083+0000 7fd8527fc700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59eccfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.084+0000 7fd8537fe700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecd6e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd848004d00 tx=0x7fd84800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a1c91174abd244 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.084+0000 7fd836ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:25:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.085+0000 7fd836ff5700 1 -- 172.21.15.45:0/1627890942 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84800e040 con 0x55e59ecc72b0 2023-10-17T01:25:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.085+0000 7fd836ff5700 1 -- 172.21.15.45:0/1627890942 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd848028630 con 0x55e59ecc72b0 2023-10-17T01:25:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.085+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e59ec41d10 con 0x55e59ecc72b0 2023-10-17T01:25:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.085+0000 7fd836ff5700 1 -- 172.21.15.45:0/1627890942 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8480094b0 con 0x55e59ecc72b0 2023-10-17T01:25:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.086+0000 7fd86cd0ab80 10 client.26521 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.086+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e59ea2dc60 con 0x55e59ecc72b0 2023-10-17T01:25:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.086+0000 7fd836ff5700 1 -- 172.21.15.45:0/1627890942 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e59ec41d10 con 0x55e59ecc72b0 2023-10-17T01:25:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.086+0000 7fd836ff5700 1 -- 172.21.15.45:0/1627890942 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fd848005290 con 0x55e59ecc72b0 2023-10-17T01:25:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.087+0000 7fd86cd0ab80 10 client.26521 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.087+0000 7fd86cd0ab80 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.087+0000 7fd86cd0ab80 20 client.26521 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.087+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ecdb600 0x55e59ecdda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.087+0000 7fd86cd0ab80 4 client.26521 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.087+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e59ec94d30 con 0x55e59ecdb600 2023-10-17T01:25:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.090+0000 7fd852ffd700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ecdb600 0x55e59ecdda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.091+0000 7fd852ffd700 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ecdb600 0x55e59ecdda20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182574ffa5b32c6c server_cookie=7f64883644c50e2c in_seq=0 out_seq=0 2023-10-17T01:25:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.092+0000 7fd836ff5700 10 client.26521 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.092+0000 7fd836ff5700 1 -- 172.21.15.45:0/1627890942 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55e59ec94d30 con 0x55e59ecdb600 2023-10-17T01:25:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.092+0000 7fd836ff5700 10 client.26521 handle_command_reply: tid=0 2023-10-17T01:25:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 client.26521 shutdown 2023-10-17T01:25:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 msgr2=0x55e59ecd6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecd6e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd848004d00 tx=0x7fd84800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 shutdown_connections 2023-10-17T01:25:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc72b0 0x55e59ecd6e90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ecdb600 0x55e59ecdda20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc5ba0 0x55e59eccf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/1627890942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc6500 0x55e59eccfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd836ff5700 0 client.26521 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.093+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 >> 172.21.15.45:0/1627890942 conn(0x55e59ec250d0 msgr2=0x55e59ecc0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.094+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 shutdown_connections 2023-10-17T01:25:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.094+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/1627890942 wait complete. 2023-10-17T01:25:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.094+0000 7fd86cd0ab80 20 client.26521 trim_cache size 0 max 16384 2023-10-17T01:25:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 Processor -- start 2023-10-17T01:25:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 -- start start 2023-10-17T01:25:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc6500 0x55e59eccd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc72b0 0x55e59ecca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ecd0b00 con 0x55e59ecc72b0 2023-10-17T01:25:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59eccedc0 con 0x55e59ecc6500 2023-10-17T01:25:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.132+0000 7fd86cd0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59eccf480 con 0x55e59ecc5ba0 2023-10-17T01:25:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.133+0000 7fd8537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc72b0 0x55e59ecca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.133+0000 7fd852ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.133+0000 7fd852ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46520/0 (socket says 172.21.15.45:46520) 2023-10-17T01:25:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.133+0000 7fd852ffd700 1 -- 172.21.15.45:0/321718047 learned_addr learned my addr 172.21.15.45:0/321718047 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.134+0000 7fd8537fe700 1 -- 172.21.15.45:0/321718047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc5ba0 msgr2=0x55e59ecc0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.134+0000 7fd8537fe700 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.134+0000 7fd8537fe700 1 -- 172.21.15.45:0/321718047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc6500 msgr2=0x55e59eccd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.134+0000 7fd8537fe700 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc6500 0x55e59eccd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.134+0000 7fd8537fe700 1 -- 172.21.15.45:0/321718047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59ea6b250 con 0x55e59ecc72b0 2023-10-17T01:25:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.134+0000 7fd8537fe700 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc72b0 0x55e59ecca260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd848004cd0 tx=0x7fd848009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ac6dbbae00561b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.136+0000 7fd839ffb700 1 -- 172.21.15.45:0/321718047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84800e040 con 0x55e59ecc72b0 2023-10-17T01:25:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.137+0000 7fd839ffb700 1 -- 172.21.15.45:0/321718047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd848005910 con 0x55e59ecc72b0 2023-10-17T01:25:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.137+0000 7fd839ffb700 1 -- 172.21.15.45:0/321718047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84803c9c0 con 0x55e59ecc72b0 2023-10-17T01:25:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.137+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/321718047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc72b0 msgr2=0x55e59ecca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.137+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc72b0 0x55e59ecca260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd848004cd0 tx=0x7fd848009960 comp rx=0 tx=0).stop 2023-10-17T01:25:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/321718047 shutdown_connections 2023-10-17T01:25:06.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc72b0 0x55e59ecca260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc5ba0 0x55e59ecc0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/321718047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc6500 0x55e59eccd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/321718047 >> 172.21.15.45:0/321718047 conn(0x55e59ec250d0 msgr2=0x55e59ecce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/321718047 shutdown_connections 2023-10-17T01:25:06.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.138+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/321718047 wait complete. 2023-10-17T01:25:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.139+0000 7fd86cd0ab80 1 -- start start 2023-10-17T01:25:06.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.139+0000 7fd86cd0ab80 1 Processor -- start 2023-10-17T01:25:06.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.143+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc5ba0 0x55e59ed251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.143+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc6500 0x55e59ed256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.143+0000 7fd86cd0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc72b0 0x55e59ed2c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.144+0000 7fd8527fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc6500 0x55e59ed256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.144+0000 7fd8527fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc6500 0x55e59ed256e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46530/0 (socket says 172.21.15.45:46530) 2023-10-17T01:25:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.144+0000 7fd8527fc700 1 -- 172.21.15.45:0/55318990 learned_addr learned my addr 172.21.15.45:0/55318990 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.144+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ed2cc60 con 0x55e59ecc5ba0 2023-10-17T01:25:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.144+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ed2cda0 con 0x55e59ecc72b0 2023-10-17T01:25:06.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.145+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ed2cf30 con 0x55e59ecc6500 2023-10-17T01:25:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.145+0000 7fd86cd0ab80 10 client.0 _ll_register_callbacks cb 0x7fd848021050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.145+0000 7fd852ffd700 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc5ba0 0x55e59ed251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.146+0000 7fd852ffd700 1 -- 172.21.15.45:0/55318990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc6500 msgr2=0x55e59ed256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.146+0000 7fd852ffd700 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc6500 0x55e59ed256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.146+0000 7fd852ffd700 1 -- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc72b0 msgr2=0x55e59ed2c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.146+0000 7fd852ffd700 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc72b0 0x55e59ed2c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.147+0000 7fd852ffd700 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e59ea2dc60 con 0x55e59ecc5ba0 2023-10-17T01:25:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.147+0000 7fd852ffd700 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc5ba0 0x55e59ed251a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55e59ecbcf90 tx=0x7fd84c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943251fe9683529f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.147+0000 7fd836ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:25:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.148+0000 7fd836ff5700 1 -- 172.21.15.45:0/55318990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c015040 con 0x55e59ecc5ba0 2023-10-17T01:25:06.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.148+0000 7fd836ff5700 1 -- 172.21.15.45:0/55318990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd84c003c90 con 0x55e59ecc5ba0 2023-10-17T01:25:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.148+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8440a19e0 con 0x55e59ecc5ba0 2023-10-17T01:25:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.148+0000 7fd836ff5700 1 -- 172.21.15.45:0/55318990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84c003e50 con 0x55e59ecc5ba0 2023-10-17T01:25:06.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.148+0000 7fd86cd0ab80 10 client.16260 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.148+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e59ea432b0 con 0x55e59ecc5ba0 2023-10-17T01:25:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd836ff5700 1 -- 172.21.15.45:0/55318990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8440a19e0 con 0x55e59ecc5ba0 2023-10-17T01:25:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd836ff5700 1 -- 172.21.15.45:0/55318990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fd84c025020 con 0x55e59ecc5ba0 2023-10-17T01:25:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd86cd0ab80 10 client.16260 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:06.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd86cd0ab80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd86cd0ab80 20 client.16260 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ed2e970 0x55e59ed30d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd86cd0ab80 4 client.16260 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:25:06.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.149+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e59ec0fd40 con 0x55e59ed2e970 2023-10-17T01:25:06.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.152+0000 7fd8527fc700 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ed2e970 0x55e59ed30d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.158+0000 7fd8527fc700 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ed2e970 0x55e59ed30d90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe94ee81cd53ee0 server_cookie=4935cf6db2b44fd8 in_seq=0 out_seq=0 2023-10-17T01:25:06.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.158+0000 7fd836ff5700 10 client.16260 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.163+0000 7fd836ff5700 1 -- 172.21.15.45:0/55318990 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e59ec0fd40 con 0x55e59ed2e970 2023-10-17T01:25:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.163+0000 7fd836ff5700 10 client.16260 handle_command_reply: tid=0 2023-10-17T01:25:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.163+0000 7fd86cd0ab80 1 client.16260 shutdown 2023-10-17T01:25:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc5ba0 msgr2=0x55e59ed251a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc5ba0 0x55e59ed251a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55e59ecbcf90 tx=0x7fd84c005f10 comp rx=0 tx=0).stop 2023-10-17T01:25:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 shutdown_connections 2023-10-17T01:25:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e59ed2e970 0x55e59ed30d90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e59ecc72b0 0x55e59ed2c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e59ecc5ba0 0x55e59ed251a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 --2- 172.21.15.45:0/55318990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e59ecc6500 0x55e59ed256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 >> 172.21.15.45:0/55318990 conn(0x55e59ec250d0 msgr2=0x55e59ecc0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd836ff5700 0 client.16260 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 shutdown_connections 2023-10-17T01:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.164+0000 7fd86cd0ab80 1 -- 172.21.15.45:0/55318990 wait complete. 2023-10-17T01:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd86cd0ab80 20 client.16260 trim_cache size 0 max 16384 2023-10-17T01:25:06.168 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:25:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:25:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:25:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:25:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:25:06.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:25:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:25:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:25:06.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:25:06.170 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd83c0a0ba0 msgr2=0x7fd83c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd83c0a0ba0 0x7fd83c0a3060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd84800b070 tx=0x7fd8480092e0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8541040f0 msgr2=0x7fd85419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8541040f0 0x7fd85419d700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd84400e9f0 tx=0x7fd84400ed00 comp rx=0 tx=0).stop 2023-10-17T01:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 shutdown_connections 2023-10-17T01:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fd83c0a0ba0 0x7fd83c0a3060 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd854104ab0 0x7fd8541a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd854102f20 0x7fd85419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 --2- 172.21.15.45:0/3256003432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8541040f0 0x7fd85419d700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.165+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 >> 172.21.15.45:0/3256003432 conn(0x7fd8540fe500 msgr2=0x7fd854100730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.166+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 shutdown_connections 2023-10-17T01:25:06.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:06.166+0000 7fd8593ff700 1 -- 172.21.15.45:0/3256003432 wait complete. 2023-10-17T01:25:06.194 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:25:07.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:06 smithi060 ceph-mon[134136]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 712 op/s 2023-10-17T01:25:07.161 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:06 smithi181 ceph-mon[134520]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 712 op/s 2023-10-17T01:25:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:06 smithi045 ceph-mon[98474]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 712 op/s 2023-10-17T01:25:09.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:08 smithi181 ceph-mon[134520]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 739 op/s 2023-10-17T01:25:09.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:08 smithi045 ceph-mon[98474]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 739 op/s 2023-10-17T01:25:09.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:08 smithi060 ceph-mon[134136]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 739 op/s 2023-10-17T01:25:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 600.0 s, 1346.4 tps, lat 74.000 ms stddev 86.851 2023-10-17T01:25:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:10 smithi045 ceph-mon[98474]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 804 op/s 2023-10-17T01:25:11.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:10 smithi060 ceph-mon[134136]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 804 op/s 2023-10-17T01:25:11.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:10 smithi181 ceph-mon[134520]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 15 MiB/s wr, 804 op/s 2023-10-17T01:25:13.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:12 smithi060 ceph-mon[134136]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 899 op/s 2023-10-17T01:25:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:12 smithi181 ceph-mon[134520]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 899 op/s 2023-10-17T01:25:13.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:12 smithi045 ceph-mon[98474]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 899 op/s 2023-10-17T01:25:15.083 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:14 smithi060 ceph-mon[134136]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.67k op/s 2023-10-17T01:25:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:14 smithi181 ceph-mon[134520]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.67k op/s 2023-10-17T01:25:15.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:14 smithi045 ceph-mon[98474]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.67k op/s 2023-10-17T01:25:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 605.0 s, 1061.4 tps, lat 95.388 ms stddev 148.110 2023-10-17T01:25:17.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:16 smithi060 ceph-mon[134136]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 1.84k op/s 2023-10-17T01:25:17.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:16 smithi181 ceph-mon[134520]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 1.84k op/s 2023-10-17T01:25:17.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:16 smithi045 ceph-mon[98474]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 30 MiB/s wr, 1.84k op/s 2023-10-17T01:25:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.241+0000 7f97a0fff700 1 -- 172.21.15.45:0/581488083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 msgr2=0x7f979c10f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.241+0000 7f97a0fff700 1 --2- 172.21.15.45:0/581488083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c10f7d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f979400b0a0 tx=0x7f979400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 -- 172.21.15.45:0/581488083 shutdown_connections 2023-10-17T01:25:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 --2- 172.21.15.45:0/581488083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c10f7d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 --2- 172.21.15.45:0/581488083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 --2- 172.21.15.45:0/581488083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c108580 0x7f979c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 -- 172.21.15.45:0/581488083 >> 172.21.15.45:0/581488083 conn(0x7f979c076330 msgr2=0x7f979c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 -- 172.21.15.45:0/581488083 shutdown_connections 2023-10-17T01:25:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.242+0000 7f97a0fff700 1 -- 172.21.15.45:0/581488083 wait complete. 2023-10-17T01:25:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.244+0000 7f97a0fff700 1 Processor -- start 2023-10-17T01:25:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.244+0000 7f97a0fff700 1 -- start start 2023-10-17T01:25:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f97a0fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c19d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f97a0fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c108580 0x7f979c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f97a0fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f97a0fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c19ec60 con 0x7f979c107390 2023-10-17T01:25:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f97a0fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c19edd0 con 0x7f979c108580 2023-10-17T01:25:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f97a0fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c19ef90 con 0x7f979c108f20 2023-10-17T01:25:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f979a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c19d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f979ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f979ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45242/0 (socket says 172.21.15.45:45242) 2023-10-17T01:25:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f979a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c19d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53466/0 (socket says 172.21.15.45:53466) 2023-10-17T01:25:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f979ad9d700 1 -- 172.21.15.45:0/1614336481 learned_addr learned my addr 172.21.15.45:0/1614336481 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.245+0000 7f9799d9b700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c108580 0x7f979c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.246+0000 7f979a59c700 1 -- 172.21.15.45:0/1614336481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 msgr2=0x7f979c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.246+0000 7f979a59c700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.246+0000 7f979a59c700 1 -- 172.21.15.45:0/1614336481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c108580 msgr2=0x7f979c19d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.246+0000 7f979a59c700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c108580 0x7f979c19d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.246+0000 7f979a59c700 1 -- 172.21.15.45:0/1614336481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794009d00 con 0x7f979c107390 2023-10-17T01:25:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.246+0000 7f979a59c700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c19d240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f978c00d740 tx=0x7f978c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.247+0000 7f97937fe700 1 -- 172.21.15.45:0/1614336481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c004220 con 0x7f979c107390 2023-10-17T01:25:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.247+0000 7f97937fe700 1 -- 172.21.15.45:0/1614336481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978c004380 con 0x7f979c107390 2023-10-17T01:25:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.247+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c19f270 con 0x7f979c107390 2023-10-17T01:25:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.247+0000 7f97937fe700 1 -- 172.21.15.45:0/1614336481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c003cf0 con 0x7f979c107390 2023-10-17T01:25:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.247+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c1a5a30 con 0x7f979c107390 2023-10-17T01:25:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.254+0000 7f97b496cb80 1 Processor -- start 2023-10-17T01:25:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.254+0000 7f97b496cb80 1 -- start start 2023-10-17T01:25:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.255+0000 7f97937fe700 1 -- 172.21.15.45:0/1614336481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f978c00f690 con 0x7f979c107390 2023-10-17T01:25:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.255+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.255+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 0x563a8381c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.255+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381cd90 0x563a8381d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f97b496cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a837739b0 con 0x563a8381b830 2023-10-17T01:25:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f97b496cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83771370 con 0x563a8381c190 2023-10-17T01:25:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f97b496cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8376cc60 con 0x563a8381cd90 2023-10-17T01:25:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 0x563a8381c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 0x563a8381c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41978/0 (socket says 172.21.15.45:41978) 2023-10-17T01:25:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 -- 172.21.15.45:0/967337999 learned_addr learned my addr 172.21.15.45:0/967337999 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f9799d9b700 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381cd90 0x563a8381d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 -- 172.21.15.45:0/967337999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381cd90 msgr2=0x563a8381d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381cd90 0x563a8381d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 -- 172.21.15.45:0/967337999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 msgr2=0x563a8381bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.256+0000 7f979a59c700 1 -- 172.21.15.45:0/967337999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8359ad80 con 0x563a8381c190 2023-10-17T01:25:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.257+0000 7f979a59c700 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 0x563a8381c790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f978c0cfd70 tx=0x7f978c0cfda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913a063eabfa2a13 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.257+0000 7f9790ff9700 1 -- 172.21.15.45:0/967337999 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0d4de0 con 0x563a8381c190 2023-10-17T01:25:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.257+0000 7f9790ff9700 1 -- 172.21.15.45:0/967337999 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978c0d1900 con 0x563a8381c190 2023-10-17T01:25:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.257+0000 7f9790ff9700 1 -- 172.21.15.45:0/967337999 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0dc840 con 0x563a8381c190 2023-10-17T01:25:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.258+0000 7f97937fe700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97880a0ba0 0x7f97880a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.258+0000 7f97937fe700 1 -- 172.21.15.45:0/1614336481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f978c0cbda0 con 0x7f979c107390 2023-10-17T01:25:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.258+0000 7f979ad9d700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97880a0ba0 0x7f97880a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.258+0000 7f97b496cb80 1 -- 172.21.15.45:0/967337999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 msgr2=0x563a8381c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.258+0000 7f97b496cb80 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 0x563a8381c790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f978c0cfd70 tx=0x7f978c0cfda0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 -- 172.21.15.45:0/967337999 shutdown_connections 2023-10-17T01:25:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381cd90 0x563a8381d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 --2- 172.21.15.45:0/967337999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381c190 0x563a8381c790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 -- 172.21.15.45:0/967337999 >> 172.21.15.45:0/967337999 conn(0x563a8377b070 msgr2=0x563a838162c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 -- 172.21.15.45:0/967337999 shutdown_connections 2023-10-17T01:25:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f979ad9d700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97880a0ba0 0x7f97880a3060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9794009fd0 tx=0x7f9794006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 -- 172.21.15.45:0/967337999 wait complete. 2023-10-17T01:25:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.259+0000 7f97b496cb80 1 -- start start 2023-10-17T01:25:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 Processor -- start 2023-10-17T01:25:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a838253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a838258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a8382c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a837094b0 con 0x563a8381b830 2023-10-17T01:25:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83709150 con 0x563a8381cd90 2023-10-17T01:25:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a835f08c0 con 0x563a8381c190 2023-10-17T01:25:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f97b496cb80 10 client.0 _ll_register_callbacks cb 0x563a83656a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.260+0000 7f979ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a8382c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.261+0000 7f979ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a8382c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:41982/0 (socket says 172.21.15.45:41982) 2023-10-17T01:25:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.261+0000 7f979ad9d700 1 -- 172.21.15.45:0/2349852177 learned_addr learned my addr 172.21.15.45:0/2349852177 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.261+0000 7f9799d9b700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a838258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979ad9d700 1 -- 172.21.15.45:0/2349852177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 msgr2=0x563a838258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a838258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979a59c700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a838253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979ad9d700 1 -- 172.21.15.45:0/2349852177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 msgr2=0x563a838253b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a838253b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979ad9d700 1 -- 172.21.15.45:0/2349852177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c1a5a30 con 0x563a8381cd90 2023-10-17T01:25:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f9799d9b700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a838258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.262+0000 7f979a59c700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a838253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.263+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a8382c930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9794003c60 tx=0x7f979400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7dd8dece5af0d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.263+0000 7f9781ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.263+0000 7f9781ffb700 1 -- 172.21.15.45:0/2349852177 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400e040 con 0x563a8381cd90 2023-10-17T01:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.263+0000 7f9781ffb700 1 -- 172.21.15.45:0/2349852177 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f97940054d0 con 0x563a8381cd90 2023-10-17T01:25:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.263+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a83797cb0 con 0x563a8381cd90 2023-10-17T01:25:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.264+0000 7f9781ffb700 1 -- 172.21.15.45:0/2349852177 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f979400cec0 con 0x563a8381cd90 2023-10-17T01:25:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.264+0000 7f97b496cb80 10 client.26582 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.264+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563a83583c00 con 0x563a8381cd90 2023-10-17T01:25:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.264+0000 7f9781ffb700 1 -- 172.21.15.45:0/2349852177 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a83797cb0 con 0x563a8381cd90 2023-10-17T01:25:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f9781ffb700 1 -- 172.21.15.45:0/2349852177 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f9794029b30 con 0x563a8381cd90 2023-10-17T01:25:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f97b496cb80 10 client.26582 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f97b496cb80 10 client.26582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f97b496cb80 20 client.26582 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83831160 0x563a83833580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f97b496cb80 4 client.26582 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a837eacd0 con 0x563a83831160 2023-10-17T01:25:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.265+0000 7f979a59c700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83831160 0x563a83833580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.266+0000 7f979a59c700 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83831160 0x563a83833580 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4d8fb3e8f56836 server_cookie=601a8e215e6bf24 in_seq=0 out_seq=0 2023-10-17T01:25:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.266+0000 7f9781ffb700 10 client.26582 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f9781ffb700 1 -- 172.21.15.45:0/2349852177 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x563a837eacd0 con 0x563a83831160 2023-10-17T01:25:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f9781ffb700 10 client.26582 handle_command_reply: tid=0 2023-10-17T01:25:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 client.26582 shutdown 2023-10-17T01:25:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 msgr2=0x563a8382c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a8382c930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9794003c60 tx=0x7f979400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 shutdown_connections 2023-10-17T01:25:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83831160 0x563a83833580 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a8382c930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f9781ffb700 0 client.26582 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a838253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2349852177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a838258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 >> 172.21.15.45:0/2349852177 conn(0x563a8377b070 msgr2=0x563a83815a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 shutdown_connections 2023-10-17T01:25:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.267+0000 7f97b496cb80 1 -- 172.21.15.45:0/2349852177 wait complete. 2023-10-17T01:25:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.268+0000 7f97b496cb80 20 client.26582 trim_cache size 0 max 16384 2023-10-17T01:25:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.289+0000 7f97b496cb80 1 Processor -- start 2023-10-17T01:25:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.289+0000 7f97b496cb80 1 -- start start 2023-10-17T01:25:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a83815a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83824060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f97b496cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83824f60 con 0x563a8381b830 2023-10-17T01:25:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f97b496cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83810990 con 0x563a8381cd90 2023-10-17T01:25:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f97b496cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83820890 con 0x563a8381c190 2023-10-17T01:25:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f9799d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a83815a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f9799d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a83815a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45260/0 (socket says 172.21.15.45:45260) 2023-10-17T01:25:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f979a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f9799d9b700 1 -- 172.21.15.45:0/729262503 learned_addr learned my addr 172.21.15.45:0/729262503 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.290+0000 7f979ad9d700 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83824060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.291+0000 7f9799d9b700 1 -- 172.21.15.45:0/729262503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 msgr2=0x563a83824060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.291+0000 7f9799d9b700 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83824060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.291+0000 7f9799d9b700 1 -- 172.21.15.45:0/729262503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 msgr2=0x563a8381d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.291+0000 7f9799d9b700 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.291+0000 7f9799d9b700 1 -- 172.21.15.45:0/729262503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a835c11f0 con 0x563a8381c190 2023-10-17T01:25:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.291+0000 7f9799d9b700 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a83815a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f97840045f0 tx=0x7f978400bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d39ad597665a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.292+0000 7f97917fa700 1 -- 172.21.15.45:0/729262503 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784003940 con 0x563a8381c190 2023-10-17T01:25:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.292+0000 7f97917fa700 1 -- 172.21.15.45:0/729262503 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9784017040 con 0x563a8381c190 2023-10-17T01:25:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.292+0000 7f97917fa700 1 -- 172.21.15.45:0/729262503 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978401b3f0 con 0x563a8381c190 2023-10-17T01:25:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 -- 172.21.15.45:0/729262503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 msgr2=0x563a83815a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a83815a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f97840045f0 tx=0x7f978400bba0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 -- 172.21.15.45:0/729262503 shutdown_connections 2023-10-17T01:25:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83824060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381b830 0x563a8381d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 --2- 172.21.15.45:0/729262503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381c190 0x563a83815a00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 -- 172.21.15.45:0/729262503 >> 172.21.15.45:0/729262503 conn(0x563a8377b070 msgr2=0x563a83816290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 -- 172.21.15.45:0/729262503 shutdown_connections 2023-10-17T01:25:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.293+0000 7f97b496cb80 1 -- 172.21.15.45:0/729262503 wait complete. 2023-10-17T01:25:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 -- start start 2023-10-17T01:25:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 Processor -- start 2023-10-17T01:25:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381b830 0x563a8387ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381c190 0x563a8387b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83881fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83882510 con 0x563a8381c190 2023-10-17T01:25:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83882650 con 0x563a8381cd90 2023-10-17T01:25:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.294+0000 7f97b496cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a83882810 con 0x563a8381b830 2023-10-17T01:25:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f97b496cb80 10 client.0 _ll_register_callbacks cb 0x7f97940077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381b830 0x563a8387ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381b830 0x563a8387ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45270/0 (socket says 172.21.15.45:45270) 2023-10-17T01:25:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979a59c700 1 -- 172.21.15.45:0/2218051554 learned_addr learned my addr 172.21.15.45:0/2218051554 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83881fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f9799d9b700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381c190 0x563a8387b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979ad9d700 1 -- 172.21.15.45:0/2218051554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381b830 msgr2=0x563a8387ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381b830 0x563a8387ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.295+0000 7f979ad9d700 1 -- 172.21.15.45:0/2218051554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381c190 msgr2=0x563a8387b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381c190 0x563a8387b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f979ad9d700 1 -- 172.21.15.45:0/2218051554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9784004270 con 0x563a8381cd90 2023-10-17T01:25:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f9799d9b700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381c190 0x563a8387b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f979ad9d700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83881fd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9794029e90 tx=0x7f979401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adbc8b8ab3eaf4ef server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f9781ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f9781ffb700 1 -- 172.21.15.45:0/2218051554 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400cec0 con 0x563a8381cd90 2023-10-17T01:25:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f978c0d1900 con 0x563a8381cd90 2023-10-17T01:25:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f9781ffb700 1 -- 172.21.15.45:0/2218051554 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f979401d570 con 0x563a8381cd90 2023-10-17T01:25:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f9781ffb700 1 -- 172.21.15.45:0/2218051554 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f979401d7b0 con 0x563a8381cd90 2023-10-17T01:25:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.296+0000 7f97b496cb80 10 client.26588 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.297+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563a83599250 con 0x563a8381cd90 2023-10-17T01:25:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.297+0000 7f9781ffb700 1 -- 172.21.15.45:0/2218051554 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978c0d1900 con 0x563a8381cd90 2023-10-17T01:25:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.298+0000 7f9781ffb700 1 -- 172.21.15.45:0/2218051554 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f979403d440 con 0x563a8381cd90 2023-10-17T01:25:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f97b496cb80 10 client.26588 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f97b496cb80 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f97b496cb80 20 client.26588 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83883e80 0x563a83886270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f97b496cb80 4 client.26588 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:25:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a83765ce0 con 0x563a83883e80 2023-10-17T01:25:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.299+0000 7f979a59c700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83883e80 0x563a83886270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.300+0000 7f979a59c700 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83883e80 0x563a83886270 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69226bcb011dbdc4 server_cookie=a353b31f0add58af in_seq=0 out_seq=0 2023-10-17T01:25:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.300+0000 7f9781ffb700 10 client.26588 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.313+0000 7f9781ffb700 1 -- 172.21.15.45:0/2218051554 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a83765ce0 con 0x563a83883e80 2023-10-17T01:25:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.313+0000 7f9781ffb700 10 client.26588 handle_command_reply: tid=0 2023-10-17T01:25:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.313+0000 7f97b496cb80 1 client.26588 shutdown 2023-10-17T01:25:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 msgr2=0x563a83881fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83881fd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9794029e90 tx=0x7f979401e000 comp rx=0 tx=0).stop 2023-10-17T01:25:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 shutdown_connections 2023-10-17T01:25:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x563a83883e80 0x563a83886270 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a8381cd90 0x563a83881fd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8381b830 0x563a8387ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 --2- 172.21.15.45:0/2218051554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a8381c190 0x563a8387b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 >> 172.21.15.45:0/2218051554 conn(0x563a8377b070 msgr2=0x563a838231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.314+0000 7f9781ffb700 0 client.26588 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 shutdown_connections 2023-10-17T01:25:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97b496cb80 1 -- 172.21.15.45:0/2218051554 wait complete. 2023-10-17T01:25:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97b496cb80 20 client.26588 trim_cache size 0 max 16384 2023-10-17T01:25:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:25:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:25:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:25:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:25:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97880a0ba0 msgr2=0x7f97880a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97880a0ba0 0x7f97880a3060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9794009fd0 tx=0x7f9794006210 comp rx=0 tx=0).stop 2023-10-17T01:25:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 msgr2=0x7f979c19d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c19d240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f978c00d740 tx=0x7f978c00da50 comp rx=0 tx=0).stop 2023-10-17T01:25:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 shutdown_connections 2023-10-17T01:25:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f97880a0ba0 0x7f97880a3060 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979c108f20 0x7f979c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f979c107390 0x7f979c19d240 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 --2- 172.21.15.45:0/1614336481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c108580 0x7f979c19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.315+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 >> 172.21.15.45:0/1614336481 conn(0x7f979c076330 msgr2=0x7f979c10ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.316+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 shutdown_connections 2023-10-17T01:25:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:18.316+0000 7f97a0fff700 1 -- 172.21.15.45:0/1614336481 wait complete. 2023-10-17T01:25:19.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:18 smithi060 ceph-mon[134136]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2023-10-17T01:25:19.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:18 smithi181 ceph-mon[134520]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2023-10-17T01:25:19.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:18 smithi045 ceph-mon[98474]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2023-10-17T01:25:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 610.0 s, 1447.0 tps, lat 68.539 ms stddev 82.772 2023-10-17T01:25:21.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:20 smithi045 ceph-mon[98474]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 34 MiB/s wr, 2.11k op/s 2023-10-17T01:25:21.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:20 smithi060 ceph-mon[134136]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 34 MiB/s wr, 2.11k op/s 2023-10-17T01:25:21.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:20 smithi181 ceph-mon[134520]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 34 MiB/s wr, 2.11k op/s 2023-10-17T01:25:23.082 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:22 smithi060 ceph-mon[134136]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2023-10-17T01:25:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:22 smithi181 ceph-mon[134520]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2023-10-17T01:25:23.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:22 smithi045 ceph-mon[98474]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2023-10-17T01:25:25.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:24 smithi060 ceph-mon[134136]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.85k op/s 2023-10-17T01:25:25.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:24 smithi181 ceph-mon[134520]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.85k op/s 2023-10-17T01:25:25.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:24 smithi045 ceph-mon[98474]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.85k op/s 2023-10-17T01:25:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 615.0 s, 1658.8 tps, lat 60.693 ms stddev 67.090 2023-10-17T01:25:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:26 smithi181 ceph-mon[134520]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-10-17T01:25:27.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:26 smithi045 ceph-mon[98474]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-10-17T01:25:27.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:26 smithi060 ceph-mon[134136]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-10-17T01:25:29.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:28 smithi060 ceph-mon[134136]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 819 op/s 2023-10-17T01:25:29.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:28 smithi181 ceph-mon[134520]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 819 op/s 2023-10-17T01:25:29.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:28 smithi045 ceph-mon[98474]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 819 op/s 2023-10-17T01:25:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 620.0 s, 1639.2 tps, lat 61.133 ms stddev 74.710 2023-10-17T01:25:31.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:30 smithi060 ceph-mon[134136]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 870 op/s 2023-10-17T01:25:31.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:30 smithi181 ceph-mon[134520]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 870 op/s 2023-10-17T01:25:31.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:30 smithi045 ceph-mon[98474]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 870 op/s 2023-10-17T01:25:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 -- 172.21.15.45:0/3530689537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec1028f0 msgr2=0x7f45ec1091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 --2- 172.21.15.45:0/3530689537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec1028f0 0x7f45ec1091a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009fc0 tx=0x7f45e400b330 comp rx=0 tx=0).stop 2023-10-17T01:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 -- 172.21.15.45:0/3530689537 shutdown_connections 2023-10-17T01:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 --2- 172.21.15.45:0/3530689537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec1028f0 0x7f45ec1091a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 --2- 172.21.15.45:0/3530689537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec100d60 0x7f45ec101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 --2- 172.21.15.45:0/3530689537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ec101f50 0x7f45ec1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.569+0000 7f45f19ff700 1 -- 172.21.15.45:0/3530689537 >> 172.21.15.45:0/3530689537 conn(0x7f45ec0fc290 msgr2=0x7f45ec0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.570+0000 7f45f19ff700 1 -- 172.21.15.45:0/3530689537 shutdown_connections 2023-10-17T01:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.570+0000 7f45f19ff700 1 -- 172.21.15.45:0/3530689537 wait complete. 2023-10-17T01:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.572+0000 7f45f19ff700 1 Processor -- start 2023-10-17T01:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.573+0000 7f45f19ff700 1 -- start start 2023-10-17T01:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45f19ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ec100d60 0x7f45ec198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45f19ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec101f50 0x7f45ec199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45f19ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 0x7f45ec19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45f19ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec19a8c0 con 0x7f45ec101f50 2023-10-17T01:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45f19ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec19aa30 con 0x7f45ec1028f0 2023-10-17T01:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45f19ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec19abf0 con 0x7f45ec100d60 2023-10-17T01:25:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45ea7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec101f50 0x7f45ec199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 0x7f45ec19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 0x7f45ec19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38758/0 (socket says 172.21.15.45:38758) 2023-10-17T01:25:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.574+0000 7f45eb7fe700 1 -- 172.21.15.45:0/124919486 learned_addr learned my addr 172.21.15.45:0/124919486 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.575+0000 7f45eb7fe700 1 -- 172.21.15.45:0/124919486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ec100d60 msgr2=0x7f45ec198ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-17T01:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.575+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ec100d60 0x7f45ec198ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.575+0000 7f45eb7fe700 1 -- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec101f50 msgr2=0x7f45ec199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.575+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec101f50 0x7f45ec199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.575+0000 7f45eb7fe700 1 -- 172.21.15.45:0/124919486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e4009c40 con 0x7f45ec1028f0 2023-10-17T01:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.575+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 0x7f45ec19a1b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f45e40094b0 tx=0x7f45e400bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.576+0000 7f45f09fd700 1 -- 172.21.15.45:0/124919486 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400e040 con 0x7f45ec1028f0 2023-10-17T01:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.576+0000 7f45f09fd700 1 -- 172.21.15.45:0/124919486 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e4004e40 con 0x7f45ec1028f0 2023-10-17T01:25:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.576+0000 7f45f09fd700 1 -- 172.21.15.45:0/124919486 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e401b860 con 0x7f45ec1028f0 2023-10-17T01:25:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.576+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ec19ae70 con 0x7f45ec1028f0 2023-10-17T01:25:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.576+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ec1a15e0 con 0x7f45ec1028f0 2023-10-17T01:25:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.582+0000 7f45f09fd700 1 -- 172.21.15.45:0/124919486 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f45e401d470 con 0x7f45ec1028f0 2023-10-17T01:25:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.586+0000 7f45f09fd700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45d40a0c70 0x7f45d40a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.587+0000 7f45f09fd700 1 -- 172.21.15.45:0/124919486 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f45e40cd540 con 0x7f45ec1028f0 2023-10-17T01:25:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.587+0000 7f45eaffd700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45d40a0c70 0x7f45d40a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.588+0000 7f45eaffd700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45d40a0c70 0x7f45d40a3130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f45e0005950 tx=0x7f45e00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.590+0000 7f4605253b80 1 Processor -- start 2023-10-17T01:25:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 -- start start 2023-10-17T01:25:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c05f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c05429b0 con 0x5607c05f39a0 2023-10-17T01:25:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c0540370 con 0x5607c05f45a0 2023-10-17T01:25:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.591+0000 7f4605253b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c053bc60 con 0x5607c05f3040 2023-10-17T01:25:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.592+0000 7f45eaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c05f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.592+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.592+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38766/0 (socket says 172.21.15.45:38766) 2023-10-17T01:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.592+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3218524839 learned_addr learned my addr 172.21.15.45:0/3218524839 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45ea7fc700 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45eaffd700 1 -- 172.21.15.45:0/3218524839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 msgr2=0x5607c05f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45eaffd700 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45eaffd700 1 -- 172.21.15.45:0/3218524839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 msgr2=0x5607c05f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45eaffd700 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45eaffd700 1 -- 172.21.15.45:0/3218524839 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c0369d80 con 0x5607c05f3040 2023-10-17T01:25:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.593+0000 7f45ea7fc700 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.595+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.596+0000 7f45eaffd700 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c05f3460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f45ec198c20 tx=0x7f45e0010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f904bad266fb89ea server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.596+0000 7f45d1ffb700 1 -- 172.21.15.45:0/3218524839 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0010ac0 con 0x5607c05f3040 2023-10-17T01:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.596+0000 7f45d1ffb700 1 -- 172.21.15.45:0/3218524839 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e000dec0 con 0x5607c05f3040 2023-10-17T01:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.596+0000 7f45d1ffb700 1 -- 172.21.15.45:0/3218524839 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e000f620 con 0x5607c05f3040 2023-10-17T01:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.597+0000 7f4605253b80 1 -- 172.21.15.45:0/3218524839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 msgr2=0x5607c05f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.597+0000 7f4605253b80 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c05f3460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f45ec198c20 tx=0x7f45e0010000 comp rx=0 tx=0).stop 2023-10-17T01:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 -- 172.21.15.45:0/3218524839 shutdown_connections 2023-10-17T01:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c05f3460 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 --2- 172.21.15.45:0/3218524839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 -- 172.21.15.45:0/3218524839 >> 172.21.15.45:0/3218524839 conn(0x5607c054a070 msgr2=0x5607c05e52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 -- 172.21.15.45:0/3218524839 shutdown_connections 2023-10-17T01:25:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.598+0000 7f4605253b80 1 -- 172.21.15.45:0/3218524839 wait complete. 2023-10-17T01:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.599+0000 7f4605253b80 1 -- start start 2023-10-17T01:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.599+0000 7f4605253b80 1 Processor -- start 2023-10-17T01:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 0x5607c05fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f45a0 0x5607c06040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c04d84b0 con 0x5607c05f39a0 2023-10-17T01:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c04d8150 con 0x5607c05f3040 2023-10-17T01:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c03bf8c0 con 0x5607c05f45a0 2023-10-17T01:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f4605253b80 10 client.0 _ll_register_callbacks cb 0x5607c0425a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.600+0000 7f45eaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 0x5607c05fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 0x5607c05fcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38774/0 (socket says 172.21.15.45:38774) 2023-10-17T01:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 -- 172.21.15.45:0/1880395291 learned_addr learned my addr 172.21.15.45:0/1880395291 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 -- 172.21.15.45:0/1880395291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f45a0 msgr2=0x5607c06040f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f45a0 0x5607c06040f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 -- 172.21.15.45:0/1880395291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 msgr2=0x5607c05fd0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05fd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.601+0000 7f45eaffd700 1 -- 172.21.15.45:0/1880395291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ec1a15e0 con 0x5607c05f3040 2023-10-17T01:25:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.602+0000 7f45eaffd700 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 0x5607c05fcb70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f45e001efb0 tx=0x7f45e0010970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc790f166db9ae85 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.602+0000 7f45caffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.604+0000 7f45caffd700 1 -- 172.21.15.45:0/1880395291 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0010e40 con 0x5607c05f3040 2023-10-17T01:25:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.604+0000 7f45caffd700 1 -- 172.21.15.45:0/1880395291 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f45e001d440 con 0x5607c05f3040 2023-10-17T01:25:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.605+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607c05b9cd0 con 0x5607c05f3040 2023-10-17T01:25:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.606+0000 7f45caffd700 1 -- 172.21.15.45:0/1880395291 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e001d640 con 0x5607c05f3040 2023-10-17T01:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.606+0000 7f4605253b80 10 client.26600 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.606+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f45e000d800 con 0x5607c05f3040 2023-10-17T01:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f45caffd700 1 -- 172.21.15.45:0/1880395291 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607c05b9cd0 con 0x5607c05f3040 2023-10-17T01:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f45caffd700 1 -- 172.21.15.45:0/1880395291 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f45e0004990 con 0x5607c05f3040 2023-10-17T01:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f4605253b80 10 client.26600 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f4605253b80 10 client.26600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f4605253b80 20 client.26600 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f4605253b80 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c05ed6b0 0x5607c05efad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f4605253b80 4 client.26600 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607c0534ce0 con 0x5607c05ed6b0 2023-10-17T01:25:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.607+0000 7f45ea7fc700 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c05ed6b0 0x5607c05efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.608+0000 7f45ea7fc700 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c05ed6b0 0x5607c05efad0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fb4586dd3735a1 server_cookie=7245e41fcbf20eb6 in_seq=0 out_seq=0 2023-10-17T01:25:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.608+0000 7f45caffd700 10 client.26600 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.609+0000 7f45caffd700 1 -- 172.21.15.45:0/1880395291 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5607c0534ce0 con 0x5607c05ed6b0 2023-10-17T01:25:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.609+0000 7f45caffd700 10 client.26600 handle_command_reply: tid=0 2023-10-17T01:25:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.609+0000 7f4605253b80 1 client.26600 shutdown 2023-10-17T01:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 msgr2=0x5607c05fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 0x5607c05fcb70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f45e001efb0 tx=0x7f45e0010970 comp rx=0 tx=0).stop 2023-10-17T01:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 shutdown_connections 2023-10-17T01:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f45a0 0x5607c06040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c05ed6b0 0x5607c05efad0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f3040 0x5607c05fcb70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 --2- 172.21.15.45:0/1880395291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c05fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f45caffd700 0 client.26600 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.610+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 >> 172.21.15.45:0/1880395291 conn(0x5607c054a070 msgr2=0x5607c05e4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.611+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 shutdown_connections 2023-10-17T01:25:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.611+0000 7f4605253b80 1 -- 172.21.15.45:0/1880395291 wait complete. 2023-10-17T01:25:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.611+0000 7f4605253b80 20 client.26600 trim_cache size 0 max 16384 2023-10-17T01:25:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 Processor -- start 2023-10-17T01:25:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 -- start start 2023-10-17T01:25:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f3040 0x5607c05e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f39a0 0x5607c05fb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c05fb630 con 0x5607c05f3040 2023-10-17T01:25:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c05fdc00 con 0x5607c05f45a0 2023-10-17T01:25:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.635+0000 7f4605253b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c05f4400 con 0x5607c05f39a0 2023-10-17T01:25:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38778/0 (socket says 172.21.15.45:38778) 2023-10-17T01:25:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45ea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f39a0 0x5607c05fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 -- 172.21.15.45:0/1502275134 learned_addr learned my addr 172.21.15.45:0/1502275134 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eaffd700 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f3040 0x5607c05e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 -- 172.21.15.45:0/1502275134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f39a0 msgr2=0x5607c05fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f39a0 0x5607c05fb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 -- 172.21.15.45:0/1502275134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f3040 msgr2=0x5607c05e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f3040 0x5607c05e4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.636+0000 7f45eb7fe700 1 -- 172.21.15.45:0/1502275134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c0352c00 con 0x5607c05f45a0 2023-10-17T01:25:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.637+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05faa00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f45e4007a10 tx=0x7f45e40caee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6edce05b12dbaa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.637+0000 7f45d27fc700 1 -- 172.21.15.45:0/1502275134 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e409b2f0 con 0x5607c05f45a0 2023-10-17T01:25:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.637+0000 7f45d27fc700 1 -- 172.21.15.45:0/1502275134 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e409abf0 con 0x5607c05f45a0 2023-10-17T01:25:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.637+0000 7f45d27fc700 1 -- 172.21.15.45:0/1502275134 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e409ae50 con 0x5607c05f45a0 2023-10-17T01:25:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 -- 172.21.15.45:0/1502275134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 msgr2=0x5607c05faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05faa00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f45e4007a10 tx=0x7f45e40caee0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 -- 172.21.15.45:0/1502275134 shutdown_connections 2023-10-17T01:25:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c05faa00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f3040 0x5607c05e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 --2- 172.21.15.45:0/1502275134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f39a0 0x5607c05fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 -- 172.21.15.45:0/1502275134 >> 172.21.15.45:0/1502275134 conn(0x5607c054a070 msgr2=0x5607c05e5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 -- 172.21.15.45:0/1502275134 shutdown_connections 2023-10-17T01:25:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 -- 172.21.15.45:0/1502275134 wait complete. 2023-10-17T01:25:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.638+0000 7f4605253b80 1 -- start start 2023-10-17T01:25:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.639+0000 7f4605253b80 1 Processor -- start 2023-10-17T01:25:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c065d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c065f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c0661df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c06623d0 con 0x5607c05f39a0 2023-10-17T01:25:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c0662540 con 0x5607c05f45a0 2023-10-17T01:25:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c0673710 con 0x5607c05f3040 2023-10-17T01:25:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f4605253b80 10 client.0 _ll_register_callbacks cb 0x7f45e40b8f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c0661df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f45eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c0661df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38782/0 (socket says 172.21.15.45:38782) 2023-10-17T01:25:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.640+0000 7f45eb7fe700 1 -- 172.21.15.45:0/157157593 learned_addr learned my addr 172.21.15.45:0/157157593 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eaffd700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c065d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45ea7fc700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c065f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eb7fe700 1 -- 172.21.15.45:0/157157593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 msgr2=0x5607c065d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c065d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eb7fe700 1 -- 172.21.15.45:0/157157593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 msgr2=0x5607c065f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c065f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eb7fe700 1 -- 172.21.15.45:0/157157593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e402c070 con 0x5607c05f45a0 2023-10-17T01:25:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45ea7fc700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c065f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:25:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c0661df0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f45e40991e0 tx=0x7f45e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10b95695ad193ea server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.641+0000 7f45caffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f45caffd700 1 -- 172.21.15.45:0/157157593 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e409b2f0 con 0x5607c05f45a0 2023-10-17T01:25:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f45caffd700 1 -- 172.21.15.45:0/157157593 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f45e40a2bd0 con 0x5607c05f45a0 2023-10-17T01:25:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607c0566cb0 con 0x5607c05f45a0 2023-10-17T01:25:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f45caffd700 1 -- 172.21.15.45:0/157157593 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e407b080 con 0x5607c05f45a0 2023-10-17T01:25:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f4605253b80 10 client.26606 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5607c03901f0 con 0x5607c05f45a0 2023-10-17T01:25:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f45caffd700 1 -- 172.21.15.45:0/157157593 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607c0566cb0 con 0x5607c05f45a0 2023-10-17T01:25:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.642+0000 7f45caffd700 1 -- 172.21.15.45:0/157157593 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f45e40a2830 con 0x5607c05f45a0 2023-10-17T01:25:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.643+0000 7f4605253b80 10 client.26606 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.643+0000 7f4605253b80 10 client.26606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.643+0000 7f4605253b80 20 client.26606 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.643+0000 7f4605253b80 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c0674bf0 0x5607c0677010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.643+0000 7f4605253b80 4 client.26606 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:25:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.643+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607c0534a00 con 0x5607c0674bf0 2023-10-17T01:25:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.644+0000 7f45eaffd700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c0674bf0 0x5607c0677010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.645+0000 7f45eaffd700 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c0674bf0 0x5607c0677010 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c1a183b7a15b90 server_cookie=6d87749179cfdf27 in_seq=0 out_seq=0 2023-10-17T01:25:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.645+0000 7f45caffd700 10 client.26606 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f45caffd700 1 -- 172.21.15.45:0/157157593 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607c0534a00 con 0x5607c0674bf0 2023-10-17T01:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f45caffd700 10 client.26606 handle_command_reply: tid=0 2023-10-17T01:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 client.26606 shutdown 2023-10-17T01:25:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 msgr2=0x5607c0661df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c0661df0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f45e40991e0 tx=0x7f45e400b330 comp rx=0 tx=0).stop 2023-10-17T01:25:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 shutdown_connections 2023-10-17T01:25:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5607c0674bf0 0x5607c0677010 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607c05f45a0 0x5607c0661df0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5607c05f3040 0x5607c065d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 --2- 172.21.15.45:0/157157593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607c05f39a0 0x5607c065f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f45caffd700 0 client.26606 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.657+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 >> 172.21.15.45:0/157157593 conn(0x5607c054a070 msgr2=0x5607c05faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.658+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 shutdown_connections 2023-10-17T01:25:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.658+0000 7f4605253b80 1 -- 172.21.15.45:0/157157593 wait complete. 2023-10-17T01:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.658+0000 7f4605253b80 20 client.26606 trim_cache size 0 max 16384 2023-10-17T01:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45d40a0c70 msgr2=0x7f45d40a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45d40a0c70 0x7f45d40a3130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f45e0005950 tx=0x7f45e00058e0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 msgr2=0x7f45ec19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 0x7f45ec19a1b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f45e40094b0 tx=0x7f45e400bcd0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 shutdown_connections 2023-10-17T01:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f45d40a0c70 0x7f45d40a3130 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45ec1028f0 0x7f45ec19a1b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ec100d60 0x7f45ec198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 --2- 172.21.15.45:0/124919486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45ec101f50 0x7f45ec199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.659+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 >> 172.21.15.45:0/124919486 conn(0x7f45ec0fc290 msgr2=0x7f45ec0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.660+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 shutdown_connections 2023-10-17T01:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:31.660+0000 7f45f19ff700 1 -- 172.21.15.45:0/124919486 wait complete. 2023-10-17T01:25:32.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:31 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:25:32.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:31 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:25:32.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:31 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:25:33.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:32 smithi181 ceph-mon[134520]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-10-17T01:25:33.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:32 smithi045 ceph-mon[98474]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-10-17T01:25:33.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:32 smithi060 ceph-mon[134136]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 648 op/s 2023-10-17T01:25:35.085 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:34 smithi060 ceph-mon[134136]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 MiB/s wr, 711 op/s 2023-10-17T01:25:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:34 smithi181 ceph-mon[134520]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 MiB/s wr, 711 op/s 2023-10-17T01:25:35.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:34 smithi045 ceph-mon[98474]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 MiB/s wr, 711 op/s 2023-10-17T01:25:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 625.0 s, 1526.8 tps, lat 65.009 ms stddev 82.346 2023-10-17T01:25:36.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:25:36.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:36.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:36 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:36.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:36.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:36 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.580+0000 7ff89bfff700 1 -- 172.21.15.45:0/3700340292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410a6c0 msgr2=0x7ff89410cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.580+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3700340292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410a6c0 0x7ff89410cb50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff89000b0a0 tx=0x7ff89000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.583 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:36.583 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:36 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 -- 172.21.15.45:0/3700340292 shutdown_connections 2023-10-17T01:25:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3700340292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff89410d090 0x7ff89410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3700340292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894107d50 0x7ff89410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3700340292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410a6c0 0x7ff89410cb50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 -- 172.21.15.45:0/3700340292 >> 172.21.15.45:0/3700340292 conn(0x7ff89406db00 msgr2=0x7ff89406ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 -- 172.21.15.45:0/3700340292 shutdown_connections 2023-10-17T01:25:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.582+0000 7ff89bfff700 1 -- 172.21.15.45:0/3700340292 wait complete. 2023-10-17T01:25:36.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.584+0000 7ff89bfff700 1 Processor -- start 2023-10-17T01:25:36.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 -- start start 2023-10-17T01:25:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 0x7ff8941a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff89410a6c0 0x7ff8941a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410d090 0x7ff8941acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941ad540 con 0x7ff894107d50 2023-10-17T01:25:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941ad680 con 0x7ff89410a6c0 2023-10-17T01:25:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.585+0000 7ff89bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941ad810 con 0x7ff89410d090 2023-10-17T01:25:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 0x7ff8941a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 0x7ff8941a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39382/0 (socket says 172.21.15.45:39382) 2023-10-17T01:25:36.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 -- 172.21.15.45:0/3840227127 learned_addr learned my addr 172.21.15.45:0/3840227127 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:36.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 -- 172.21.15.45:0/3840227127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410d090 msgr2=0x7ff8941acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410d090 0x7ff8941acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 -- 172.21.15.45:0/3840227127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff89410a6c0 msgr2=0x7ff8941a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff89410a6c0 0x7ff8941a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.587+0000 7ff899d9b700 1 -- 172.21.15.45:0/3840227127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890009d00 con 0x7ff894107d50 2023-10-17T01:25:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.592+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 0x7ff8941a59c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00d950 tx=0x7ff88c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.592+0000 7ff88affd700 1 -- 172.21.15.45:0/3840227127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c009b40 con 0x7ff894107d50 2023-10-17T01:25:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.592+0000 7ff88affd700 1 -- 172.21.15.45:0/3840227127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88c009ca0 con 0x7ff894107d50 2023-10-17T01:25:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.593+0000 7ff88affd700 1 -- 172.21.15.45:0/3840227127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00f620 con 0x7ff894107d50 2023-10-17T01:25:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.593+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8941adaf0 con 0x7ff894107d50 2023-10-17T01:25:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.593+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8941adfb0 con 0x7ff894107d50 2023-10-17T01:25:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.601+0000 7ff88affd700 1 -- 172.21.15.45:0/3840227127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff88c010460 con 0x7ff894107d50 2023-10-17T01:25:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.603+0000 7ff88affd700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff8800a0c70 0x7ff8800a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.605+0000 7ff8af980b80 1 Processor -- start 2023-10-17T01:25:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.606+0000 7ff8af980b80 1 -- start start 2023-10-17T01:25:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.606+0000 7ff89959a700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff8800a0c70 0x7ff8800a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.606+0000 7ff88affd700 1 -- 172.21.15.45:0/3840227127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff88c09a1a0 con 0x7ff894107d50 2023-10-17T01:25:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.607+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8616ba0 0x55feb8616fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.607+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8617500 0x55feb8617d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.607+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb86182b0 0x55feb8618730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.607+0000 7ff8af980b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb856ea10 con 0x55feb8616ba0 2023-10-17T01:25:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.607+0000 7ff8af980b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb856c3d0 con 0x55feb8617500 2023-10-17T01:25:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.607+0000 7ff8af980b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb8567cc0 con 0x55feb86182b0 2023-10-17T01:25:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.608+0000 7ff89959a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8617500 0x55feb8617d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.608+0000 7ff89959a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8617500 0x55feb8617d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:38804/0 (socket says 172.21.15.45:38804) 2023-10-17T01:25:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.608+0000 7ff89959a700 1 -- 172.21.15.45:0/132234529 learned_addr learned my addr 172.21.15.45:0/132234529 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.608+0000 7ff89959a700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff8800a0c70 0x7ff8800a3130 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff89000b070 tx=0x7ff89000b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.608+0000 7ff899d9b700 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8616ba0 0x55feb8616fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.609+0000 7ff899d9b700 1 -- 172.21.15.45:0/132234529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb86182b0 msgr2=0x55feb8618730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.609+0000 7ff899d9b700 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb86182b0 0x55feb8618730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.609+0000 7ff899d9b700 1 -- 172.21.15.45:0/132234529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8617500 msgr2=0x55feb8617d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.609+0000 7ff899d9b700 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8617500 0x55feb8617d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.610+0000 7ff899d9b700 1 -- 172.21.15.45:0/132234529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feb8395de0 con 0x55feb8616ba0 2023-10-17T01:25:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.611+0000 7ff899d9b700 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8616ba0 0x55feb8616fc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff88c094610 tx=0x7ff88c0a0560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a0378ecac1a2d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.612+0000 7ff87ffff700 1 -- 172.21.15.45:0/132234529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c009e10 con 0x55feb8616ba0 2023-10-17T01:25:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.612+0000 7ff87ffff700 1 -- 172.21.15.45:0/132234529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88c0b7e90 con 0x55feb8616ba0 2023-10-17T01:25:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.612+0000 7ff87ffff700 1 -- 172.21.15.45:0/132234529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0995e0 con 0x55feb8616ba0 2023-10-17T01:25:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.614+0000 7ff8af980b80 1 -- 172.21.15.45:0/132234529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8616ba0 msgr2=0x55feb8616fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.614+0000 7ff8af980b80 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8616ba0 0x55feb8616fc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff88c094610 tx=0x7ff88c0a0560 comp rx=0 tx=0).stop 2023-10-17T01:25:36.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.615+0000 7ff8af980b80 1 -- 172.21.15.45:0/132234529 shutdown_connections 2023-10-17T01:25:36.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.615+0000 7ff8af980b80 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb86182b0 0x55feb8618730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.615+0000 7ff8af980b80 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8616ba0 0x55feb8616fc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.615+0000 7ff8af980b80 1 --2- 172.21.15.45:0/132234529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8617500 0x55feb8617d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.615+0000 7ff8af980b80 1 -- 172.21.15.45:0/132234529 >> 172.21.15.45:0/132234529 conn(0x55feb85760d0 msgr2=0x55feb8611950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.615+0000 7ff8af980b80 1 -- 172.21.15.45:0/132234529 shutdown_connections 2023-10-17T01:25:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.616+0000 7ff8af980b80 1 -- 172.21.15.45:0/132234529 wait complete. 2023-10-17T01:25:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.616+0000 7ff8af980b80 1 -- start start 2023-10-17T01:25:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.617+0000 7ff8af980b80 1 Processor -- start 2023-10-17T01:25:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.618+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8620980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 0x55feb8620ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8625ef0 0x55feb86282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff8af980b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb8504510 con 0x55feb86182b0 2023-10-17T01:25:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff8af980b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb85041b0 con 0x55feb8625ef0 2023-10-17T01:25:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff8af980b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb83eb920 con 0x55feb8617500 2023-10-17T01:25:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff8af980b80 10 client.0 _ll_register_callbacks cb 0x55feb8451a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.619+0000 7ff89959a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 0x55feb8620ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 0x55feb8620ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39402/0 (socket says 172.21.15.45:39402) 2023-10-17T01:25:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 -- 172.21.15.45:0/2865387959 learned_addr learned my addr 172.21.15.45:0/2865387959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff899d9b700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8620980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 -- 172.21.15.45:0/2865387959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 msgr2=0x55feb8620980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8620980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 -- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8625ef0 msgr2=0x55feb86282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8625ef0 0x55feb86282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff89959a700 1 -- 172.21.15.45:0/2865387959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8941adfb0 con 0x55feb86182b0 2023-10-17T01:25:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55feb8592d10 con 0x55feb86182b0 2023-10-17T01:25:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.620+0000 7ff899d9b700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8620980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.621+0000 7ff89959a700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 0x55feb8620ec0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff890007930 tx=0x7ff8900099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6023c66d98e30f03 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.621+0000 7ff87d7fa700 10 client.16299 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:25:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.621+0000 7ff87d7fa700 1 -- 172.21.15.45:0/2865387959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89000e040 con 0x55feb86182b0 2023-10-17T01:25:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.622+0000 7ff87d7fa700 1 -- 172.21.15.45:0/2865387959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff890003c00 con 0x55feb86182b0 2023-10-17T01:25:36.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.622+0000 7ff87d7fa700 1 -- 172.21.15.45:0/2865387959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff890003e00 con 0x55feb86182b0 2023-10-17T01:25:36.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.622+0000 7ff8af980b80 10 client.16299 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:36.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.622+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x55feb837ec60 con 0x55feb86182b0 2023-10-17T01:25:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.622+0000 7ff87d7fa700 1 -- 172.21.15.45:0/2865387959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff89001d470 con 0x55feb86182b0 2023-10-17T01:25:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff8af980b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff8af980b80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff8af980b80 20 client.16299 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff8af980b80 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb862c8d0 0x55feb862ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff8af980b80 4 client.16299 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:36.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55feb85e5d30 con 0x55feb862c8d0 2023-10-17T01:25:36.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.623+0000 7ff899d9b700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb862c8d0 0x55feb862ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.624+0000 7ff87d7fa700 10 client.16299 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:36.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.624+0000 7ff899d9b700 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb862c8d0 0x55feb862ecf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159614ec0a172803 server_cookie=42b40471e650a853 in_seq=0 out_seq=0 2023-10-17T01:25:36.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.625+0000 7ff87d7fa700 1 -- 172.21.15.45:0/2865387959 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55feb85e5d30 con 0x55feb862c8d0 2023-10-17T01:25:36.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.625+0000 7ff87d7fa700 10 client.16299 handle_command_reply: tid=0 2023-10-17T01:25:36.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.625+0000 7ff8af980b80 1 client.16299 shutdown 2023-10-17T01:25:36.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 msgr2=0x55feb8620ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 0x55feb8620ec0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff890007930 tx=0x7ff8900099e0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 shutdown_connections 2023-10-17T01:25:36.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb862c8d0 0x55feb862ecf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb8625ef0 0x55feb86282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8620980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 --2- 172.21.15.45:0/2865387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb86182b0 0x55feb8620ec0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff87d7fa700 0 client.16299 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:36.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.626+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 >> 172.21.15.45:0/2865387959 conn(0x55feb85760d0 msgr2=0x55feb8611020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:36.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.628+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 shutdown_connections 2023-10-17T01:25:36.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.629+0000 7ff8af980b80 1 -- 172.21.15.45:0/2865387959 wait complete. 2023-10-17T01:25:36.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.630+0000 7ff8af980b80 20 client.16299 trim_cache size 0 max 16384 2023-10-17T01:25:36.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.670+0000 7ff8af980b80 1 Processor -- start 2023-10-17T01:25:36.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.670+0000 7ff8af980b80 1 -- start start 2023-10-17T01:25:36.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8628c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb861f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8625ef0 0x55feb861b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff8af980b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb862c6f0 con 0x55feb8625ef0 2023-10-17T01:25:36.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff8af980b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb86204f0 con 0x55feb86182b0 2023-10-17T01:25:36.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff8af980b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb860e420 con 0x55feb8617500 2023-10-17T01:25:36.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff899d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8628c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff899d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8628c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54494/0 (socket says 172.21.15.45:54494) 2023-10-17T01:25:36.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff899d9b700 1 -- 172.21.15.45:0/3617922404 learned_addr learned my addr 172.21.15.45:0/3617922404 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:36.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.671+0000 7ff89a59c700 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8625ef0 0x55feb861b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff899d9b700 1 -- 172.21.15.45:0/3617922404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 msgr2=0x55feb861f390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:25:36.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb861f390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff899d9b700 1 -- 172.21.15.45:0/3617922404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8625ef0 msgr2=0x55feb861b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8625ef0 0x55feb861b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff899d9b700 1 -- 172.21.15.45:0/3617922404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feb83bc250 con 0x55feb8617500 2023-10-17T01:25:36.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff89a59c700 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8625ef0 0x55feb861b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:36.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8628c00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff88c096a50 tx=0x7ff88c010c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b56ba41a863b5e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff888ff9700 1 -- 172.21.15.45:0/3617922404 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c097df0 con 0x55feb8617500 2023-10-17T01:25:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff888ff9700 1 -- 172.21.15.45:0/3617922404 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88c097f50 con 0x55feb8617500 2023-10-17T01:25:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.672+0000 7ff888ff9700 1 -- 172.21.15.45:0/3617922404 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c096410 con 0x55feb8617500 2023-10-17T01:25:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 -- 172.21.15.45:0/3617922404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 msgr2=0x55feb8628c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8628c00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff88c096a50 tx=0x7ff88c010c40 comp rx=0 tx=0).stop 2023-10-17T01:25:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 -- 172.21.15.45:0/3617922404 shutdown_connections 2023-10-17T01:25:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8625ef0 0x55feb861b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8617500 0x55feb8628c00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3617922404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb861f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.673+0000 7ff8af980b80 1 -- 172.21.15.45:0/3617922404 >> 172.21.15.45:0/3617922404 conn(0x55feb85760d0 msgr2=0x55feb86118c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.674+0000 7ff8af980b80 1 -- 172.21.15.45:0/3617922404 shutdown_connections 2023-10-17T01:25:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.674+0000 7ff8af980b80 1 -- 172.21.15.45:0/3617922404 wait complete. 2023-10-17T01:25:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.674+0000 7ff8af980b80 1 -- start start 2023-10-17T01:25:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.674+0000 7ff8af980b80 1 Processor -- start 2023-10-17T01:25:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8617500 0x55feb8678770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb867acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 0x55feb867d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb867da10 con 0x55feb8617500 2023-10-17T01:25:36.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb867db80 con 0x55feb86182b0 2023-10-17T01:25:36.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb867dd40 con 0x55feb8625ef0 2023-10-17T01:25:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff8af980b80 10 client.0 _ll_register_callbacks cb 0x7ff8900074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff89959a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb867acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff89a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 0x55feb867d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff89a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 0x55feb867d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54496/0 (socket says 172.21.15.45:54496) 2023-10-17T01:25:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.675+0000 7ff89a59c700 1 -- 172.21.15.45:0/3187366539 learned_addr learned my addr 172.21.15.45:0/3187366539 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.676+0000 7ff89a59c700 1 -- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 msgr2=0x55feb867acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.676+0000 7ff89a59c700 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb867acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.676+0000 7ff89a59c700 1 -- 172.21.15.45:0/3187366539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8617500 msgr2=0x55feb8678770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.676+0000 7ff89a59c700 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8617500 0x55feb8678770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.676+0000 7ff89a59c700 1 -- 172.21.15.45:0/3187366539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55feb8395de0 con 0x55feb8625ef0 2023-10-17T01:25:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.676+0000 7ff89a59c700 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 0x55feb867d2b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff884011690 tx=0x7ff8840119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e21b44a9500efa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff87d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:25:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff87d7fa700 1 -- 172.21.15.45:0/3187366539 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884004220 con 0x55feb8625ef0 2023-10-17T01:25:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff87d7fa700 1 -- 172.21.15.45:0/3187366539 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff884004380 con 0x55feb8625ef0 2023-10-17T01:25:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff88c0b7e90 con 0x55feb8625ef0 2023-10-17T01:25:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff87d7fa700 1 -- 172.21.15.45:0/3187366539 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff884004540 con 0x55feb8625ef0 2023-10-17T01:25:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff8af980b80 10 client.26569 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.677+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55feb83942b0 con 0x55feb8625ef0 2023-10-17T01:25:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff87d7fa700 1 -- 172.21.15.45:0/3187366539 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88c0b7e90 con 0x55feb8625ef0 2023-10-17T01:25:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff87d7fa700 1 -- 172.21.15.45:0/3187366539 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff884012a00 con 0x55feb8625ef0 2023-10-17T01:25:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff8af980b80 10 client.26569 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff8af980b80 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff8af980b80 20 client.26569 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb867f690 0x55feb8681ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff8af980b80 4 client.26569 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:25:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55feb8560d40 con 0x55feb867f690 2023-10-17T01:25:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.678+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb867f690 0x55feb8681ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.679+0000 7ff899d9b700 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb867f690 0x55feb8681ab0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2463e51ba3561e2 server_cookie=6e8c9aac67ccb501 in_seq=0 out_seq=0 2023-10-17T01:25:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.679+0000 7ff87d7fa700 10 client.26569 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.679+0000 7ff87d7fa700 1 -- 172.21.15.45:0/3187366539 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55feb8560d40 con 0x55feb867f690 2023-10-17T01:25:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.679+0000 7ff87d7fa700 10 client.26569 handle_command_reply: tid=0 2023-10-17T01:25:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.679+0000 7ff8af980b80 1 client.26569 shutdown 2023-10-17T01:25:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 msgr2=0x55feb867d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 0x55feb867d2b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff884011690 tx=0x7ff8840119a0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 shutdown_connections 2023-10-17T01:25:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55feb8625ef0 0x55feb867d2b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8617500 0x55feb8678770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55feb867f690 0x55feb8681ab0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 --2- 172.21.15.45:0/3187366539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55feb86182b0 0x55feb867acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff87d7fa700 0 client.26569 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 >> 172.21.15.45:0/3187366539 conn(0x55feb85760d0 msgr2=0x55feb8628c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 shutdown_connections 2023-10-17T01:25:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 1 -- 172.21.15.45:0/3187366539 wait complete. 2023-10-17T01:25:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.680+0000 7ff8af980b80 20 client.26569 trim_cache size 0 max 16384 2023-10-17T01:25:36.693 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:25:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:25:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:25:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:25:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:25:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:25:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:25:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:25:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:25:36.694 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:25:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff8800a0c70 msgr2=0x7ff8800a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff8800a0c70 0x7ff8800a3130 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff89000b070 tx=0x7ff89000b700 comp rx=0 tx=0).stop 2023-10-17T01:25:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 msgr2=0x7ff8941a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 0x7ff8941a59c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00d950 tx=0x7ff88c00dd10 comp rx=0 tx=0).stop 2023-10-17T01:25:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 shutdown_connections 2023-10-17T01:25:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff8800a0c70 0x7ff8800a3130 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89410d090 0x7ff8941acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff894107d50 0x7ff8941a59c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 --2- 172.21.15.45:0/3840227127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff89410a6c0 0x7ff8941a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.681+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 >> 172.21.15.45:0/3840227127 conn(0x7ff89406db00 msgr2=0x7ff8941089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.682+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 shutdown_connections 2023-10-17T01:25:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:25:36.682+0000 7ff89bfff700 1 -- 172.21.15.45:0/3840227127 wait complete. 2023-10-17T01:25:36.712 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:25:37.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:37 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:37 smithi045 ceph-mon[98474]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 616 op/s 2023-10-17T01:25:37.582 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.582 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.582 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.583 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:37 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.583 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:37 smithi060 ceph-mon[134136]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 616 op/s 2023-10-17T01:25:37.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:37 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:37.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:37 smithi181 ceph-mon[134520]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 616 op/s 2023-10-17T01:25:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:25:39.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:25:39.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:39.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:25:39.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:25:39.079 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:25:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:25:39.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:25:40.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:39 smithi045 ceph-mon[98474]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-10-17T01:25:40.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:39 smithi060 ceph-mon[134136]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-10-17T01:25:40.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:39 smithi181 ceph-mon[134520]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-10-17T01:25:40.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 630.0 s, 1662.2 tps, lat 59.867 ms stddev 75.815 2023-10-17T01:25:41.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:40 smithi060 ceph-mon[134136]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 692 op/s 2023-10-17T01:25:41.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:40 smithi181 ceph-mon[134520]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 692 op/s 2023-10-17T01:25:41.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:40 smithi045 ceph-mon[98474]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 692 op/s 2023-10-17T01:25:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:42 smithi181 ceph-mon[134520]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 586 op/s 2023-10-17T01:25:43.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:42 smithi045 ceph-mon[98474]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 586 op/s 2023-10-17T01:25:43.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:42 smithi060 ceph-mon[134136]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 586 op/s 2023-10-17T01:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 -- 172.21.15.45:0/4286665609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d8100940 msgr2=0x7f06d8100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/4286665609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d8100940 0x7f06d8100dc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009990 tx=0x7f06d0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 -- 172.21.15.45:0/4286665609 shutdown_connections 2023-10-17T01:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/4286665609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d8100940 0x7f06d8100dc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/4286665609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d80ff620 0x7f06d80ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/4286665609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80fff80 0x7f06d8100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.899+0000 7f06dc9ff700 1 -- 172.21.15.45:0/4286665609 >> 172.21.15.45:0/4286665609 conn(0x7f06d80fb1a0 msgr2=0x7f06d80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.900+0000 7f06dc9ff700 1 -- 172.21.15.45:0/4286665609 shutdown_connections 2023-10-17T01:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.900+0000 7f06dc9ff700 1 -- 172.21.15.45:0/4286665609 wait complete. 2023-10-17T01:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.902+0000 7f06dc9ff700 1 Processor -- start 2023-10-17T01:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.903+0000 7f06dc9ff700 1 -- start start 2023-10-17T01:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.904+0000 7f06dc9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ff620 0x7f06d819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.904+0000 7f06dc9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d80fff80 0x7f06d819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.904+0000 7f06dc9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8100940 0x7f06d819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.904+0000 7f06dc9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d819ead0 con 0x7f06d80ff620 2023-10-17T01:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.904+0000 7f06dc9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d819ec40 con 0x7f06d8100940 2023-10-17T01:25:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.904+0000 7f06dc9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d819ee00 con 0x7f06d80fff80 2023-10-17T01:25:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8100940 0x7f06d819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d80fff80 0x7f06d819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8100940 0x7f06d819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58950/0 (socket says 172.21.15.45:58950) 2023-10-17T01:25:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d6d9d700 1 -- 172.21.15.45:0/1162063907 learned_addr learned my addr 172.21.15.45:0/1162063907 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d5d9b700 1 -- 172.21.15.45:0/1162063907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8100940 msgr2=0x7f06d819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8100940 0x7f06d819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.905+0000 7f06d5d9b700 1 -- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ff620 msgr2=0x7f06d819d0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-17T01:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.906+0000 7f06d659c700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ff620 0x7f06d819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.906+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ff620 0x7f06d819d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.906+0000 7f06d5d9b700 1 -- 172.21.15.45:0/1162063907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00095f0 con 0x7f06d80fff80 2023-10-17T01:25:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.906+0000 7f06d659c700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ff620 0x7f06d819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:25:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.906+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d80fff80 0x7f06d819d5f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00d750 tx=0x7f06cc00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.907+0000 7f06c77fe700 1 -- 172.21.15.45:0/1162063907 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc004220 con 0x7f06d80fff80 2023-10-17T01:25:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.907+0000 7f06c77fe700 1 -- 172.21.15.45:0/1162063907 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06cc004cf0 con 0x7f06d80fff80 2023-10-17T01:25:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.907+0000 7f06c77fe700 1 -- 172.21.15.45:0/1162063907 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00f690 con 0x7f06d80fff80 2023-10-17T01:25:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.907+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d819f0e0 con 0x7f06d80fff80 2023-10-17T01:25:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.907+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d81a58a0 con 0x7f06d80fff80 2023-10-17T01:25:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.913+0000 7f06f0366b80 1 Processor -- start 2023-10-17T01:25:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.914+0000 7f06f0366b80 1 -- start start 2023-10-17T01:25:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.915+0000 7f06c77fe700 1 -- 172.21.15.45:0/1162063907 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f06cc004750 con 0x7f06d80fff80 2023-10-17T01:25:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.915+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 0x561aa96bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.915+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96be190 0x561aa96be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 0x561aa96bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 0x561aa96bdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58962/0 (socket says 172.21.15.45:58962) 2023-10-17T01:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 -- 172.21.15.45:0/1276876708 learned_addr learned my addr 172.21.15.45:0/1276876708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa96bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa96159b0 con 0x561aa96be190 2023-10-17T01:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa9613370 con 0x561aa96bd830 2023-10-17T01:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa960ec60 con 0x561aa96bed90 2023-10-17T01:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d659c700 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96be190 0x561aa96be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa96bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 -- 172.21.15.45:0/1276876708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 msgr2=0x561aa96bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa96bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 -- 172.21.15.45:0/1276876708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96be190 msgr2=0x561aa96be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96be190 0x561aa96be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.916+0000 7f06d6d9d700 1 -- 172.21.15.45:0/1276876708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aa943cd80 con 0x561aa96bd830 2023-10-17T01:25:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa96bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:25:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 0x561aa96bdc50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005c00 tx=0x7f06d0005c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42694a2b138d40a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06c4ff9700 1 -- 172.21.15.45:0/1276876708 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d001d050 con 0x561aa96bd830 2023-10-17T01:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06c4ff9700 1 -- 172.21.15.45:0/1276876708 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d001e040 con 0x561aa96bd830 2023-10-17T01:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06c4ff9700 1 -- 172.21.15.45:0/1276876708 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0005570 con 0x561aa96bd830 2023-10-17T01:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06c77fe700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06c00a0ba0 0x7f06c00a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.917+0000 7f06c77fe700 1 -- 172.21.15.45:0/1162063907 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f06cc0cbdc0 con 0x7f06d80fff80 2023-10-17T01:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06d659c700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06c00a0ba0 0x7f06c00a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 msgr2=0x561aa96bdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 0x561aa96bdc50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005c00 tx=0x7f06d0005c30 comp rx=0 tx=0).stop 2023-10-17T01:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 shutdown_connections 2023-10-17T01:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa96bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bd830 0x561aa96bdc50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1276876708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96be190 0x561aa96be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 >> 172.21.15.45:0/1276876708 conn(0x561aa961d070 msgr2=0x561aa96b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 shutdown_connections 2023-10-17T01:25:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.918+0000 7f06f0366b80 1 -- 172.21.15.45:0/1276876708 wait complete. 2023-10-17T01:25:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.919+0000 7f06d659c700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06c00a0ba0 0x7f06c00a3060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x561aa96bd5d0 tx=0x7f06c800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.920+0000 7f06f0366b80 1 -- start start 2023-10-17T01:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.921+0000 7f06f0366b80 1 Processor -- start 2023-10-17T01:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 0x561aa96c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96be190 0x561aa96c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bed90 0x561aa96ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa95ab4b0 con 0x561aa96bed90 2023-10-17T01:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa95ab150 con 0x561aa96be190 2023-10-17T01:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa94928c0 con 0x561aa96bd830 2023-10-17T01:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.923+0000 7f06f0366b80 10 client.0 _ll_register_callbacks cb 0x561aa94f8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 0x561aa96c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bed90 0x561aa96ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 0x561aa96c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43754/0 (socket says 172.21.15.45:43754) 2023-10-17T01:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96be190 0x561aa96c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d659c700 1 -- 172.21.15.45:0/1183747677 learned_addr learned my addr 172.21.15.45:0/1183747677 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 -- 172.21.15.45:0/1183747677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 msgr2=0x561aa96c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 0x561aa96c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 -- 172.21.15.45:0/1183747677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bed90 msgr2=0x561aa96ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bed90 0x561aa96ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 -- 172.21.15.45:0/1183747677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d810bcc0 con 0x561aa96be190 2023-10-17T01:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d659c700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 0x561aa96c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bed90 0x561aa96ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96be190 0x561aa96c7860 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0c9940 tx=0x7f06cc099050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1092eed6fab161cd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.924+0000 7f06bdffb700 1 -- 172.21.15.45:0/1183747677 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0057a0 con 0x561aa96be190 2023-10-17T01:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06bdffb700 1 -- 172.21.15.45:0/1183747677 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f06cc099440 con 0x561aa96be190 2023-10-17T01:25:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aa9639cb0 con 0x561aa96be190 2023-10-17T01:25:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06bdffb700 1 -- 172.21.15.45:0/1183747677 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06cc09d270 con 0x561aa96be190 2023-10-17T01:25:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 10 client.26627 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561aa9425c00 con 0x561aa96be190 2023-10-17T01:25:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06bdffb700 1 -- 172.21.15.45:0/1183747677 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06cc0b5080 con 0x561aa96be190 2023-10-17T01:25:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06bdffb700 1 -- 172.21.15.45:0/1183747677 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f06cc00e350 con 0x561aa96be190 2023-10-17T01:25:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 10 client.26627 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 20 client.26627 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa96d3080 0x561aa96d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 4 client.26627 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.925+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aa968ccd0 con 0x561aa96d3080 2023-10-17T01:25:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.926+0000 7f06d659c700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa96d3080 0x561aa96d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.926+0000 7f06d659c700 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa96d3080 0x561aa96d5470 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9d912464d46361 server_cookie=f85e48548a47f6d3 in_seq=0 out_seq=0 2023-10-17T01:25:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.926+0000 7f06bdffb700 10 client.26627 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.927+0000 7f06bdffb700 1 -- 172.21.15.45:0/1183747677 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x561aa968ccd0 con 0x561aa96d3080 2023-10-17T01:25:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.927+0000 7f06bdffb700 10 client.26627 handle_command_reply: tid=0 2023-10-17T01:25:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.928+0000 7f06f0366b80 1 client.26627 shutdown 2023-10-17T01:25:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96be190 msgr2=0x561aa96c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96be190 0x561aa96c7860 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0c9940 tx=0x7f06cc099050 comp rx=0 tx=0).stop 2023-10-17T01:25:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 shutdown_connections 2023-10-17T01:25:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bed90 0x561aa96ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bd830 0x561aa96c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa96d3080 0x561aa96d5470 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 --2- 172.21.15.45:0/1183747677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96be190 0x561aa96c7860 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0c9940 tx=0x7f06cc099050 comp rx=0 tx=0).stop 2023-10-17T01:25:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.929+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 >> 172.21.15.45:0/1183747677 conn(0x561aa961d070 msgr2=0x561aa96b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.930+0000 7f06bdffb700 0 client.26627 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.930+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 shutdown_connections 2023-10-17T01:25:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.930+0000 7f06f0366b80 1 -- 172.21.15.45:0/1183747677 wait complete. 2023-10-17T01:25:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.931+0000 7f06f0366b80 20 client.26627 trim_cache size 0 max 16384 2023-10-17T01:25:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.953+0000 7f06f0366b80 1 Processor -- start 2023-10-17T01:25:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.953+0000 7f06f0366b80 1 -- start start 2023-10-17T01:25:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa96c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bed90 0x561aa96b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96cc7b0 0x561aa96cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06f0366b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa96c8510 con 0x561aa96bd830 2023-10-17T01:25:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06f0366b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa96cecc0 con 0x561aa96bed90 2023-10-17T01:25:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06f0366b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa96c67d0 con 0x561aa96cc7b0 2023-10-17T01:25:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06d659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa96c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bed90 0x561aa96b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06d659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa96c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48990/0 (socket says 172.21.15.45:48990) 2023-10-17T01:25:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96cc7b0 0x561aa96cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 -- 172.21.15.45:0/116689834 learned_addr learned my addr 172.21.15.45:0/116689834 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.954+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bed90 0x561aa96b7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58986/0 (socket says 172.21.15.45:58986) 2023-10-17T01:25:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 -- 172.21.15.45:0/116689834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96cc7b0 msgr2=0x561aa96cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96cc7b0 0x561aa96cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 -- 172.21.15.45:0/116689834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bed90 msgr2=0x561aa96b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bed90 0x561aa96b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 -- 172.21.15.45:0/116689834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aa94631f0 con 0x561aa96bd830 2023-10-17T01:25:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96cc7b0 0x561aa96cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.955+0000 7f06d659c700 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa96c6000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f06c8007ea0 tx=0x7f06c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e30dcd2fb4a9f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06c57fa700 1 -- 172.21.15.45:0/116689834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8028cc0 con 0x561aa96bd830 2023-10-17T01:25:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06c57fa700 1 -- 172.21.15.45:0/116689834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06c8028e20 con 0x561aa96bd830 2023-10-17T01:25:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06c57fa700 1 -- 172.21.15.45:0/116689834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8017de0 con 0x561aa96bd830 2023-10-17T01:25:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 -- 172.21.15.45:0/116689834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 msgr2=0x561aa96c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa96c6000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f06c8007ea0 tx=0x7f06c8013000 comp rx=0 tx=0).stop 2023-10-17T01:25:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 -- 172.21.15.45:0/116689834 shutdown_connections 2023-10-17T01:25:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96cc7b0 0x561aa96cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa96c6000 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 --2- 172.21.15.45:0/116689834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96bed90 0x561aa96b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 -- 172.21.15.45:0/116689834 >> 172.21.15.45:0/116689834 conn(0x561aa961d070 msgr2=0x561aa96b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.956+0000 7f06f0366b80 1 -- 172.21.15.45:0/116689834 shutdown_connections 2023-10-17T01:25:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.957+0000 7f06f0366b80 1 -- 172.21.15.45:0/116689834 wait complete. 2023-10-17T01:25:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.957+0000 7f06f0366b80 1 -- start start 2023-10-17T01:25:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.957+0000 7f06f0366b80 1 Processor -- start 2023-10-17T01:25:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa971eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa97210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 0x561aa97236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa9723e00 con 0x561aa96bd830 2023-10-17T01:25:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa9723f70 con 0x561aa96cc7b0 2023-10-17T01:25:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa9724130 con 0x561aa96bed90 2023-10-17T01:25:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06f0366b80 10 client.0 _ll_register_callbacks cb 0x561aa94f8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa97210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 0x561aa97236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 0x561aa97236c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58994/0 (socket says 172.21.15.45:58994) 2023-10-17T01:25:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa97210f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43768/0 (socket says 172.21.15.45:43768) 2023-10-17T01:25:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 -- 172.21.15.45:0/702694638 learned_addr learned my addr 172.21.15.45:0/702694638 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d659c700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa971eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 -- 172.21.15.45:0/702694638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 msgr2=0x561aa97210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa97210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 -- 172.21.15.45:0/702694638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 msgr2=0x561aa971eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.958+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa971eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06d6d9d700 1 -- 172.21.15.45:0/702694638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aa943cd80 con 0x561aa96cc7b0 2023-10-17T01:25:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06d5d9b700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa97210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06d6d9d700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 0x561aa97236c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009350 tx=0x7f06d0009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ade2a76943df8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06bdffb700 1 -- 172.21.15.45:0/702694638 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0003d40 con 0x561aa96cc7b0 2023-10-17T01:25:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06d001e040 con 0x561aa96cc7b0 2023-10-17T01:25:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.959+0000 7f06bdffb700 1 -- 172.21.15.45:0/702694638 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f06d000f460 con 0x561aa96cc7b0 2023-10-17T01:25:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06bdffb700 1 -- 172.21.15.45:0/702694638 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d001d020 con 0x561aa96cc7b0 2023-10-17T01:25:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 10 client.26633 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f06c8011040 con 0x561aa96cc7b0 2023-10-17T01:25:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06bdffb700 1 -- 172.21.15.45:0/702694638 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d003f020 con 0x561aa96cc7b0 2023-10-17T01:25:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06bdffb700 1 -- 172.21.15.45:0/702694638 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f06d00235f0 con 0x561aa96cc7b0 2023-10-17T01:25:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 10 client.26633 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 10 client.26633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 20 client.26633 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa9725660 0x561aa9727a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 4 client.26633 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:25:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.960+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561aa9607ce0 con 0x561aa9725660 2023-10-17T01:25:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.961+0000 7f06d659c700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa9725660 0x561aa9727a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.962+0000 7f06d659c700 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa9725660 0x561aa9727a50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d77de297b8a352 server_cookie=d488e8b77eac960a in_seq=0 out_seq=0 2023-10-17T01:25:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.962+0000 7f06bdffb700 10 client.26633 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.977+0000 7f06bdffb700 1 -- 172.21.15.45:0/702694638 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561aa9607ce0 con 0x561aa9725660 2023-10-17T01:25:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.977+0000 7f06bdffb700 10 client.26633 handle_command_reply: tid=0 2023-10-17T01:25:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.977+0000 7f06f0366b80 1 client.26633 shutdown 2023-10-17T01:25:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 msgr2=0x561aa97236c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 0x561aa97236c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009350 tx=0x7f06d0009fd0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 shutdown_connections 2023-10-17T01:25:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x561aa9725660 0x561aa9727a50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561aa96cc7b0 0x561aa97236c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561aa96bd830 0x561aa971eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 --2- 172.21.15.45:0/702694638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561aa96bed90 0x561aa97210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 >> 172.21.15.45:0/702694638 conn(0x561aa961d070 msgr2=0x561aa96c5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.978+0000 7f06bdffb700 0 client.26633 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.979+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 shutdown_connections 2023-10-17T01:25:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.979+0000 7f06f0366b80 1 -- 172.21.15.45:0/702694638 wait complete. 2023-10-17T01:25:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.980+0000 7f06f0366b80 20 client.26633 trim_cache size 0 max 16384 2023-10-17T01:25:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:25:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:25:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:25:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:25:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06c00a0ba0 msgr2=0x7f06c00a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06c00a0ba0 0x7f06c00a3060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x561aa96bd5d0 tx=0x7f06c800b040 comp rx=0 tx=0).stop 2023-10-17T01:25:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d80fff80 msgr2=0x7f06d819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d80fff80 0x7f06d819d5f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00d750 tx=0x7f06cc00da60 comp rx=0 tx=0).stop 2023-10-17T01:25:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 shutdown_connections 2023-10-17T01:25:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f06c00a0ba0 0x7f06c00a3060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8100940 0x7f06d819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ff620 0x7f06d819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 --2- 172.21.15.45:0/1162063907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06d80fff80 0x7f06d819d5f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 >> 172.21.15.45:0/1162063907 conn(0x7f06d80fb1a0 msgr2=0x7f06d80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 shutdown_connections 2023-10-17T01:25:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:44.981+0000 7f06dc9ff700 1 -- 172.21.15.45:0/1162063907 wait complete. 2023-10-17T01:25:45.080 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:44 smithi060 ceph-mon[134136]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 669 op/s 2023-10-17T01:25:45.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:44 smithi181 ceph-mon[134520]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 669 op/s 2023-10-17T01:25:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:44 smithi045 ceph-mon[98474]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 669 op/s 2023-10-17T01:25:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 635.0 s, 1701.4 tps, lat 59.357 ms stddev 83.339 2023-10-17T01:25:47.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:46 smithi060 ceph-mon[134136]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 570 op/s 2023-10-17T01:25:47.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:46 smithi181 ceph-mon[134520]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 570 op/s 2023-10-17T01:25:47.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:46 smithi045 ceph-mon[98474]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 570 op/s 2023-10-17T01:25:49.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:48 smithi060 ceph-mon[134136]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 636 op/s 2023-10-17T01:25:49.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:48 smithi181 ceph-mon[134520]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 636 op/s 2023-10-17T01:25:49.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:48 smithi045 ceph-mon[98474]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 636 op/s 2023-10-17T01:25:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 640.0 s, 1658.2 tps, lat 60.135 ms stddev 70.593 2023-10-17T01:25:51.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:50 smithi181 ceph-mon[134520]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 730 op/s 2023-10-17T01:25:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:50 smithi045 ceph-mon[98474]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 730 op/s 2023-10-17T01:25:51.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:50 smithi060 ceph-mon[134136]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 730 op/s 2023-10-17T01:25:53.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:52 smithi060 ceph-mon[134136]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 660 op/s 2023-10-17T01:25:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:52 smithi181 ceph-mon[134520]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 660 op/s 2023-10-17T01:25:53.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:52 smithi045 ceph-mon[98474]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 660 op/s 2023-10-17T01:25:55.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:54 smithi181 ceph-mon[134520]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2023-10-17T01:25:55.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:54 smithi045 ceph-mon[98474]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2023-10-17T01:25:55.339 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:54 smithi060 ceph-mon[134136]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2023-10-17T01:25:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 645.0 s, 1357.7 tps, lat 71.951 ms stddev 94.957 2023-10-17T01:25:57.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:56 smithi060 ceph-mon[134136]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 625 op/s 2023-10-17T01:25:57.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:56 smithi181 ceph-mon[134520]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 625 op/s 2023-10-17T01:25:57.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:56 smithi045 ceph-mon[98474]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 625 op/s 2023-10-17T01:25:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.199+0000 7f8ec57ff700 1 -- 172.21.15.45:0/1362095439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec0104100 msgr2=0x7f8ec01064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.199+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/1362095439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec0104100 0x7f8ec01064f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000b0a0 tx=0x7f8eb000b3b0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 -- 172.21.15.45:0/1362095439 shutdown_connections 2023-10-17T01:25:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/1362095439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0106a30 0x7f8ec0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/1362095439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec01017d0 0x7f8ec0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/1362095439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec0104100 0x7f8ec01064f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 -- 172.21.15.45:0/1362095439 >> 172.21.15.45:0/1362095439 conn(0x7f8ec00fb1a0 msgr2=0x7f8ec00fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 -- 172.21.15.45:0/1362095439 shutdown_connections 2023-10-17T01:25:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.200+0000 7f8ec57ff700 1 -- 172.21.15.45:0/1362095439 wait complete. 2023-10-17T01:25:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.201+0000 7f8ec57ff700 1 Processor -- start 2023-10-17T01:25:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 -- start start 2023-10-17T01:25:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec01017d0 0x7f8ec0198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 0x7f8ec0199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec0106a30 0x7f8ec019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec019a850 con 0x7f8ec0106a30 2023-10-17T01:25:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec019a9c0 con 0x7f8ec01017d0 2023-10-17T01:25:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.202+0000 7f8ec57ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec019ab80 con 0x7f8ec0104100 2023-10-17T01:25:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 0x7f8ec0199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec01017d0 0x7f8ec0198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 0x7f8ec0199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54500/0 (socket says 172.21.15.45:54500) 2023-10-17T01:25:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/3865628379 learned_addr learned my addr 172.21.15.45:0/3865628379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec0106a30 0x7f8ec019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/3865628379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec01017d0 msgr2=0x7f8ec0198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec01017d0 0x7f8ec0198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec0106a30 msgr2=0x7f8ec019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec0106a30 0x7f8ec019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/3865628379 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0009d00 con 0x7f8ec0104100 2023-10-17T01:25:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec01017d0 0x7f8ec0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:25:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.203+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec0106a30 0x7f8ec019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:25:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.204+0000 7f8ebe7fc700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 0x7f8ec0199410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0007ab0 tx=0x7f8eb0004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.204+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3865628379 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000e040 con 0x7f8ec0104100 2023-10-17T01:25:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.204+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3865628379 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb0005710 con 0x7f8ec0104100 2023-10-17T01:25:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.204+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec019ae00 con 0x7f8ec0104100 2023-10-17T01:25:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.204+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3865628379 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb001dbc0 con 0x7f8ec0104100 2023-10-17T01:25:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.204+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec01a1580 con 0x7f8ec0104100 2023-10-17T01:25:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.211+0000 7f8ed91d3b80 1 Processor -- start 2023-10-17T01:25:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.211+0000 7f8ed91d3b80 1 -- start start 2023-10-17T01:25:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.211+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cbadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.211+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 0x560903cb9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.211+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cb9b60 0x560903cb9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.212+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903c149b0 con 0x560903cbb330 2023-10-17T01:25:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.212+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903c12370 con 0x560903cb9b60 2023-10-17T01:25:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.212+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903c0dc60 con 0x560903cba9b0 2023-10-17T01:25:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 0x560903cb9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 0x560903cb9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33042/0 (socket says 172.21.15.45:33042) 2023-10-17T01:25:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 -- 172.21.15.45:0/1719960536 learned_addr learned my addr 172.21.15.45:0/1719960536 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 -- 172.21.15.45:0/1719960536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 msgr2=0x560903cbadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cbadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 -- 172.21.15.45:0/1719960536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cb9b60 msgr2=0x560903cb9fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:25:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.213+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cb9b60 0x560903cb9fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.214+0000 7f8ebeffd700 1 -- 172.21.15.45:0/1719960536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560903a3bd80 con 0x560903cbb330 2023-10-17T01:25:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.214+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 0x560903cb9580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8007fd0 tx=0x7f8eb8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f57ff843758db9 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.216+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3865628379 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f8eb0005880 con 0x7f8ec0104100 2023-10-17T01:25:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.218+0000 7f8eb57fa700 1 -- 172.21.15.45:0/1719960536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8007c50 con 0x560903cbb330 2023-10-17T01:25:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.218+0000 7f8eb57fa700 1 -- 172.21.15.45:0/1719960536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb8015070 con 0x560903cbb330 2023-10-17T01:25:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.218+0000 7f8eb57fa700 1 -- 172.21.15.45:0/1719960536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8019aa0 con 0x560903cbb330 2023-10-17T01:25:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.219+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8eac0a0c20 0x7f8eac0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.219+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3865628379 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8eb00cda70 con 0x7f8ec0104100 2023-10-17T01:25:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.219+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1719960536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 msgr2=0x560903cb9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.219+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 0x560903cb9580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8007fd0 tx=0x7f8eb8010830 comp rx=0 tx=0).stop 2023-10-17T01:25:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.220+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1719960536 shutdown_connections 2023-10-17T01:25:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.220+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cb9b60 0x560903cb9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.220+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cbadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.220+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1719960536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cbb330 0x560903cb9580 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.220+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1719960536 >> 172.21.15.45:0/1719960536 conn(0x560903c1c070 msgr2=0x560903cb72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.220+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8eac0a0c20 0x7f8eac0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.221+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8eac0a0c20 0x7f8eac0a30e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x560903cb9510 tx=0x7f8ea8005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.222+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1719960536 shutdown_connections 2023-10-17T01:25:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.222+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1719960536 wait complete. 2023-10-17T01:25:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.222+0000 7f8ed91d3b80 1 -- start start 2023-10-17T01:25:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.223+0000 7f8ed91d3b80 1 Processor -- start 2023-10-17T01:25:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.223+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cd7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.224+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cd7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.224+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cde970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.224+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903baa4b0 con 0x560903cb9b60 2023-10-17T01:25:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.224+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903baa150 con 0x560903cbb330 2023-10-17T01:25:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.224+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903a918c0 con 0x560903cba9b0 2023-10-17T01:25:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.224+0000 7f8ed91d3b80 10 client.0 _ll_register_callbacks cb 0x560903af7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.225+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cde970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.225+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cde970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:37160/0 (socket says 172.21.15.45:37160) 2023-10-17T01:25:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.225+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/3030032486 learned_addr learned my addr 172.21.15.45:0/3030032486 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.226+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/3030032486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 msgr2=0x560903cd7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.226+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cd7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.226+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/3030032486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 msgr2=0x560903cd7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.226+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cd7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.226+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/3030032486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb8007db0 con 0x560903cbb330 2023-10-17T01:25:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.227+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cde970 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8010700 tx=0x7f8ea80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2626da1ce910f88 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.227+0000 7f8ea67fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:25:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.231+0000 7f8ea67fc700 1 -- 172.21.15.45:0/3030032486 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801d860 con 0x560903cbb330 2023-10-17T01:25:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.231+0000 7f8ea67fc700 1 -- 172.21.15.45:0/3030032486 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8ea801d9c0 con 0x560903cbb330 2023-10-17T01:25:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.231+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560903c38cb0 con 0x560903cbb330 2023-10-17T01:25:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.232+0000 7f8ea67fc700 1 -- 172.21.15.45:0/3030032486 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea801db80 con 0x560903cbb330 2023-10-17T01:25:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.232+0000 7f8ed91d3b80 10 client.26642 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.232+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560903a24c00 con 0x560903cbb330 2023-10-17T01:25:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.238+0000 7f8ea67fc700 1 -- 172.21.15.45:0/3030032486 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560903c38cb0 con 0x560903cbb330 2023-10-17T01:25:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.238+0000 7f8ea67fc700 1 -- 172.21.15.45:0/3030032486 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8ea800f3c0 con 0x560903cbb330 2023-10-17T01:25:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.240+0000 7f8ed91d3b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.240+0000 7f8ed91d3b80 10 client.26642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.240+0000 7f8ed91d3b80 20 client.26642 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.240+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903ce3500 0x560903ce5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.240+0000 7f8ed91d3b80 4 client.26642 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:25:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.240+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560903c8bcd0 con 0x560903ce3500 2023-10-17T01:25:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.245+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903ce3500 0x560903ce5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.245+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903ce3500 0x560903ce5920 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174b8b6fe430f7ec server_cookie=13c317138b33fc0f in_seq=0 out_seq=0 2023-10-17T01:25:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.246+0000 7f8ea67fc700 10 client.26642 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.249+0000 7f8ea67fc700 1 -- 172.21.15.45:0/3030032486 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x560903c8bcd0 con 0x560903ce3500 2023-10-17T01:25:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.249+0000 7f8ea67fc700 10 client.26642 handle_command_reply: tid=0 2023-10-17T01:25:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.249+0000 7f8ed91d3b80 1 client.26642 shutdown 2023-10-17T01:25:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.250+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 msgr2=0x560903cde970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.250+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cde970 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8010700 tx=0x7f8ea80047c0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 shutdown_connections 2023-10-17T01:25:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903ce3500 0x560903ce5920 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cde970 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cd7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/3030032486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cd7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 >> 172.21.15.45:0/3030032486 conn(0x560903c1c070 msgr2=0x560903cb6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ea67fc700 0 client.26642 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 shutdown_connections 2023-10-17T01:25:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.251+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/3030032486 wait complete. 2023-10-17T01:25:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.252+0000 7f8ed91d3b80 20 client.26642 trim_cache size 0 max 16384 2023-10-17T01:25:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.276+0000 7f8ed91d3b80 1 Processor -- start 2023-10-17T01:25:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.276+0000 7f8ed91d3b80 1 -- start start 2023-10-17T01:25:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.276+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cc1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cc1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cbdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903cb85c0 con 0x560903cb9b60 2023-10-17T01:25:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903cdef70 con 0x560903cbb330 2023-10-17T01:25:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903cd68a0 con 0x560903cba9b0 2023-10-17T01:25:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cc1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cbdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ebe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cc1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cbdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:37168/0 (socket says 172.21.15.45:37168) 2023-10-17T01:25:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.277+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/621469496 learned_addr learned my addr 172.21.15.45:0/621469496 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.278+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/621469496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 msgr2=0x560903cc1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.278+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cc1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.278+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/621469496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 msgr2=0x560903cc1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.278+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cc1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.278+0000 7f8ebf7fe700 1 -- 172.21.15.45:0/621469496 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560903a621f0 con 0x560903cbb330 2023-10-17T01:25:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.278+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cbdc60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800f750 tx=0x7f8ea8005ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c187fe128b110c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.279+0000 7f8eb5ffb700 1 -- 172.21.15.45:0/621469496 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801db80 con 0x560903cbb330 2023-10-17T01:25:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.279+0000 7f8eb5ffb700 1 -- 172.21.15.45:0/621469496 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ea800fc80 con 0x560903cbb330 2023-10-17T01:25:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.279+0000 7f8eb5ffb700 1 -- 172.21.15.45:0/621469496 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80399e0 con 0x560903cbb330 2023-10-17T01:25:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/621469496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 msgr2=0x560903cbdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cbdc60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800f750 tx=0x7f8ea8005ba0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/621469496 shutdown_connections 2023-10-17T01:25:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cbdc60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cc1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/621469496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cc1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/621469496 >> 172.21.15.45:0/621469496 conn(0x560903c1c070 msgr2=0x560903cd6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/621469496 shutdown_connections 2023-10-17T01:25:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.280+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/621469496 wait complete. 2023-10-17T01:25:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.281+0000 7f8ed91d3b80 1 -- start start 2023-10-17T01:25:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.281+0000 7f8ed91d3b80 1 Processor -- start 2023-10-17T01:25:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.281+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cdc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.281+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cdaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.281+0000 7f8ed91d3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cdafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903cdcca0 con 0x560903cb9b60 2023-10-17T01:25:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903cdb510 con 0x560903cbb330 2023-10-17T01:25:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ed91d3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560903cdb6d0 con 0x560903cba9b0 2023-10-17T01:25:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ed91d3b80 10 client.0 _ll_register_callbacks cb 0x7f8ea800ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:25:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cdaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cdaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54542/0 (socket says 172.21.15.45:54542) 2023-10-17T01:25:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/1057955560 learned_addr learned my addr 172.21.15.45:0/1057955560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:25:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 msgr2=0x560903cdafd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:25:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cdafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cdafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/1057955560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 msgr2=0x560903cdc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:25:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cdc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebe7fc700 1 -- 172.21.15.45:0/1057955560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560903a24c00 con 0x560903cba9b0 2023-10-17T01:25:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.282+0000 7f8ebf7fe700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cdafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:25:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.283+0000 7f8ebe7fc700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cdaa90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8eb009b0b0 tx=0x7f8eb009aa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190e24347a9db007 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:25:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.283+0000 7f8ea67fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:25:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.284+0000 7f8ea67fc700 1 -- 172.21.15.45:0/1057955560 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb009d070 con 0x560903cba9b0 2023-10-17T01:25:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.284+0000 7f8ea67fc700 1 -- 172.21.15.45:0/1057955560 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8eb001d480 con 0x560903cba9b0 2023-10-17T01:25:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.284+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eb8015070 con 0x560903cba9b0 2023-10-17T01:25:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.285+0000 7f8ea67fc700 1 -- 172.21.15.45:0/1057955560 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb009b820 con 0x560903cba9b0 2023-10-17T01:25:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.285+0000 7f8ed91d3b80 10 client.26593 fetch_fsmap learned FSMap version 40 2023-10-17T01:25:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.285+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560903a3a250 con 0x560903cba9b0 2023-10-17T01:25:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.285+0000 7f8ea67fc700 1 -- 172.21.15.45:0/1057955560 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb8015070 con 0x560903cba9b0 2023-10-17T01:25:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.286+0000 7f8ea67fc700 1 -- 172.21.15.45:0/1057955560 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8eb001dd20 con 0x560903cba9b0 2023-10-17T01:25:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.286+0000 7f8ed91d3b80 10 client.26593 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:25:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.286+0000 7f8ed91d3b80 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:25:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.286+0000 7f8ed91d3b80 20 client.26593 populate_metadata read hostname 'smithi045' 2023-10-17T01:25:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.286+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903d472d0 0x560903d496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:25:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.287+0000 7f8ed91d3b80 4 client.26593 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:25:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.287+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560903c06ce0 con 0x560903d472d0 2023-10-17T01:25:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.287+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903d472d0 0x560903d496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:25:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.289+0000 7f8ebeffd700 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903d472d0 0x560903d496f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4e3ec0eebe583b server_cookie=a21aacd8822b738d in_seq=0 out_seq=0 2023-10-17T01:25:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.289+0000 7f8ea67fc700 10 client.26593 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.311+0000 7f8ea67fc700 1 -- 172.21.15.45:0/1057955560 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560903c06ce0 con 0x560903d472d0 2023-10-17T01:25:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.311+0000 7f8ea67fc700 10 client.26593 handle_command_reply: tid=0 2023-10-17T01:25:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.312+0000 7f8ed91d3b80 1 client.26593 shutdown 2023-10-17T01:25:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.312+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 msgr2=0x560903cdaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.312+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cdaa90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8eb009b0b0 tx=0x7f8eb009aa80 comp rx=0 tx=0).stop 2023-10-17T01:25:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 shutdown_connections 2023-10-17T01:25:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560903d472d0 0x560903d496f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560903cbb330 0x560903cdafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560903cb9b60 0x560903cdc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ed91d3b80 1 --2- 172.21.15.45:0/1057955560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560903cba9b0 0x560903cdaa90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 >> 172.21.15.45:0/1057955560 conn(0x560903c1c070 msgr2=0x560903cc1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.313+0000 7f8ea67fc700 0 client.26593 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:25:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.314+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 shutdown_connections 2023-10-17T01:25:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.314+0000 7f8ed91d3b80 1 -- 172.21.15.45:0/1057955560 wait complete. 2023-10-17T01:25:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.314+0000 7f8ed91d3b80 20 client.26593 trim_cache size 0 max 16384 2023-10-17T01:25:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:25:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:25:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:25:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:25:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8eac0a0c20 msgr2=0x7f8eac0a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8eac0a0c20 0x7f8eac0a30e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x560903cb9510 tx=0x7f8ea8005f30 comp rx=0 tx=0).stop 2023-10-17T01:25:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 msgr2=0x7f8ec0199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:25:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 0x7f8ec0199410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0007ab0 tx=0x7f8eb0004d00 comp rx=0 tx=0).stop 2023-10-17T01:25:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 shutdown_connections 2023-10-17T01:25:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f8eac0a0c20 0x7f8eac0a30e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec0106a30 0x7f8ec019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ec01017d0 0x7f8ec0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 --2- 172.21.15.45:0/3865628379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec0104100 0x7f8ec0199410 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:25:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 >> 172.21.15.45:0/3865628379 conn(0x7f8ec00fb1a0 msgr2=0x7f8ec00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:25:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 shutdown_connections 2023-10-17T01:25:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:25:58.315+0000 7f8ec57ff700 1 -- 172.21.15.45:0/3865628379 wait complete. 2023-10-17T01:25:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:25:58 smithi181 ceph-mon[134520]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 681 op/s 2023-10-17T01:25:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:25:58 smithi045 ceph-mon[98474]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 681 op/s 2023-10-17T01:25:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:25:58 smithi060 ceph-mon[134136]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 681 op/s 2023-10-17T01:26:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 650.0 s, 1037.0 tps, lat 98.745 ms stddev 122.476 2023-10-17T01:26:01.078 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:00 smithi060 ceph-mon[134136]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 841 op/s 2023-10-17T01:26:01.160 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:00 smithi181 ceph-mon[134520]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 841 op/s 2023-10-17T01:26:01.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:00 smithi045 ceph-mon[98474]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 841 op/s 2023-10-17T01:26:03.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:02 smithi181 ceph-mon[134520]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 766 op/s 2023-10-17T01:26:03.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:02 smithi045 ceph-mon[98474]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 766 op/s 2023-10-17T01:26:03.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:02 smithi060 ceph-mon[134136]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 766 op/s 2023-10-17T01:26:05.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:04 smithi181 ceph-mon[134520]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 891 op/s 2023-10-17T01:26:05.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:04 smithi045 ceph-mon[98474]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 891 op/s 2023-10-17T01:26:05.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:04 smithi060 ceph-mon[134136]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 891 op/s 2023-10-17T01:26:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 655.0 s, 1175.0 tps, lat 83.961 ms stddev 100.574 2023-10-17T01:26:06.713 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:26:07.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 -- 172.21.15.45:0/2297378322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8101f50 msgr2=0x7f48d81023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 --2- 172.21.15.45:0/2297378322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8101f50 0x7f48d81023b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0099c0 tx=0x7f48cc009cd0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 -- 172.21.15.45:0/2297378322 shutdown_connections 2023-10-17T01:26:07.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 --2- 172.21.15.45:0/2297378322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d81028f0 0x7f48d81091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 --2- 172.21.15.45:0/2297378322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8100d60 0x7f48d8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 --2- 172.21.15.45:0/2297378322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8101f50 0x7f48d81023b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 -- 172.21.15.45:0/2297378322 >> 172.21.15.45:0/2297378322 conn(0x7f48d80fc2d0 msgr2=0x7f48d80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 -- 172.21.15.45:0/2297378322 shutdown_connections 2023-10-17T01:26:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.018+0000 7f48de3ff700 1 -- 172.21.15.45:0/2297378322 wait complete. 2023-10-17T01:26:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 Processor -- start 2023-10-17T01:26:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 -- start start 2023-10-17T01:26:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8100d60 0x7f48d8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 0x7f48d81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d81028f0 0x7f48d81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d81a07b0 con 0x7f48d8100d60 2023-10-17T01:26:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d81a0920 con 0x7f48d8101f50 2023-10-17T01:26:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.020+0000 7f48de3ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d81a0ae0 con 0x7f48d81028f0 2023-10-17T01:26:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 0x7f48d81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 0x7f48d81991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:37196/0 (socket says 172.21.15.45:37196) 2023-10-17T01:26:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 -- 172.21.15.45:0/1331912913 learned_addr learned my addr 172.21.15.45:0/1331912913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48dc99c700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d81028f0 0x7f48d81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d7fff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8100d60 0x7f48d8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 -- 172.21.15.45:0/1331912913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d81028f0 msgr2=0x7f48d81a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d81028f0 0x7f48d81a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 -- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8100d60 msgr2=0x7f48d8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8100d60 0x7f48d8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d77fe700 1 -- 172.21.15.45:0/1331912913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0095f0 con 0x7f48d8101f50 2023-10-17T01:26:07.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.021+0000 7f48d7fff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8100d60 0x7f48d8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:26:07.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.022+0000 7f48d77fe700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 0x7f48d81991b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009fd0 tx=0x7f48cc00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:07.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.022+0000 7f48d57fa700 1 -- 172.21.15.45:0/1331912913 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc01d070 con 0x7f48d8101f50 2023-10-17T01:26:07.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.022+0000 7f48d57fa700 1 -- 172.21.15.45:0/1331912913 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48cc0054f0 con 0x7f48d8101f50 2023-10-17T01:26:07.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.022+0000 7f48d57fa700 1 -- 172.21.15.45:0/1331912913 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc004bb0 con 0x7f48d8101f50 2023-10-17T01:26:07.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.022+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d81a0d60 con 0x7f48d8101f50 2023-10-17T01:26:07.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.022+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d81a1180 con 0x7f48d8101f50 2023-10-17T01:26:07.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.026+0000 7f48f1d7cb80 1 Processor -- start 2023-10-17T01:26:07.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.027+0000 7f48d57fa700 1 -- 172.21.15.45:0/1331912913 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f48cc004d10 con 0x7f48d8101f50 2023-10-17T01:26:07.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.027+0000 7f48f1d7cb80 1 -- start start 2023-10-17T01:26:07.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.028+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc23b0 0x55e966cc27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.028+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc2d10 0x55e966cc3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.028+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966cc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.028+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966c11a10 con 0x55e966cc23b0 2023-10-17T01:26:07.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.028+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966c0f3d0 con 0x55e966cc3ac0 2023-10-17T01:26:07.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.028+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966c0acc0 con 0x55e966cc2d10 2023-10-17T01:26:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.029+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc2d10 0x55e966cc3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.029+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc2d10 0x55e966cc3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54564/0 (socket says 172.21.15.45:54564) 2023-10-17T01:26:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.029+0000 7f48d7fff700 1 -- 172.21.15.45:0/3220406945 learned_addr learned my addr 172.21.15.45:0/3220406945 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.030+0000 7f48d57fa700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48c00a0c70 0x7f48c00a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.030+0000 7f48d57fa700 1 -- 172.21.15.45:0/1331912913 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f48cc0cd1a0 con 0x7f48d8101f50 2023-10-17T01:26:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.030+0000 7f48dc99c700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48c00a0c70 0x7f48c00a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.032+0000 7f48dc99c700 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc23b0 0x55e966cc27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.032+0000 7f48dc99c700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48c00a0c70 0x7f48c00a3130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009ec0 tx=0x7f48d0009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.034+0000 7f48dc99c700 1 -- 172.21.15.45:0/3220406945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc2d10 msgr2=0x55e966cc3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.034+0000 7f48dc99c700 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc2d10 0x55e966cc3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.034+0000 7f48dc99c700 1 -- 172.21.15.45:0/3220406945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 msgr2=0x55e966cc3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.034+0000 7f48dc99c700 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966cc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.034+0000 7f48dc99c700 1 -- 172.21.15.45:0/3220406945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e966a38de0 con 0x55e966cc23b0 2023-10-17T01:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.036+0000 7f48dc99c700 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc23b0 0x55e966cc27d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009fd0 tx=0x7f48d000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44a487d8093e27d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.040+0000 7f48be7fc700 1 -- 172.21.15.45:0/3220406945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000e070 con 0x55e966cc23b0 2023-10-17T01:26:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.040+0000 7f48be7fc700 1 -- 172.21.15.45:0/3220406945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d000f040 con 0x55e966cc23b0 2023-10-17T01:26:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.040+0000 7f48be7fc700 1 -- 172.21.15.45:0/3220406945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0016750 con 0x55e966cc23b0 2023-10-17T01:26:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.041+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3220406945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc23b0 msgr2=0x55e966cc27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.041+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc23b0 0x55e966cc27d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009fd0 tx=0x7f48d000e000 comp rx=0 tx=0).stop 2023-10-17T01:26:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.042+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3220406945 shutdown_connections 2023-10-17T01:26:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.042+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966cc3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.042+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc23b0 0x55e966cc27d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.042+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3220406945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc2d10 0x55e966cc3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.042+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3220406945 >> 172.21.15.45:0/3220406945 conn(0x55e966c190d0 msgr2=0x55e966cb4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.046+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3220406945 shutdown_connections 2023-10-17T01:26:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.046+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3220406945 wait complete. 2023-10-17T01:26:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.047+0000 7f48f1d7cb80 1 -- start start 2023-10-17T01:26:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.047+0000 7f48f1d7cb80 1 Processor -- start 2023-10-17T01:26:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.048+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc23b0 0x55e966cdd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.048+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966cdd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.048+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc3ac0 0x55e966ce47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.049+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966ba7510 con 0x55e966cc2d10 2023-10-17T01:26:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.049+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966ba71b0 con 0x55e966cc23b0 2023-10-17T01:26:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.049+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966a8e920 con 0x55e966cc3ac0 2023-10-17T01:26:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.049+0000 7f48f1d7cb80 10 client.0 _ll_register_callbacks cb 0x55e966af4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.050+0000 7f48d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966cdd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.051+0000 7f48d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966cdd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33104/0 (socket says 172.21.15.45:33104) 2023-10-17T01:26:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.051+0000 7f48d77fe700 1 -- 172.21.15.45:0/80330360 learned_addr learned my addr 172.21.15.45:0/80330360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.052+0000 7f48d77fe700 1 -- 172.21.15.45:0/80330360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc3ac0 msgr2=0x55e966ce47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.052+0000 7f48d77fe700 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc3ac0 0x55e966ce47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.052+0000 7f48d77fe700 1 -- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc23b0 msgr2=0x55e966cdd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.052+0000 7f48d77fe700 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc23b0 0x55e966cdd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.052+0000 7f48d77fe700 1 -- 172.21.15.45:0/80330360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d0008da0 con 0x55e966cc2d10 2023-10-17T01:26:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.054+0000 7f48d77fe700 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966cdd760 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0a2300 tx=0x7f48cc09a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0370de59c4431f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.054+0000 7f48b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:26:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.056+0000 7f48b77fe700 1 -- 172.21.15.45:0/80330360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc0cc030 con 0x55e966cc2d10 2023-10-17T01:26:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.056+0000 7f48b77fe700 1 -- 172.21.15.45:0/80330360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f48cc020570 con 0x55e966cc2d10 2023-10-17T01:26:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.056+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e966c35d10 con 0x55e966cc2d10 2023-10-17T01:26:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.057+0000 7f48b77fe700 1 -- 172.21.15.45:0/80330360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48cc09af50 con 0x55e966cc2d10 2023-10-17T01:26:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.057+0000 7f48f1d7cb80 10 client.16341 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.057+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e966a21c60 con 0x55e966cc2d10 2023-10-17T01:26:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48b77fe700 1 -- 172.21.15.45:0/80330360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48cc07d080 con 0x55e966cc2d10 2023-10-17T01:26:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48b77fe700 1 -- 172.21.15.45:0/80330360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f48cc099d40 con 0x55e966cc2d10 2023-10-17T01:26:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48f1d7cb80 10 client.16341 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48f1d7cb80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48f1d7cb80 20 client.16341 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966ce8c30 0x55e966ceb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48f1d7cb80 4 client.16341 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:26:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.059+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e966c88d30 con 0x55e966ce8c30 2023-10-17T01:26:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.061+0000 7f48d7fff700 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966ce8c30 0x55e966ceb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.064+0000 7f48b77fe700 10 client.16341 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.064+0000 7f48d7fff700 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966ce8c30 0x55e966ceb020 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38257c28e95f9bbe server_cookie=10dccb97dc01f4e3 in_seq=0 out_seq=0 2023-10-17T01:26:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.068+0000 7f48b77fe700 1 -- 172.21.15.45:0/80330360 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55e966c88d30 con 0x55e966ce8c30 2023-10-17T01:26:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.068+0000 7f48b77fe700 10 client.16341 handle_command_reply: tid=0 2023-10-17T01:26:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.068+0000 7f48f1d7cb80 1 client.16341 shutdown 2023-10-17T01:26:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 msgr2=0x55e966cdd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966cdd760 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0a2300 tx=0x7f48cc09a260 comp rx=0 tx=0).stop 2023-10-17T01:26:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 shutdown_connections 2023-10-17T01:26:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc3ac0 0x55e966ce47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966ce8c30 0x55e966ceb020 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc23b0 0x55e966cdd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/80330360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966cdd760 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 >> 172.21.15.45:0/80330360 conn(0x55e966c190d0 msgr2=0x55e966cb4020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.069+0000 7f48b77fe700 0 client.16341 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.070+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 shutdown_connections 2023-10-17T01:26:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.070+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/80330360 wait complete. 2023-10-17T01:26:07.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.071+0000 7f48f1d7cb80 20 client.16341 trim_cache size 0 max 16384 2023-10-17T01:26:07.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 Processor -- start 2023-10-17T01:26:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 -- start start 2023-10-17T01:26:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966cdc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc2d10 0x55e966cb5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc3ac0 0x55e966cb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966cc9d30 con 0x55e966cc3ac0 2023-10-17T01:26:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966cb6340 con 0x55e966cc2d10 2023-10-17T01:26:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966cb5b90 con 0x55e966cc23b0 2023-10-17T01:26:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966cdc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966cdc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54574/0 (socket says 172.21.15.45:54574) 2023-10-17T01:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48d7fff700 1 -- 172.21.15.45:0/1657463103 learned_addr learned my addr 172.21.15.45:0/1657463103 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.098+0000 7f48d77fe700 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc2d10 0x55e966cb5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48dc99c700 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc3ac0 0x55e966cb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48d7fff700 1 -- 172.21.15.45:0/1657463103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc2d10 msgr2=0x55e966cb5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48d7fff700 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc2d10 0x55e966cb5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48d7fff700 1 -- 172.21.15.45:0/1657463103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc3ac0 msgr2=0x55e966cb4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48d7fff700 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc3ac0 0x55e966cb4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48d7fff700 1 -- 172.21.15.45:0/1657463103 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e966a5f250 con 0x55e966cc23b0 2023-10-17T01:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48d7fff700 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966cdc8d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55e966caf320 tx=0x7f48c801da10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564bf9afd2bc9bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48beffd700 1 -- 172.21.15.45:0/1657463103 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8017040 con 0x55e966cc23b0 2023-10-17T01:26:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48beffd700 1 -- 172.21.15.45:0/1657463103 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48c801de40 con 0x55e966cc23b0 2023-10-17T01:26:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.099+0000 7f48beffd700 1 -- 172.21.15.45:0/1657463103 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8011970 con 0x55e966cc23b0 2023-10-17T01:26:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.100+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/1657463103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 msgr2=0x55e966cdc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.100+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966cdc8d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55e966caf320 tx=0x7f48c801da10 comp rx=0 tx=0).stop 2023-10-17T01:26:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/1657463103 shutdown_connections 2023-10-17T01:26:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc3ac0 0x55e966cb4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966cdc8d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/1657463103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc2d10 0x55e966cb5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/1657463103 >> 172.21.15.45:0/1657463103 conn(0x55e966c190d0 msgr2=0x55e966cb48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/1657463103 shutdown_connections 2023-10-17T01:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/1657463103 wait complete. 2023-10-17T01:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.101+0000 7f48f1d7cb80 1 -- start start 2023-10-17T01:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.102+0000 7f48f1d7cb80 1 Processor -- start 2023-10-17T01:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.102+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966d327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966d32d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48f1d7cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966d39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966d39e70 con 0x55e966cc2d10 2023-10-17T01:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966d39fe0 con 0x55e966cc3ac0 2023-10-17T01:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48f1d7cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e966d3a1a0 con 0x55e966cc23b0 2023-10-17T01:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48f1d7cb80 10 client.0 _ll_register_callbacks cb 0x55e966af4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966d327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966d327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:54580/0 (socket says 172.21.15.45:54580) 2023-10-17T01:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 -- 172.21.15.45:0/3657178720 learned_addr learned my addr 172.21.15.45:0/3657178720 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48dc99c700 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966d39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 -- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 msgr2=0x55e966d39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966d39930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 -- 172.21.15.45:0/3657178720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 msgr2=0x55e966d32d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966d32d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.103+0000 7f48d7fff700 1 -- 172.21.15.45:0/3657178720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c8009b40 con 0x55e966cc23b0 2023-10-17T01:26:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48d7fff700 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966d327c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f48c80046b0 tx=0x7f48c80116f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=830de4ba7e11df62 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:26:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48b77fe700 1 -- 172.21.15.45:0/3657178720 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8017070 con 0x55e966cc23b0 2023-10-17T01:26:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48b77fe700 1 -- 172.21.15.45:0/3657178720 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f48c8011bd0 con 0x55e966cc23b0 2023-10-17T01:26:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d000f040 con 0x55e966cc23b0 2023-10-17T01:26:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48b77fe700 1 -- 172.21.15.45:0/3657178720 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48c8011db0 con 0x55e966cc23b0 2023-10-17T01:26:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48f1d7cb80 10 client.26608 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e966a372b0 con 0x55e966cc23b0 2023-10-17T01:26:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.104+0000 7f48b77fe700 1 -- 172.21.15.45:0/3657178720 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d000f040 con 0x55e966cc23b0 2023-10-17T01:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48b77fe700 1 -- 172.21.15.45:0/3657178720 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f48c8005030 con 0x55e966cc23b0 2023-10-17T01:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48f1d7cb80 10 client.26608 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48f1d7cb80 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48f1d7cb80 20 client.26608 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966d3b9f0 0x55e966d3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48f1d7cb80 4 client.26608 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.105+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e966c03d40 con 0x55e966d3b9f0 2023-10-17T01:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.106+0000 7f48d77fe700 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966d3b9f0 0x55e966d3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.106+0000 7f48d77fe700 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966d3b9f0 0x55e966d3de10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327fe7818c70e708 server_cookie=aab455621fd4909e in_seq=0 out_seq=0 2023-10-17T01:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.106+0000 7f48b77fe700 10 client.26608 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48b77fe700 1 -- 172.21.15.45:0/3657178720 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e966c03d40 con 0x55e966d3b9f0 2023-10-17T01:26:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48b77fe700 10 client.26608 handle_command_reply: tid=0 2023-10-17T01:26:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 client.26608 shutdown 2023-10-17T01:26:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 msgr2=0x55e966d327c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966d327c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f48c80046b0 tx=0x7f48c80116f0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 shutdown_connections 2023-10-17T01:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55e966d3b9f0 0x55e966d3de10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e966cc3ac0 0x55e966d39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e966cc23b0 0x55e966d327c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 --2- 172.21.15.45:0/3657178720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e966cc2d10 0x55e966d32d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 >> 172.21.15.45:0/3657178720 conn(0x55e966c190d0 msgr2=0x55e966ce4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.107+0000 7f48b77fe700 0 client.26608 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.108+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 shutdown_connections 2023-10-17T01:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.108+0000 7f48f1d7cb80 1 -- 172.21.15.45:0/3657178720 wait complete. 2023-10-17T01:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.108+0000 7f48f1d7cb80 20 client.26608 trim_cache size 0 max 16384 2023-10-17T01:26:07.121 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:26:07.121 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:26:07.121 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:26:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:26:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:26:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:26:07.122 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:26:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:26:07.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:26:07.123 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.110+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48c00a0c70 msgr2=0x7f48c00a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.110+0000 7f48de3ff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48c00a0c70 0x7f48c00a3130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009ec0 tx=0x7f48d0009920 comp rx=0 tx=0).stop 2023-10-17T01:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.110+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 msgr2=0x7f48d81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.110+0000 7f48de3ff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 0x7f48d81991b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009fd0 tx=0x7f48cc00f670 comp rx=0 tx=0).stop 2023-10-17T01:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 shutdown_connections 2023-10-17T01:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f48c00a0c70 0x7f48c00a3130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d81028f0 0x7f48d81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48d8100d60 0x7f48d8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 --2- 172.21.15.45:0/1331912913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48d8101f50 0x7f48d81991b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 >> 172.21.15.45:0/1331912913 conn(0x7f48d80fc2d0 msgr2=0x7f48d8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:07.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 shutdown_connections 2023-10-17T01:26:07.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:07.111+0000 7f48de3ff700 1 -- 172.21.15.45:0/1331912913 wait complete. 2023-10-17T01:26:07.133 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:06 smithi045 ceph-mon[98474]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-17T01:26:07.136 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:26:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:06 smithi181 ceph-mon[134520]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-17T01:26:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:06 smithi060 ceph-mon[134136]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-17T01:26:09.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:08 smithi181 ceph-mon[134520]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-17T01:26:09.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:08 smithi045 ceph-mon[98474]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-17T01:26:09.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:08 smithi060 ceph-mon[134136]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-17T01:26:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 660.0 s, 1282.6 tps, lat 78.617 ms stddev 90.373 2023-10-17T01:26:11.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:10 smithi181 ceph-mon[134520]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 872 op/s 2023-10-17T01:26:11.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:10 smithi045 ceph-mon[98474]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 872 op/s 2023-10-17T01:26:11.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:10 smithi060 ceph-mon[134136]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 872 op/s 2023-10-17T01:26:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/1994988052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 msgr2=0x7f0bf410f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/1994988052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 0x7f0bf410f740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009990 tx=0x7f0bec009ca0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/1994988052 shutdown_connections 2023-10-17T01:26:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/1994988052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 0x7f0bf410f740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/1994988052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 0x7f0bf40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/1994988052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf4073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/1994988052 >> 172.21.15.45:0/1994988052 conn(0x7f0bf40fc2d0 msgr2=0x7f0bf40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/1994988052 shutdown_connections 2023-10-17T01:26:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.532+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/1994988052 wait complete. 2023-10-17T01:26:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.533+0000 7f0bfb7ff700 1 Processor -- start 2023-10-17T01:26:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.533+0000 7f0bfb7ff700 1 -- start start 2023-10-17T01:26:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bfb7ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bfb7ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 0x7f0bf419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bfb7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 0x7f0bf419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bfb7ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf419ea00 con 0x7f0bf4074d90 2023-10-17T01:26:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bfb7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf419eb70 con 0x7f0bf410d350 2023-10-17T01:26:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bfb7ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf419ed30 con 0x7f0bf40737b0 2023-10-17T01:26:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf9d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 0x7f0bf419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:39792/0 (socket says 172.21.15.45:39792) 2023-10-17T01:26:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 -- 172.21.15.45:0/3306879446 learned_addr learned my addr 172.21.15.45:0/3306879446 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 0x7f0bf419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 -- 172.21.15.45:0/3306879446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 msgr2=0x7f0bf419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 0x7f0bf419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 -- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 msgr2=0x7f0bf419d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 0x7f0bf419d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.534+0000 7f0bf959b700 1 -- 172.21.15.45:0/3306879446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0095f0 con 0x7f0bf40737b0 2023-10-17T01:26:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.535+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 0x7f0bf419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:26:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.535+0000 7f0bf959b700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf419cfe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0be8007e20 tx=0x7f0be800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.535+0000 7f0bf27fc700 1 -- 172.21.15.45:0/3306879446 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be800ad40 con 0x7f0bf40737b0 2023-10-17T01:26:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.535+0000 7f0bf27fc700 1 -- 172.21.15.45:0/3306879446 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be80128e0 con 0x7f0bf40737b0 2023-10-17T01:26:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.535+0000 7f0bf27fc700 1 -- 172.21.15.45:0/3306879446 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be80117e0 con 0x7f0bf40737b0 2023-10-17T01:26:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.535+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf419f010 con 0x7f0bf40737b0 2023-10-17T01:26:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.536+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf41a57d0 con 0x7f0bf40737b0 2023-10-17T01:26:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.540+0000 7f0bf27fc700 1 -- 172.21.15.45:0/3306879446 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0be8012a50 con 0x7f0bf40737b0 2023-10-17T01:26:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.542+0000 7f0c0f0c0b80 1 Processor -- start 2023-10-17T01:26:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.542+0000 7f0bf27fc700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0be40a0c70 0x7f0be40a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.542+0000 7f0c0f0c0b80 1 -- start start 2023-10-17T01:26:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0bf27fc700 1 -- 172.21.15.45:0/3306879446 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0be80cc150 con 0x7f0bf40737b0 2023-10-17T01:26:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcb5bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0bf9d9c700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0be40a0c70 0x7f0be40a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5c190 0x555bfcb5c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5cd90 0x555bfcb5d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcab39b0 con 0x555bfcb5b830 2023-10-17T01:26:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcab1370 con 0x555bfcb5c190 2023-10-17T01:26:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.543+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcaacc60 con 0x555bfcb5cd90 2023-10-17T01:26:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.544+0000 7f0bf959b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcb5bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.544+0000 7f0bf959b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcb5bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55298/0 (socket says 172.21.15.45:55298) 2023-10-17T01:26:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.544+0000 7f0bf959b700 1 -- 172.21.15.45:0/2489622637 learned_addr learned my addr 172.21.15.45:0/2489622637 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.544+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5c190 0x555bfcb5c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.545+0000 7f0bf959b700 1 -- 172.21.15.45:0/2489622637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5cd90 msgr2=0x555bfcb5d210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:26:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.545+0000 7f0bf959b700 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5cd90 0x555bfcb5d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.545+0000 7f0bf959b700 1 -- 172.21.15.45:0/2489622637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5c190 msgr2=0x555bfcb5c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.545+0000 7f0bf959b700 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5c190 0x555bfcb5c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.545+0000 7f0bf959b700 1 -- 172.21.15.45:0/2489622637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bfc8dad80 con 0x555bfcb5b830 2023-10-17T01:26:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.546+0000 7f0bf959b700 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcb5bc50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0be8099700 tx=0x7f0be8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d471fbcfbcd9560 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.547+0000 7f0bdf7fe700 1 -- 172.21.15.45:0/2489622637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be809d5a0 con 0x555bfcb5b830 2023-10-17T01:26:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.547+0000 7f0bf9d9c700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0be40a0c70 0x7f0be40a3130 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x555bfcb5d750 tx=0x7f0bec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.547+0000 7f0bdf7fe700 1 -- 172.21.15.45:0/2489622637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be80a17e0 con 0x555bfcb5b830 2023-10-17T01:26:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.547+0000 7f0bdf7fe700 1 -- 172.21.15.45:0/2489622637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8098c60 con 0x555bfcb5b830 2023-10-17T01:26:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.547+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2489622637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 msgr2=0x555bfcb5bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.547+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcb5bc50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0be8099700 tx=0x7f0be8007f70 comp rx=0 tx=0).stop 2023-10-17T01:26:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2489622637 shutdown_connections 2023-10-17T01:26:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5cd90 0x555bfcb5d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcb5bc50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2489622637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5c190 0x555bfcb5c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2489622637 >> 172.21.15.45:0/2489622637 conn(0x555bfcabb070 msgr2=0x555bfcb562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2489622637 shutdown_connections 2023-10-17T01:26:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2489622637 wait complete. 2023-10-17T01:26:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.548+0000 7f0c0f0c0b80 1 -- start start 2023-10-17T01:26:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 Processor -- start 2023-10-17T01:26:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5b830 0x555bfcb65390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcb658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5cd90 0x555bfcb6c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfca494b0 con 0x555bfcb5cd90 2023-10-17T01:26:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfca49150 con 0x555bfcb5b830 2023-10-17T01:26:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc9308c0 con 0x555bfcb5c190 2023-10-17T01:26:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.549+0000 7f0c0f0c0b80 10 client.0 _ll_register_callbacks cb 0x555bfc996a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcb658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcb658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:39798/0 (socket says 172.21.15.45:39798) 2023-10-17T01:26:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/3141284769 learned_addr learned my addr 172.21.15.45:0/3141284769 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5b830 0x555bfcb65390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf9d9c700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5cd90 0x555bfcb6c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5b830 msgr2=0x555bfcb65390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5b830 0x555bfcb65390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/3141284769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5cd90 msgr2=0x555bfcb6c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5cd90 0x555bfcb6c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/3141284769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf41a57d0 con 0x555bfcb5c190 2023-10-17T01:26:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf959b700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5b830 0x555bfcb65390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcb658d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0be0013790 tx=0x7f0be0013aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cda212bd60e97c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.550+0000 7f0bdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:26:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0bdcff9700 1 -- 172.21.15.45:0/3141284769 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004220 con 0x555bfcb5c190 2023-10-17T01:26:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bfcad7cb0 con 0x555bfcb5c190 2023-10-17T01:26:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0bdcff9700 1 -- 172.21.15.45:0/3141284769 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0be0004380 con 0x555bfcb5c190 2023-10-17T01:26:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0bdcff9700 1 -- 172.21.15.45:0/3141284769 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0be0004540 con 0x555bfcb5c190 2023-10-17T01:26:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0c0f0c0b80 10 client.26617 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555bfc8c3c00 con 0x555bfcb5c190 2023-10-17T01:26:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0bdcff9700 1 -- 172.21.15.45:0/3141284769 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bfcad7cb0 con 0x555bfcb5c190 2023-10-17T01:26:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.551+0000 7f0bdcff9700 1 -- 172.21.15.45:0/3141284769 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0be0011a90 con 0x555bfcb5c190 2023-10-17T01:26:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0c0f0c0b80 10 client.26617 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0c0f0c0b80 10 client.26617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0c0f0c0b80 20 client.26617 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcb711a0 0x555bfcb735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0c0f0c0b80 4 client.26617 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:26:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bfcb2acd0 con 0x555bfcb711a0 2023-10-17T01:26:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.552+0000 7f0bf959b700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcb711a0 0x555bfcb735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.554+0000 7f0bf959b700 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcb711a0 0x555bfcb735c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279ab0b11b225216 server_cookie=e9231fd1dd7369fb in_seq=0 out_seq=0 2023-10-17T01:26:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.554+0000 7f0bdcff9700 10 client.26617 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0bdcff9700 1 -- 172.21.15.45:0/3141284769 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x555bfcb2acd0 con 0x555bfcb711a0 2023-10-17T01:26:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0bdcff9700 10 client.26617 handle_command_reply: tid=0 2023-10-17T01:26:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 client.26617 shutdown 2023-10-17T01:26:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 msgr2=0x555bfcb658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcb658d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0be0013790 tx=0x7f0be0013aa0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 shutdown_connections 2023-10-17T01:26:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5cd90 0x555bfcb6c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcb711a0 0x555bfcb735c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5b830 0x555bfcb65390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/3141284769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcb658d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 >> 172.21.15.45:0/3141284769 conn(0x555bfcabb070 msgr2=0x555bfcb55a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.555+0000 7f0bdcff9700 0 client.26617 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.556+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 shutdown_connections 2023-10-17T01:26:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.556+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/3141284769 wait complete. 2023-10-17T01:26:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.556+0000 7f0c0f0c0b80 20 client.26617 trim_cache size 0 max 16384 2023-10-17T01:26:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.588+0000 7f0c0f0c0b80 1 Processor -- start 2023-10-17T01:26:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 -- start start 2023-10-17T01:26:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5b830 0x555bfcb631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 0x555bfcb5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcb60230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcb66580 con 0x555bfcb5c190 2023-10-17T01:26:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcb64840 con 0x555bfcb5cd90 2023-10-17T01:26:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.589+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcb64f00 con 0x555bfcb5b830 2023-10-17T01:26:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.590+0000 7f0bf8d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 0x555bfcb5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.590+0000 7f0bf8d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 0x555bfcb5fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55308/0 (socket says 172.21.15.45:55308) 2023-10-17T01:26:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.590+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/2380379816 learned_addr learned my addr 172.21.15.45:0/2380379816 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.591+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/2380379816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5b830 msgr2=0x555bfcb631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.591+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5b830 0x555bfcb631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.591+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/2380379816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 msgr2=0x555bfcb60230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.591+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcb60230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.591+0000 7f0bf8d9a700 1 -- 172.21.15.45:0/2380379816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bfc9011f0 con 0x555bfcb5c190 2023-10-17T01:26:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.593+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 0x555bfcb5fcf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0be0011e40 tx=0x7f0be0013df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fae52e03ccded14 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.594+0000 7f0bdffff700 1 -- 172.21.15.45:0/2380379816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0011880 con 0x555bfcb5c190 2023-10-17T01:26:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.594+0000 7f0bdffff700 1 -- 172.21.15.45:0/2380379816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be0027480 con 0x555bfcb5c190 2023-10-17T01:26:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.594+0000 7f0bdffff700 1 -- 172.21.15.45:0/2380379816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0027ab0 con 0x555bfcb5c190 2023-10-17T01:26:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.596+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2380379816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 msgr2=0x555bfcb5fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.596+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 0x555bfcb5fcf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0be0011e40 tx=0x7f0be0013df0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.598+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2380379816 shutdown_connections 2023-10-17T01:26:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.598+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcb60230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.598+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5b830 0x555bfcb631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.598+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/2380379816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5c190 0x555bfcb5fcf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.598+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2380379816 >> 172.21.15.45:0/2380379816 conn(0x555bfcabb070 msgr2=0x555bfcb64060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.599+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2380379816 shutdown_connections 2023-10-17T01:26:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.599+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/2380379816 wait complete. 2023-10-17T01:26:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.600+0000 7f0c0f0c0b80 1 -- start start 2023-10-17T01:26:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.600+0000 7f0c0f0c0b80 1 Processor -- start 2023-10-17T01:26:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcbcdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcbd0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcbd2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcbd2f40 con 0x555bfcb5b830 2023-10-17T01:26:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcbd30b0 con 0x555bfcb5cd90 2023-10-17T01:26:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfcbd3270 con 0x555bfcb5c190 2023-10-17T01:26:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.601+0000 7f0c0f0c0b80 10 client.0 _ll_register_callbacks cb 0x7f0be0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.603+0000 7f0bf9d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcbd2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.603+0000 7f0bf9d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcbd2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51772/0 (socket says 172.21.15.45:51772) 2023-10-17T01:26:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.603+0000 7f0bf9d9c700 1 -- 172.21.15.45:0/508436950 learned_addr learned my addr 172.21.15.45:0/508436950 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.603+0000 7f0bf959b700 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcbcdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.604+0000 7f0bf959b700 1 -- 172.21.15.45:0/508436950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 msgr2=0x555bfcbd0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.604+0000 7f0bf959b700 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcbd0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.604+0000 7f0bf959b700 1 -- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 msgr2=0x555bfcbd2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.604+0000 7f0bf959b700 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcbd2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.604+0000 7f0bf959b700 1 -- 172.21.15.45:0/508436950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bfc8c3c00 con 0x555bfcb5b830 2023-10-17T01:26:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.605+0000 7f0bf959b700 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcbcdd00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0be8099470 tx=0x7f0be800aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902994a3f31742d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.605+0000 7f0bdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:26:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.605+0000 7f0bdcff9700 1 -- 172.21.15.45:0/508436950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8097bc0 con 0x555bfcb5b830 2023-10-17T01:26:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.605+0000 7f0bdcff9700 1 -- 172.21.15.45:0/508436950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0be8098e70 con 0x555bfcb5b830 2023-10-17T01:26:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.606+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0be80a17e0 con 0x555bfcb5b830 2023-10-17T01:26:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.606+0000 7f0bdcff9700 1 -- 172.21.15.45:0/508436950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0be80999d0 con 0x555bfcb5b830 2023-10-17T01:26:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.606+0000 7f0c0f0c0b80 10 client.16359 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.606+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555bfc8d9250 con 0x555bfcb5b830 2023-10-17T01:26:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0bdcff9700 1 -- 172.21.15.45:0/508436950 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be80a17e0 con 0x555bfcb5b830 2023-10-17T01:26:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0bdcff9700 1 -- 172.21.15.45:0/508436950 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0be80a9070 con 0x555bfcb5b830 2023-10-17T01:26:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0c0f0c0b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0c0f0c0b80 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0c0f0c0b80 20 client.16359 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcbd4820 0x555bfcbd6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0c0f0c0b80 4 client.16359 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:26:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.608+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555bfcaa5ce0 con 0x555bfcbd4820 2023-10-17T01:26:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.622+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcbd4820 0x555bfcbd6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.640+0000 7f0bf8d9a700 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcbd4820 0x555bfcbd6c40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37b33bd4d1a671b server_cookie=bb602a92df9854e2 in_seq=0 out_seq=0 2023-10-17T01:26:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.640+0000 7f0bdcff9700 10 client.16359 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.675+0000 7f0bdcff9700 1 -- 172.21.15.45:0/508436950 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555bfcaa5ce0 con 0x555bfcbd4820 2023-10-17T01:26:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.675+0000 7f0bdcff9700 10 client.16359 handle_command_reply: tid=0 2023-10-17T01:26:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.675+0000 7f0c0f0c0b80 1 client.16359 shutdown 2023-10-17T01:26:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.675+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 msgr2=0x555bfcbcdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.675+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcbcdd00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0be8099470 tx=0x7f0be800aea0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.675+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 shutdown_connections 2023-10-17T01:26:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x555bfcbd4820 0x555bfcbd6c40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0bdcff9700 0 client.16359 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555bfcb5cd90 0x555bfcbd2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bfcb5b830 0x555bfcbcdd00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 --2- 172.21.15.45:0/508436950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555bfcb5c190 0x555bfcbd0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 >> 172.21.15.45:0/508436950 conn(0x555bfcabb070 msgr2=0x555bfcb631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 shutdown_connections 2023-10-17T01:26:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 1 -- 172.21.15.45:0/508436950 wait complete. 2023-10-17T01:26:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.676+0000 7f0c0f0c0b80 20 client.16359 trim_cache size 0 max 16384 2023-10-17T01:26:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:26:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:26:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:26:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:26:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0be40a0c70 msgr2=0x7f0be40a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0be40a0c70 0x7f0be40a3130 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x555bfcb5d750 tx=0x7f0bec005fd0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 msgr2=0x7f0bf419cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf419cfe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0be8007e20 tx=0x7f0be800a340 comp rx=0 tx=0).stop 2023-10-17T01:26:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 shutdown_connections 2023-10-17T01:26:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0be40a0c70 0x7f0be40a3130 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bf410d350 0x7f0bf419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40737b0 0x7f0bf419cfe0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 --2- 172.21.15.45:0/3306879446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bf4074d90 0x7f0bf419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 >> 172.21.15.45:0/3306879446 conn(0x7f0bf40fc2d0 msgr2=0x7f0bf41006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 shutdown_connections 2023-10-17T01:26:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:11.677+0000 7f0bfb7ff700 1 -- 172.21.15.45:0/3306879446 wait complete. 2023-10-17T01:26:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:12 smithi181 ceph-mon[134520]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 659 op/s 2023-10-17T01:26:13.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:12 smithi045 ceph-mon[98474]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 659 op/s 2023-10-17T01:26:13.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:12 smithi060 ceph-mon[134136]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 659 op/s 2023-10-17T01:26:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:14 smithi181 ceph-mon[134520]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 730 op/s 2023-10-17T01:26:15.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:14 smithi045 ceph-mon[98474]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 730 op/s 2023-10-17T01:26:15.332 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:14 smithi060 ceph-mon[134136]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 730 op/s 2023-10-17T01:26:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 665.0 s, 1406.6 tps, lat 71.089 ms stddev 85.125 2023-10-17T01:26:17.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:16 smithi181 ceph-mon[134520]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 555 op/s 2023-10-17T01:26:17.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:16 smithi045 ceph-mon[98474]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 555 op/s 2023-10-17T01:26:17.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:16 smithi060 ceph-mon[134136]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 555 op/s 2023-10-17T01:26:19.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:18 smithi181 ceph-mon[134520]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 592 op/s 2023-10-17T01:26:19.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:18 smithi045 ceph-mon[98474]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 592 op/s 2023-10-17T01:26:19.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:18 smithi060 ceph-mon[134136]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 592 op/s 2023-10-17T01:26:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 670.0 s, 1336.4 tps, lat 71.712 ms stddev 93.981 2023-10-17T01:26:21.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:20 smithi181 ceph-mon[134520]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 723 op/s 2023-10-17T01:26:21.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:20 smithi045 ceph-mon[98474]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 723 op/s 2023-10-17T01:26:21.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:20 smithi060 ceph-mon[134136]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 723 op/s 2023-10-17T01:26:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:22 smithi181 ceph-mon[134520]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 863 op/s 2023-10-17T01:26:23.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:22 smithi045 ceph-mon[98474]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 863 op/s 2023-10-17T01:26:23.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:22 smithi060 ceph-mon[134136]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 863 op/s 2023-10-17T01:26:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 -- 172.21.15.45:0/2792640664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 msgr2=0x7f8460101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 --2- 172.21.15.45:0/2792640664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460101180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84500099c0 tx=0x7f8450009cd0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 -- 172.21.15.45:0/2792640664 shutdown_connections 2023-10-17T01:26:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 --2- 172.21.15.45:0/2792640664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84601028f0 0x7f84601091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 --2- 172.21.15.45:0/2792640664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460101180 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 --2- 172.21.15.45:0/2792640664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 0x7f84601023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.882+0000 7f8464dff700 1 -- 172.21.15.45:0/2792640664 >> 172.21.15.45:0/2792640664 conn(0x7f84600fc290 msgr2=0x7f84600fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.883+0000 7f8464dff700 1 -- 172.21.15.45:0/2792640664 shutdown_connections 2023-10-17T01:26:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.883+0000 7f8464dff700 1 -- 172.21.15.45:0/2792640664 wait complete. 2023-10-17T01:26:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.885+0000 7f8464dff700 1 Processor -- start 2023-10-17T01:26:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 -- start start 2023-10-17T01:26:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 0x7f8460073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84601028f0 0x7f8460073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a0ac0 con 0x7f8460101f50 2023-10-17T01:26:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a0c00 con 0x7f8460100d60 2023-10-17T01:26:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f8464dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a0d40 con 0x7f84601028f0 2023-10-17T01:26:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f845e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f845e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50824/0 (socket says 172.21.15.45:50824) 2023-10-17T01:26:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f845e59c700 1 -- 172.21.15.45:0/398196234 learned_addr learned my addr 172.21.15.45:0/398196234 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.886+0000 7f845ed9d700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84601028f0 0x7f8460073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845dd9b700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 0x7f8460073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845e59c700 1 -- 172.21.15.45:0/398196234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84601028f0 msgr2=0x7f8460073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845e59c700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84601028f0 0x7f8460073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845e59c700 1 -- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 msgr2=0x7f8460073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845e59c700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 0x7f8460073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845e59c700 1 -- 172.21.15.45:0/398196234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84500095f0 con 0x7f8460100d60 2023-10-17T01:26:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845dd9b700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 0x7f8460073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:26:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.887+0000 7f845e59c700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460073140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f84500058e0 tx=0x7f845000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.888+0000 7f844f7fe700 1 -- 172.21.15.45:0/398196234 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845001d070 con 0x7f8460100d60 2023-10-17T01:26:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.888+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84601a0f70 con 0x7f8460100d60 2023-10-17T01:26:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.888+0000 7f844f7fe700 1 -- 172.21.15.45:0/398196234 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8450005510 con 0x7f8460100d60 2023-10-17T01:26:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.888+0000 7f844f7fe700 1 -- 172.21.15.45:0/398196234 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450004b10 con 0x7f8460100d60 2023-10-17T01:26:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.888+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84601a12e0 con 0x7f8460100d60 2023-10-17T01:26:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.894+0000 7f844f7fe700 1 -- 172.21.15.45:0/398196234 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f845000fbb0 con 0x7f8460100d60 2023-10-17T01:26:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.894+0000 7f847864ab80 1 Processor -- start 2023-10-17T01:26:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.894+0000 7f847864ab80 1 -- start start 2023-10-17T01:26:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f847864ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f847864ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f847864ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f847864ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5d299b0 con 0x5629a5dd1830 2023-10-17T01:26:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f847864ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5d27370 con 0x5629a5dd2190 2023-10-17T01:26:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f847864ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5d22c60 con 0x5629a5dd2d90 2023-10-17T01:26:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f845dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f845dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55358/0 (socket says 172.21.15.45:55358) 2023-10-17T01:26:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.895+0000 7f845dd9b700 1 -- 172.21.15.45:0/962204033 learned_addr learned my addr 172.21.15.45:0/962204033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.896+0000 7f845dd9b700 1 -- 172.21.15.45:0/962204033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 msgr2=0x5629a5dd2790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:26:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.896+0000 7f845dd9b700 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd2790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.896+0000 7f845dd9b700 1 -- 172.21.15.45:0/962204033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 msgr2=0x5629a5dd1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.896+0000 7f845dd9b700 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.896+0000 7f845dd9b700 1 -- 172.21.15.45:0/962204033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629a5b50d80 con 0x5629a5dd2d90 2023-10-17T01:26:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.897+0000 7f845dd9b700 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd3210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8454009d80 tx=0x7f84540116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e0c576b60761c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.897+0000 7f844f7fe700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84480a0b20 0x7f84480a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.897+0000 7f844f7fe700 1 -- 172.21.15.45:0/398196234 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f84500ccec0 con 0x7f8460100d60 2023-10-17T01:26:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.898+0000 7f845ed9d700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84480a0b20 0x7f84480a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.898+0000 7f844cff9700 1 -- 172.21.15.45:0/962204033 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454011ed0 con 0x5629a5dd2d90 2023-10-17T01:26:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.898+0000 7f844cff9700 1 -- 172.21.15.45:0/962204033 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8454012910 con 0x5629a5dd2d90 2023-10-17T01:26:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.898+0000 7f844cff9700 1 -- 172.21.15.45:0/962204033 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400f830 con 0x5629a5dd2d90 2023-10-17T01:26:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 -- 172.21.15.45:0/962204033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 msgr2=0x5629a5dd3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd3210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8454009d80 tx=0x7f84540116c0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f845ed9d700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84480a0b20 0x7f84480a2fe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f845800f520 tx=0x7f8458009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 -- 172.21.15.45:0/962204033 shutdown_connections 2023-10-17T01:26:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd3210 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 --2- 172.21.15.45:0/962204033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 -- 172.21.15.45:0/962204033 >> 172.21.15.45:0/962204033 conn(0x5629a5d31070 msgr2=0x5629a5dcc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 -- 172.21.15.45:0/962204033 shutdown_connections 2023-10-17T01:26:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.899+0000 7f847864ab80 1 -- 172.21.15.45:0/962204033 wait complete. 2023-10-17T01:26:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 -- start start 2023-10-17T01:26:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 Processor -- start 2023-10-17T01:26:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 0x5629a5ddb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd2190 0x5629a5ddb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2d90 0x5629a5de2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5cbf4b0 con 0x5629a5dd2190 2023-10-17T01:26:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5cbf150 con 0x5629a5dd2d90 2023-10-17T01:26:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5ba68c0 con 0x5629a5dd1830 2023-10-17T01:26:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.900+0000 7f847864ab80 10 client.0 _ll_register_callbacks cb 0x5629a5c0ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 0x5629a5ddb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 0x5629a5ddb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55372/0 (socket says 172.21.15.45:55372) 2023-10-17T01:26:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 -- 172.21.15.45:0/1348229097 learned_addr learned my addr 172.21.15.45:0/1348229097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845ed9d700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2d90 0x5629a5de2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845dd9b700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd2190 0x5629a5ddb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 -- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2d90 msgr2=0x5629a5de2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2d90 0x5629a5de2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 -- 172.21.15.45:0/1348229097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd2190 msgr2=0x5629a5ddb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd2190 0x5629a5ddb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 -- 172.21.15.45:0/1348229097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84540099b0 con 0x5629a5dd1830 2023-10-17T01:26:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f845e59c700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 0x5629a5ddb3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8450096ad0 tx=0x7f8450004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edaf269a5565fe1a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f8445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:26:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f8445ffb700 1 -- 172.21.15.45:0/1348229097 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845009acd0 con 0x5629a5dd1830 2023-10-17T01:26:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.901+0000 7f8445ffb700 1 -- 172.21.15.45:0/1348229097 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f845009a5d0 con 0x5629a5dd1830 2023-10-17T01:26:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629a5d4dcb0 con 0x5629a5dd1830 2023-10-17T01:26:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f8445ffb700 1 -- 172.21.15.45:0/1348229097 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f845009e400 con 0x5629a5dd1830 2023-10-17T01:26:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 10 client.26629 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5629a5b39c00 con 0x5629a5dd1830 2023-10-17T01:26:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f8445ffb700 1 -- 172.21.15.45:0/1348229097 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629a5d4dcb0 con 0x5629a5dd1830 2023-10-17T01:26:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f8445ffb700 1 -- 172.21.15.45:0/1348229097 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8450099ac0 con 0x5629a5dd1830 2023-10-17T01:26:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 10 client.26629 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 20 client.26629 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5de6dc0 0x5629a5de91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 4 client.26629 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:26:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629a5da0cd0 con 0x5629a5de6dc0 2023-10-17T01:26:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.902+0000 7f845dd9b700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5de6dc0 0x5629a5de91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.903+0000 7f845dd9b700 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5de6dc0 0x5629a5de91e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94b26b7648cd8fb server_cookie=50a78e728e551ded in_seq=0 out_seq=0 2023-10-17T01:26:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.904+0000 7f8445ffb700 10 client.26629 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.904+0000 7f8445ffb700 1 -- 172.21.15.45:0/1348229097 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5629a5da0cd0 con 0x5629a5de6dc0 2023-10-17T01:26:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.904+0000 7f8445ffb700 10 client.26629 handle_command_reply: tid=0 2023-10-17T01:26:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.905+0000 7f847864ab80 1 client.26629 shutdown 2023-10-17T01:26:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 msgr2=0x5629a5ddb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 0x5629a5ddb3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8450096ad0 tx=0x7f8450004f70 comp rx=0 tx=0).stop 2023-10-17T01:26:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 shutdown_connections 2023-10-17T01:26:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5de6dc0 0x5629a5de91e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2d90 0x5629a5de2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd1830 0x5629a5ddb3d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 --2- 172.21.15.45:0/1348229097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd2190 0x5629a5ddb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 >> 172.21.15.45:0/1348229097 conn(0x5629a5d31070 msgr2=0x5629a5dcba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f8445ffb700 0 client.26629 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 shutdown_connections 2023-10-17T01:26:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.906+0000 7f847864ab80 1 -- 172.21.15.45:0/1348229097 wait complete. 2023-10-17T01:26:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.907+0000 7f847864ab80 20 client.26629 trim_cache size 0 max 16384 2023-10-17T01:26:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 Processor -- start 2023-10-17T01:26:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 -- start start 2023-10-17T01:26:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5d4dcb0 con 0x5629a5dd1830 2023-10-17T01:26:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5ddc5c0 con 0x5629a5dd2190 2023-10-17T01:26:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.925+0000 7f847864ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5dda880 con 0x5629a5dd2d90 2023-10-17T01:26:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:55382/0 (socket says 172.21.15.45:55382) 2023-10-17T01:26:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50836/0 (socket says 172.21.15.45:50836) 2023-10-17T01:26:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845ed9d700 1 -- 172.21.15.45:0/2071436407 learned_addr learned my addr 172.21.15.45:0/2071436407 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845dd9b700 1 -- 172.21.15.45:0/2071436407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 msgr2=0x5629a5dd6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845dd9b700 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845dd9b700 1 -- 172.21.15.45:0/2071436407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 msgr2=0x5629a5dd91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.926+0000 7f845e59c700 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.927+0000 7f845dd9b700 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.927+0000 7f845dd9b700 1 -- 172.21.15.45:0/2071436407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629a5b771f0 con 0x5629a5dd2190 2023-10-17T01:26:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.927+0000 7f845e59c700 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:26:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.928+0000 7f845dd9b700 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd5d20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205e0536f7d909c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.928+0000 7f844d7fa700 1 -- 172.21.15.45:0/2071436407 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540095c0 con 0x5629a5dd2190 2023-10-17T01:26:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.928+0000 7f844d7fa700 1 -- 172.21.15.45:0/2071436407 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8454009370 con 0x5629a5dd2190 2023-10-17T01:26:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.928+0000 7f844d7fa700 1 -- 172.21.15.45:0/2071436407 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454005040 con 0x5629a5dd2190 2023-10-17T01:26:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 -- 172.21.15.45:0/2071436407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 msgr2=0x5629a5dd5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd5d20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454005940 comp rx=0 tx=0).stop 2023-10-17T01:26:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 -- 172.21.15.45:0/2071436407 shutdown_connections 2023-10-17T01:26:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5dd6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5dd91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 --2- 172.21.15.45:0/2071436407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5dd5d20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 -- 172.21.15.45:0/2071436407 >> 172.21.15.45:0/2071436407 conn(0x5629a5d31070 msgr2=0x5629a5dda0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 -- 172.21.15.45:0/2071436407 shutdown_connections 2023-10-17T01:26:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.929+0000 7f847864ab80 1 -- 172.21.15.45:0/2071436407 wait complete. 2023-10-17T01:26:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 -- start start 2023-10-17T01:26:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 Processor -- start 2023-10-17T01:26:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5e32910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5e34e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5e374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5e37c30 con 0x5629a5dd1830 2023-10-17T01:26:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5e37da0 con 0x5629a5dd2190 2023-10-17T01:26:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a5e37f60 con 0x5629a5dd2d90 2023-10-17T01:26:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.930+0000 7f847864ab80 10 client.0 _ll_register_callbacks cb 0x7f8450096a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5e34e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5e374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5e34e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50838/0 (socket says 172.21.15.45:50838) 2023-10-17T01:26:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 -- 172.21.15.45:0/872089766 learned_addr learned my addr 172.21.15.45:0/872089766 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845e59c700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5e32910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 -- 172.21.15.45:0/872089766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 msgr2=0x5629a5e374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5e374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 -- 172.21.15.45:0/872089766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 msgr2=0x5629a5e32910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5e32910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 -- 172.21.15.45:0/872089766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629a5b50d80 con 0x5629a5dd2190 2023-10-17T01:26:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845ed9d700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5e374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f845dd9b700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5e34e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8454011ac0 tx=0x7f8454011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9116b6ca101c94c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.931+0000 7f8445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:26:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f8445ffb700 1 -- 172.21.15.45:0/872089766 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454012450 con 0x5629a5dd2190 2023-10-17T01:26:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f8445ffb700 1 -- 172.21.15.45:0/872089766 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f84540125b0 con 0x5629a5dd2190 2023-10-17T01:26:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8454012910 con 0x5629a5dd2190 2023-10-17T01:26:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f8445ffb700 1 -- 172.21.15.45:0/872089766 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8454012770 con 0x5629a5dd2190 2023-10-17T01:26:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f847864ab80 10 client.26684 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5629a5b4f250 con 0x5629a5dd2190 2023-10-17T01:26:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f8445ffb700 1 -- 172.21.15.45:0/872089766 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8454012910 con 0x5629a5dd2190 2023-10-17T01:26:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.932+0000 7f8445ffb700 1 -- 172.21.15.45:0/872089766 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8454044050 con 0x5629a5dd2190 2023-10-17T01:26:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f847864ab80 10 client.26684 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f847864ab80 10 client.26684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f847864ab80 20 client.26684 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f847864ab80 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5e39430 0x5629a5e3b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f847864ab80 4 client.26684 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:26:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629a5d1bce0 con 0x5629a5e39430 2023-10-17T01:26:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.933+0000 7f845e59c700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5e39430 0x5629a5e3b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.934+0000 7f845e59c700 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5e39430 0x5629a5e3b850 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338577e9b0f1de24 server_cookie=33f6ee0e6132f44b in_seq=0 out_seq=0 2023-10-17T01:26:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.934+0000 7f8445ffb700 10 client.26684 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f8445ffb700 1 -- 172.21.15.45:0/872089766 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629a5d1bce0 con 0x5629a5e39430 2023-10-17T01:26:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f8445ffb700 10 client.26684 handle_command_reply: tid=0 2023-10-17T01:26:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 client.26684 shutdown 2023-10-17T01:26:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 msgr2=0x5629a5e34e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5e34e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8454011ac0 tx=0x7f8454011bf0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 shutdown_connections 2023-10-17T01:26:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5629a5dd2d90 0x5629a5e374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629a5dd1830 0x5629a5e32910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5629a5e39430 0x5629a5e3b850 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 --2- 172.21.15.45:0/872089766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629a5dd2190 0x5629a5e34e60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f8445ffb700 0 client.26684 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 >> 172.21.15.45:0/872089766 conn(0x5629a5d31070 msgr2=0x5629a5dd91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 shutdown_connections 2023-10-17T01:26:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 1 -- 172.21.15.45:0/872089766 wait complete. 2023-10-17T01:26:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.947+0000 7f847864ab80 20 client.26684 trim_cache size 0 max 16384 2023-10-17T01:26:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:26:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:26:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:26:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:26:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84480a0b20 msgr2=0x7f84480a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84480a0b20 0x7f84480a2fe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f845800f520 tx=0x7f8458009920 comp rx=0 tx=0).stop 2023-10-17T01:26:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 msgr2=0x7f8460073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460073140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f84500058e0 tx=0x7f845000f690 comp rx=0 tx=0).stop 2023-10-17T01:26:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 shutdown_connections 2023-10-17T01:26:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84480a0b20 0x7f84480a2fe0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84601028f0 0x7f8460073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8460100d60 0x7f8460073140 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 --2- 172.21.15.45:0/398196234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8460101f50 0x7f8460073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 >> 172.21.15.45:0/398196234 conn(0x7f84600fc290 msgr2=0x7f84600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 shutdown_connections 2023-10-17T01:26:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:24.948+0000 7f8464dff700 1 -- 172.21.15.45:0/398196234 wait complete. 2023-10-17T01:26:25.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:24 smithi181 ceph-mon[134520]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 1.88k op/s 2023-10-17T01:26:25.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:24 smithi045 ceph-mon[98474]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 1.88k op/s 2023-10-17T01:26:25.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:24 smithi060 ceph-mon[134136]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 1.88k op/s 2023-10-17T01:26:25.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 675.0 s, 1074.5 tps, lat 97.445 ms stddev 162.511 2023-10-17T01:26:27.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:26 smithi181 ceph-mon[134520]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-17T01:26:27.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:26 smithi045 ceph-mon[98474]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-17T01:26:27.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:26 smithi060 ceph-mon[134136]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-17T01:26:29.159 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:28 smithi181 ceph-mon[134520]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-10-17T01:26:29.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:28 smithi045 ceph-mon[98474]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-10-17T01:26:29.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:28 smithi060 ceph-mon[134136]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-10-17T01:26:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 680.0 s, 1469.6 tps, lat 67.594 ms stddev 76.684 2023-10-17T01:26:31.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:30 smithi181 ceph-mon[134520]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 1.90k op/s 2023-10-17T01:26:31.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:30 smithi045 ceph-mon[98474]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 1.90k op/s 2023-10-17T01:26:31.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:30 smithi060 ceph-mon[134136]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 1.90k op/s 2023-10-17T01:26:33.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:32 smithi181 ceph-mon[134520]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-17T01:26:33.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:32 smithi045 ceph-mon[98474]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-17T01:26:33.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:32 smithi060 ceph-mon[134136]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-17T01:26:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:34 smithi181 ceph-mon[134520]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 1.65k op/s 2023-10-17T01:26:35.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:34 smithi045 ceph-mon[98474]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 1.65k op/s 2023-10-17T01:26:35.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:34 smithi060 ceph-mon[134136]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 1.65k op/s 2023-10-17T01:26:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 685.0 s, 1516.5 tps, lat 65.837 ms stddev 82.973 2023-10-17T01:26:37.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:26:37.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:36 smithi181 ceph-mon[134520]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 607 op/s 2023-10-17T01:26:37.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:36 smithi045 ceph-mon[98474]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 607 op/s 2023-10-17T01:26:37.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:36 smithi060 ceph-mon[134136]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 607 op/s 2023-10-17T01:26:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 -- 172.21.15.45:0/1071599024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 msgr2=0x7f82c0100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 --2- 172.21.15.45:0/1071599024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c0100060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f82b00099c0 tx=0x7f82b0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 -- 172.21.15.45:0/1071599024 shutdown_connections 2023-10-17T01:26:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 --2- 172.21.15.45:0/1071599024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c01017d0 0x7f82c0108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 --2- 172.21.15.45:0/1071599024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c0100060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 --2- 172.21.15.45:0/1071599024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c0100e30 0x7f82c0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 -- 172.21.15.45:0/1071599024 >> 172.21.15.45:0/1071599024 conn(0x7f82c00fb1b0 msgr2=0x7f82c00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 -- 172.21.15.45:0/1071599024 shutdown_connections 2023-10-17T01:26:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.476+0000 7f82c4fff700 1 -- 172.21.15.45:0/1071599024 wait complete. 2023-10-17T01:26:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.478+0000 7f82c4fff700 1 Processor -- start 2023-10-17T01:26:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.478+0000 7f82c4fff700 1 -- start start 2023-10-17T01:26:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82c4fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c01948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82c4fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 0x7f82c0194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82c4fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c01017d0 0x7f82c019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82c4fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019c430 con 0x7f82c01017d0 2023-10-17T01:26:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82c4fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019c5a0 con 0x7f82c00ffc40 2023-10-17T01:26:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82c4fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019c760 con 0x7f82c0100e30 2023-10-17T01:26:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 0x7f82c0194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 0x7f82c0194df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37324/0 (socket says 172.21.15.45:37324) 2023-10-17T01:26:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82b7fff700 1 -- 172.21.15.45:0/2356411450 learned_addr learned my addr 172.21.15.45:0/2356411450 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:37.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82bed9d700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c01017d0 0x7f82c019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.479+0000 7f82be59c700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c01948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82b7fff700 1 -- 172.21.15.45:0/2356411450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 msgr2=0x7f82c01948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82b7fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c01948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82b7fff700 1 -- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c01017d0 msgr2=0x7f82c019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82b7fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c01017d0 0x7f82c019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82b7fff700 1 -- 172.21.15.45:0/2356411450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b8009c40 con 0x7f82c0100e30 2023-10-17T01:26:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82be59c700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c01948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:26:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.480+0000 7f82b7fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 0x7f82c0194df0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e9a0 tx=0x7f82a800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.481+0000 7f82b77fe700 1 -- 172.21.15.45:0/2356411450 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8009790 con 0x7f82c0100e30 2023-10-17T01:26:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.481+0000 7f82b77fe700 1 -- 172.21.15.45:0/2356411450 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a800cbd0 con 0x7f82c0100e30 2023-10-17T01:26:37.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.481+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b00095f0 con 0x7f82c0100e30 2023-10-17T01:26:37.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.481+0000 7f82b77fe700 1 -- 172.21.15.45:0/2356411450 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8018680 con 0x7f82c0100e30 2023-10-17T01:26:37.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.481+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c019cdb0 con 0x7f82c0100e30 2023-10-17T01:26:37.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.484+0000 7f82b77fe700 1 -- 172.21.15.45:0/2356411450 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f82a800cd40 con 0x7f82c0100e30 2023-10-17T01:26:37.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.488+0000 7f82d8993b80 1 Processor -- start 2023-10-17T01:26:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.488+0000 7f82d8993b80 1 -- start start 2023-10-17T01:26:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.488+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 0x558dce1c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.488+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c2500 0x558dce1c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.488+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82d8993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce119a10 con 0x558dce1c2500 2023-10-17T01:26:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82d8993b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1173d0 con 0x558dce1c32b0 2023-10-17T01:26:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82d8993b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce112cc0 con 0x558dce1c1ba0 2023-10-17T01:26:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 0x558dce1c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 0x558dce1c1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37338/0 (socket says 172.21.15.45:37338) 2023-10-17T01:26:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82bed9d700 1 -- 172.21.15.45:0/4088474167 learned_addr learned my addr 172.21.15.45:0/4088474167 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:37.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.489+0000 7f82b77fe700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82ac0a0ba0 0x7f82ac0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.490+0000 7f82b77fe700 1 -- 172.21.15.45:0/2356411450 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82a8014070 con 0x7f82c0100e30 2023-10-17T01:26:37.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.490+0000 7f82bed9d700 1 -- 172.21.15.45:0/4088474167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 msgr2=0x558dce1c3730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:26:37.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.490+0000 7f82bed9d700 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1c3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.490+0000 7f82bed9d700 1 -- 172.21.15.45:0/4088474167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c2500 msgr2=0x558dce1c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.490+0000 7f82bed9d700 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c2500 0x558dce1c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.490+0000 7f82bed9d700 1 -- 172.21.15.45:0/4088474167 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dcdf40de0 con 0x558dce1c1ba0 2023-10-17T01:26:37.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.491+0000 7f82bed9d700 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 0x558dce1c1fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e840 tx=0x7f82b800eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f825e9c7a19abc7d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.491+0000 7f82b4ff9700 1 -- 172.21.15.45:0/4088474167 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800e040 con 0x558dce1c1ba0 2023-10-17T01:26:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.491+0000 7f82b4ff9700 1 -- 172.21.15.45:0/4088474167 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b800f040 con 0x558dce1c1ba0 2023-10-17T01:26:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.491+0000 7f82b4ff9700 1 -- 172.21.15.45:0/4088474167 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8007760 con 0x558dce1c1ba0 2023-10-17T01:26:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.491+0000 7f82bed9d700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82ac0a0ba0 0x7f82ac0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.492+0000 7f82d8993b80 1 -- 172.21.15.45:0/4088474167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 msgr2=0x558dce1c1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.492+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 0x558dce1c1fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e840 tx=0x7f82b800eb50 comp rx=0 tx=0).stop 2023-10-17T01:26:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 -- 172.21.15.45:0/4088474167 shutdown_connections 2023-10-17T01:26:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c1ba0 0x558dce1c1fc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4088474167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c2500 0x558dce1c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 -- 172.21.15.45:0/4088474167 >> 172.21.15.45:0/4088474167 conn(0x558dce1210d0 msgr2=0x558dce1bc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 -- 172.21.15.45:0/4088474167 shutdown_connections 2023-10-17T01:26:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82bed9d700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82ac0a0ba0 0x7f82ac0a3060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e840 tx=0x7f82b800eb50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.493+0000 7f82d8993b80 1 -- 172.21.15.45:0/4088474167 wait complete. 2023-10-17T01:26:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.494+0000 7f82d8993b80 1 -- start start 2023-10-17T01:26:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.494+0000 7f82d8993b80 1 Processor -- start 2023-10-17T01:26:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce0af510 con 0x558dce1c1ba0 2023-10-17T01:26:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce0af1b0 con 0x558dce1c2500 2023-10-17T01:26:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcdf96920 con 0x558dce1c32b0 2023-10-17T01:26:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82d8993b80 10 client.0 _ll_register_callbacks cb 0x558dcdffca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40174/0 (socket says 172.21.15.45:40174) 2023-10-17T01:26:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1d2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37350/0 (socket says 172.21.15.45:37350) 2023-10-17T01:26:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.495+0000 7f82b7fff700 1 -- 172.21.15.45:0/4097453031 learned_addr learned my addr 172.21.15.45:0/4097453031 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82be59c700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82bed9d700 1 -- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 msgr2=0x558dce1cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82bed9d700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82bed9d700 1 -- 172.21.15.45:0/4097453031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 msgr2=0x558dce1cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82bed9d700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82bed9d700 1 -- 172.21.15.45:0/4097453031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b800e460 con 0x558dce1c32b0 2023-10-17T01:26:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82be59c700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:26:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.496+0000 7f82b7fff700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.497+0000 7f82bed9d700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1d2ec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009040 tx=0x7f82b8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=611f33ac8a77e13b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.497+0000 7f82a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:26:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.497+0000 7f82a5ffb700 1 -- 172.21.15.45:0/4097453031 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8004ca0 con 0x558dce1c32b0 2023-10-17T01:26:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.498+0000 7f82a5ffb700 1 -- 172.21.15.45:0/4097453031 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82b8004e00 con 0x558dce1c32b0 2023-10-17T01:26:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.498+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dce13dd10 con 0x558dce1c32b0 2023-10-17T01:26:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.498+0000 7f82a5ffb700 1 -- 172.21.15.45:0/4097453031 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82b8007b90 con 0x558dce1c32b0 2023-10-17T01:26:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.498+0000 7f82d8993b80 10 client.26647 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.498+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558dcdf29c60 con 0x558dce1c32b0 2023-10-17T01:26:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82a5ffb700 1 -- 172.21.15.45:0/4097453031 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558dce13dd10 con 0x558dce1c32b0 2023-10-17T01:26:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82a5ffb700 1 -- 172.21.15.45:0/4097453031 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f82b8020420 con 0x558dce1c32b0 2023-10-17T01:26:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82d8993b80 10 client.26647 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82d8993b80 10 client.26647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82d8993b80 20 client.26647 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce1d7590 0x558dce1d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82d8993b80 4 client.26647 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.499+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dce190d30 con 0x558dce1d7590 2023-10-17T01:26:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.500+0000 7f82be59c700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce1d7590 0x558dce1d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.500+0000 7f82be59c700 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce1d7590 0x558dce1d99b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd43901cc86f011 server_cookie=989a78dea883758a in_seq=0 out_seq=0 2023-10-17T01:26:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.500+0000 7f82a5ffb700 10 client.26647 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.501+0000 7f82a5ffb700 1 -- 172.21.15.45:0/4097453031 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x558dce190d30 con 0x558dce1d7590 2023-10-17T01:26:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.501+0000 7f82a5ffb700 10 client.26647 handle_command_reply: tid=0 2023-10-17T01:26:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.501+0000 7f82d8993b80 1 client.26647 shutdown 2023-10-17T01:26:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 msgr2=0x558dce1d2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1d2ec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009040 tx=0x7f82b8009920 comp rx=0 tx=0).stop 2023-10-17T01:26:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 shutdown_connections 2023-10-17T01:26:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1d2ec0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce1d7590 0x558dce1d99b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 --2- 172.21.15.45:0/4097453031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 >> 172.21.15.45:0/4097453031 conn(0x558dce1210d0 msgr2=0x558dce1bc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:37.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82a5ffb700 0 client.26647 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:37.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 shutdown_connections 2023-10-17T01:26:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 1 -- 172.21.15.45:0/4097453031 wait complete. 2023-10-17T01:26:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.502+0000 7f82d8993b80 20 client.26647 trim_cache size 0 max 16384 2023-10-17T01:26:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.530+0000 7f82d8993b80 1 Processor -- start 2023-10-17T01:26:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.530+0000 7f82d8993b80 1 -- start start 2023-10-17T01:26:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82d8993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1ccb60 con 0x558dce1c1ba0 2023-10-17T01:26:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82d8993b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1b9450 con 0x558dce1c2500 2023-10-17T01:26:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82d8993b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1cadf0 con 0x558dce1c32b0 2023-10-17T01:26:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37364/0 (socket says 172.21.15.45:37364) 2023-10-17T01:26:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82bed9d700 1 -- 172.21.15.45:0/3480900104 learned_addr learned my addr 172.21.15.45:0/3480900104 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.531+0000 7f82b7fff700 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82bed9d700 1 -- 172.21.15.45:0/3480900104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 msgr2=0x558dce1d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82bed9d700 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1d37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82bed9d700 1 -- 172.21.15.45:0/3480900104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 msgr2=0x558dce1c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82be59c700 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82bed9d700 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82bed9d700 1 -- 172.21.15.45:0/3480900104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dcdf67250 con 0x558dce1c32b0 2023-10-17T01:26:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82be59c700 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:26:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.532+0000 7f82bed9d700 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1c6200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f82b802bb50 tx=0x7f82b80078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd402ebd8c7b8c3d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:37.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.533+0000 7f82b57fa700 1 -- 172.21.15.45:0/3480900104 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800e100 con 0x558dce1c32b0 2023-10-17T01:26:37.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.533+0000 7f82b57fa700 1 -- 172.21.15.45:0/3480900104 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b8020a50 con 0x558dce1c32b0 2023-10-17T01:26:37.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.533+0000 7f82b57fa700 1 -- 172.21.15.45:0/3480900104 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b802bbd0 con 0x558dce1c32b0 2023-10-17T01:26:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.533+0000 7f82d8993b80 1 -- 172.21.15.45:0/3480900104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 msgr2=0x558dce1c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.533+0000 7f82d8993b80 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1c6200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f82b802bb50 tx=0x7f82b80078e0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 -- 172.21.15.45:0/3480900104 shutdown_connections 2023-10-17T01:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c32b0 0x558dce1c6200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 --2- 172.21.15.45:0/3480900104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c2500 0x558dce1d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 -- 172.21.15.45:0/3480900104 >> 172.21.15.45:0/3480900104 conn(0x558dce1210d0 msgr2=0x558dce1ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:37.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 -- 172.21.15.45:0/3480900104 shutdown_connections 2023-10-17T01:26:37.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 -- 172.21.15.45:0/3480900104 wait complete. 2023-10-17T01:26:37.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.534+0000 7f82d8993b80 1 -- start start 2023-10-17T01:26:37.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.535+0000 7f82d8993b80 1 Processor -- start 2023-10-17T01:26:37.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.535+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.535+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c2500 0x558dce1ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.535+0000 7f82d8993b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82d8993b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1d02f0 con 0x558dce1c1ba0 2023-10-17T01:26:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82d8993b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1d0460 con 0x558dce1c32b0 2023-10-17T01:26:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82d8993b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dce1d0620 con 0x558dce1c2500 2023-10-17T01:26:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82d8993b80 10 client.0 _ll_register_callbacks cb 0x558dcdffca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c2500 0x558dce1ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.536+0000 7f82be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58036/0 (socket says 172.21.15.45:58036) 2023-10-17T01:26:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.537+0000 7f82be59c700 1 -- 172.21.15.45:0/1692518108 learned_addr learned my addr 172.21.15.45:0/1692518108 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:37.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.537+0000 7f82be59c700 1 -- 172.21.15.45:0/1692518108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c2500 msgr2=0x558dce1ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.537+0000 7f82be59c700 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c2500 0x558dce1ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.537+0000 7f82be59c700 1 -- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 msgr2=0x558dce1cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.537+0000 7f82be59c700 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.537+0000 7f82be59c700 1 -- 172.21.15.45:0/1692518108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dcdf29c60 con 0x558dce1c1ba0 2023-10-17T01:26:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.538+0000 7f82be59c700 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1ce860 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f82b0000c00 tx=0x7f82b0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697b36e40ab63e59 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.538+0000 7f82a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:26:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.538+0000 7f82a5ffb700 1 -- 172.21.15.45:0/1692518108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0027240 con 0x558dce1c1ba0 2023-10-17T01:26:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.538+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82b800f040 con 0x558dce1c1ba0 2023-10-17T01:26:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.539+0000 7f82a5ffb700 1 -- 172.21.15.45:0/1692518108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82b001d040 con 0x558dce1c1ba0 2023-10-17T01:26:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.539+0000 7f82a5ffb700 1 -- 172.21.15.45:0/1692518108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82b000bd80 con 0x558dce1c1ba0 2023-10-17T01:26:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.539+0000 7f82d8993b80 10 client.16383 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.539+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558dcdf3f2b0 con 0x558dce1c1ba0 2023-10-17T01:26:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82a5ffb700 1 -- 172.21.15.45:0/1692518108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b800f040 con 0x558dce1c1ba0 2023-10-17T01:26:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82a5ffb700 1 -- 172.21.15.45:0/1692518108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f82b0027be0 con 0x558dce1c1ba0 2023-10-17T01:26:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82d8993b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:37.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82d8993b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:37.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82d8993b80 20 client.16383 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:37.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82d8993b80 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce229b50 0x558dce22bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:37.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82d8993b80 4 client.16383 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:26:37.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558dce10bd40 con 0x558dce229b50 2023-10-17T01:26:37.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.540+0000 7f82b7fff700 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce229b50 0x558dce22bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.541+0000 7f82b7fff700 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce229b50 0x558dce22bf70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7019ff1d3511509d server_cookie=f160fb8ce6c5a77d in_seq=0 out_seq=0 2023-10-17T01:26:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.541+0000 7f82a5ffb700 10 client.16383 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82a5ffb700 1 -- 172.21.15.45:0/1692518108 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558dce10bd40 con 0x558dce229b50 2023-10-17T01:26:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82a5ffb700 10 client.16383 handle_command_reply: tid=0 2023-10-17T01:26:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 client.16383 shutdown 2023-10-17T01:26:37.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 msgr2=0x558dce1ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1ce860 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f82b0000c00 tx=0x7f82b0005c90 comp rx=0 tx=0).stop 2023-10-17T01:26:37.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 shutdown_connections 2023-10-17T01:26:37.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558dce229b50 0x558dce22bf70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558dce1c32b0 0x558dce1cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dce1c1ba0 0x558dce1ce860 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 --2- 172.21.15.45:0/1692518108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558dce1c2500 0x558dce1ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 >> 172.21.15.45:0/1692518108 conn(0x558dce1210d0 msgr2=0x558dce1d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.542+0000 7f82a5ffb700 0 client.16383 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 shutdown_connections 2023-10-17T01:26:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82d8993b80 1 -- 172.21.15.45:0/1692518108 wait complete. 2023-10-17T01:26:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82d8993b80 20 client.16383 trim_cache size 0 max 16384 2023-10-17T01:26:37.552 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:26:37.552 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:26:37.552 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:26:37.553 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:26:37.553 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:26:37.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:26:37.553 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:26:37.553 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:26:37.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:26:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:26:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82ac0a0ba0 msgr2=0x7f82ac0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82c4fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82ac0a0ba0 0x7f82ac0a3060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82b800e840 tx=0x7f82b800eb50 comp rx=0 tx=0).stop 2023-10-17T01:26:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 msgr2=0x7f82c0194df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.543+0000 7f82c4fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 0x7f82c0194df0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e9a0 tx=0x7f82a800ed60 comp rx=0 tx=0).stop 2023-10-17T01:26:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 shutdown_connections 2023-10-17T01:26:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f82ac0a0ba0 0x7f82ac0a3060 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c01017d0 0x7f82c019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c00ffc40 0x7f82c01948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 --2- 172.21.15.45:0/2356411450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c0100e30 0x7f82c0194df0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 >> 172.21.15.45:0/2356411450 conn(0x7f82c00fb1b0 msgr2=0x7f82c0067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 shutdown_connections 2023-10-17T01:26:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:26:37.544+0000 7f82c4fff700 1 -- 172.21.15.45:0/2356411450 wait complete. 2023-10-17T01:26:37.586 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:26:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.177+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/1579858711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 msgr2=0x7f5c98101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.177+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/1579858711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98101180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5c940099c0 tx=0x7f5c94009cd0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/1579858711 shutdown_connections 2023-10-17T01:26:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/1579858711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 0x7f5c981091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/1579858711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98101180 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/1579858711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 0x7f5c981023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/1579858711 >> 172.21.15.45:0/1579858711 conn(0x7f5c980fc290 msgr2=0x7f5c980fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/1579858711 shutdown_connections 2023-10-17T01:26:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.178+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/1579858711 wait complete. 2023-10-17T01:26:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.179+0000 7f5c9f5ff700 1 Processor -- start 2023-10-17T01:26:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.180+0000 7f5c9f5ff700 1 -- start start 2023-10-17T01:26:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9f5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9f5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 0x7f5c98194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9f5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 0x7f5c98195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9f5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c98196390 con 0x7f5c98101f50 2023-10-17T01:26:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9f5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c98196500 con 0x7f5c98100d60 2023-10-17T01:26:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9f5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c981966c0 con 0x7f5c981028f0 2023-10-17T01:26:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.181+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9db9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 0x7f5c98195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9db9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 0x7f5c98195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:58436/0 (socket says 172.21.15.45:58436) 2023-10-17T01:26:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9db9c700 1 -- 172.21.15.45:0/3507588708 learned_addr learned my addr 172.21.15.45:0/3507588708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98194970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45466/0 (socket says 172.21.15.45:45466) 2023-10-17T01:26:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9cb9a700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 0x7f5c98194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3507588708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 msgr2=0x7f5c98195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 0x7f5c98195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.182+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 msgr2=0x7f5c98194eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 0x7f5c98194eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3507588708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940095f0 con 0x7f5c98100d60 2023-10-17T01:26:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c9cb9a700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 0x7f5c98194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:26:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98194970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5c940098e0 tx=0x7f5c9400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/3507588708 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9401d070 con 0x7f5c98100d60 2023-10-17T01:26:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/3507588708 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c940173f0 con 0x7f5c98100d60 2023-10-17T01:26:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c98196940 con 0x7f5c98100d60 2023-10-17T01:26:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.183+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/3507588708 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400e640 con 0x7f5c98100d60 2023-10-17T01:26:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.184+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9819d160 con 0x7f5c98100d60 2023-10-17T01:26:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.192+0000 7f5cb2f61b80 1 Processor -- start 2023-10-17T01:26:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 -- start start 2023-10-17T01:26:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d5341dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 0x5561d5340580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d5340fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d529b9b0 con 0x5561d53419b0 2023-10-17T01:26:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d5299370 con 0x5561d5340b60 2023-10-17T01:26:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d5294c60 con 0x5561d5342330 2023-10-17T01:26:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.193+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/3507588708 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f5c94017560 con 0x7f5c98100d60 2023-10-17T01:26:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.196+0000 7f5c8e7fc700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5c840a0c70 0x7f5c840a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.197+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 0x5561d5340580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.197+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 0x5561d5340580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:58438/0 (socket says 172.21.15.45:58438) 2023-10-17T01:26:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.197+0000 7f5c9d39b700 1 -- 172.21.15.45:0/1562028657 learned_addr learned my addr 172.21.15.45:0/1562028657 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.197+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/3507588708 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5c940ccfe0 con 0x7f5c98100d60 2023-10-17T01:26:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.197+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5c840a0c70 0x7f5c840a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.198+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d5341dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.198+0000 7f5c9d39b700 1 -- 172.21.15.45:0/1562028657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 msgr2=0x5561d5340fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.198+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d5340fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.198+0000 7f5c9d39b700 1 -- 172.21.15.45:0/1562028657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 msgr2=0x5561d5341dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.198+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d5341dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.198+0000 7f5c9d39b700 1 -- 172.21.15.45:0/1562028657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d50c2d80 con 0x5561d5342330 2023-10-17T01:26:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.200+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5c840a0c70 0x7f5c840a3130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x5561d5340510 tx=0x7f5c9000dd30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.201+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 0x5561d5340580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c940971c0 tx=0x7f5c9400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d1b7dea7b63d1fb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.209+0000 7f5c837fe700 1 -- 172.21.15.45:0/1562028657 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9409ae50 con 0x5561d5342330 2023-10-17T01:26:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.209+0000 7f5c837fe700 1 -- 172.21.15.45:0/1562028657 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c94028340 con 0x5561d5342330 2023-10-17T01:26:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.209+0000 7f5c837fe700 1 -- 172.21.15.45:0/1562028657 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94098b80 con 0x5561d5342330 2023-10-17T01:26:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.212+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/1562028657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 msgr2=0x5561d5340580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.212+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 0x5561d5340580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c940971c0 tx=0x7f5c9400b540 comp rx=0 tx=0).stop 2023-10-17T01:26:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/1562028657 shutdown_connections 2023-10-17T01:26:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d5340fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d5341dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/1562028657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5342330 0x5561d5340580 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c940971c0 tx=0x7f5c9400b540 comp rx=0 tx=0).stop 2023-10-17T01:26:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/1562028657 >> 172.21.15.45:0/1562028657 conn(0x5561d52a3070 msgr2=0x5561d533e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/1562028657 shutdown_connections 2023-10-17T01:26:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.213+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/1562028657 wait complete. 2023-10-17T01:26:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.214+0000 7f5cb2f61b80 1 -- start start 2023-10-17T01:26:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.214+0000 7f5cb2f61b80 1 Processor -- start 2023-10-17T01:26:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.215+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d535e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.215+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d535e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.215+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5365bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.215+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d52314b0 con 0x5561d53637e0 2023-10-17T01:26:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.216+0000 7f5c9db9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5365bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.216+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d5231150 con 0x5561d5340b60 2023-10-17T01:26:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.216+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d51188c0 con 0x5561d53419b0 2023-10-17T01:26:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.216+0000 7f5c9db9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5365bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37282/0 (socket says 172.21.15.45:37282) 2023-10-17T01:26:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.217+0000 7f5cb2f61b80 10 client.0 _ll_register_callbacks cb 0x5561d517ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.217+0000 7f5c9cb9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d535e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.217+0000 7f5c9cb9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d535e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:58450/0 (socket says 172.21.15.45:58450) 2023-10-17T01:26:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.217+0000 7f5c9cb9a700 1 -- 172.21.15.45:0/2398155370 learned_addr learned my addr 172.21.15.45:0/2398155370 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.218+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d535e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.218+0000 7f5c9db9c700 1 -- 172.21.15.45:0/2398155370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 msgr2=0x5561d535e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.218+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d535e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.218+0000 7f5c9db9c700 1 -- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 msgr2=0x5561d535e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.218+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d535e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.218+0000 7f5c9db9c700 1 -- 172.21.15.45:0/2398155370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9819d160 con 0x5561d53637e0 2023-10-17T01:26:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.222+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5365bd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000f1a0 tx=0x7f5c90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805c92dc79eb56cd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.222+0000 7f5c80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:26:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.223+0000 7f5c80ff9700 1 -- 172.21.15.45:0/2398155370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90012050 con 0x5561d53637e0 2023-10-17T01:26:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.223+0000 7f5c80ff9700 1 -- 172.21.15.45:0/2398155370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5c90008860 con 0x5561d53637e0 2023-10-17T01:26:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.223+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561d52bfcb0 con 0x5561d53637e0 2023-10-17T01:26:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.223+0000 7f5c80ff9700 1 -- 172.21.15.45:0/2398155370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c90008a20 con 0x5561d53637e0 2023-10-17T01:26:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.223+0000 7f5cb2f61b80 10 client.16389 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.223+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5561d50abc00 con 0x5561d53637e0 2023-10-17T01:26:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.224+0000 7f5c80ff9700 1 -- 172.21.15.45:0/2398155370 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561d52bfcb0 con 0x5561d53637e0 2023-10-17T01:26:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5c80ff9700 1 -- 172.21.15.45:0/2398155370 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5c90023070 con 0x5561d53637e0 2023-10-17T01:26:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5cb2f61b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5cb2f61b80 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5cb2f61b80 20 client.16389 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d536a6b0 0x5561d536cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5cb2f61b80 4 client.16389 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:26:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.225+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561d5312cd0 con 0x5561d536a6b0 2023-10-17T01:26:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.226+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d536a6b0 0x5561d536cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.227+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d536a6b0 0x5561d536cad0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c1b6a198762145 server_cookie=584878c69ade56 in_seq=0 out_seq=0 2023-10-17T01:26:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.227+0000 7f5c80ff9700 10 client.16389 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.228+0000 7f5c80ff9700 1 -- 172.21.15.45:0/2398155370 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5561d5312cd0 con 0x5561d536a6b0 2023-10-17T01:26:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.228+0000 7f5c80ff9700 10 client.16389 handle_command_reply: tid=0 2023-10-17T01:26:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.228+0000 7f5cb2f61b80 1 client.16389 shutdown 2023-10-17T01:26:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.228+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 msgr2=0x5561d5365bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.228+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5365bd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000f1a0 tx=0x7f5c90004ad0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.228+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 shutdown_connections 2023-10-17T01:26:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5365bd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d536a6b0 0x5561d536cad0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d535e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/2398155370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d535e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 >> 172.21.15.45:0/2398155370 conn(0x5561d52a3070 msgr2=0x5561d533da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5c80ff9700 0 client.16389 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 shutdown_connections 2023-10-17T01:26:38.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.229+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/2398155370 wait complete. 2023-10-17T01:26:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.230+0000 7f5cb2f61b80 20 client.16389 trim_cache size 0 max 16384 2023-10-17T01:26:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.264+0000 7f5cb2f61b80 1 Processor -- start 2023-10-17T01:26:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.264+0000 7f5cb2f61b80 1 -- start start 2023-10-17T01:26:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.265+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 0x5561d5365bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.265+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d533da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d53637e0 0x5561d533cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d5312cd0 con 0x5561d53419b0 2023-10-17T01:26:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d52bfcb0 con 0x5561d53637e0 2023-10-17T01:26:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d53389e0 con 0x5561d5340b60 2023-10-17T01:26:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 0x5561d5365bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 0x5561d5365bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:58456/0 (socket says 172.21.15.45:58456) 2023-10-17T01:26:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3101233988 learned_addr learned my addr 172.21.15.45:0/3101233988 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9db9c700 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d53637e0 0x5561d533cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3101233988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d53637e0 msgr2=0x5561d533cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d53637e0 0x5561d533cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3101233988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 msgr2=0x5561d533da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d533da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3101233988 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d50e91f0 con 0x5561d5340b60 2023-10-17T01:26:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.266+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 0x5561d5365bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c9409a6e0 tx=0x7f5c940284b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f39c15594c7ff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.267+0000 7f5c83fff700 1 -- 172.21.15.45:0/3101233988 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9409ae50 con 0x5561d5340b60 2023-10-17T01:26:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.267+0000 7f5c83fff700 1 -- 172.21.15.45:0/3101233988 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9409e580 con 0x5561d5340b60 2023-10-17T01:26:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.267+0000 7f5c83fff700 1 -- 172.21.15.45:0/3101233988 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9409e160 con 0x5561d5340b60 2023-10-17T01:26:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3101233988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 msgr2=0x5561d5365bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 0x5561d5365bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c9409a6e0 tx=0x7f5c940284b0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3101233988 shutdown_connections 2023-10-17T01:26:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d53637e0 0x5561d533cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d5340b60 0x5561d5365bd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3101233988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53419b0 0x5561d533da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.268+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3101233988 >> 172.21.15.45:0/3101233988 conn(0x5561d52a3070 msgr2=0x5561d533e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.270+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3101233988 shutdown_connections 2023-10-17T01:26:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.272+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3101233988 wait complete. 2023-10-17T01:26:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.272+0000 7f5cb2f61b80 1 -- start start 2023-10-17T01:26:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.272+0000 7f5cb2f61b80 1 Processor -- start 2023-10-17T01:26:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.273+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d53786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d5378c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5cb2f61b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5379140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d53ba530 con 0x5561d53637e0 2023-10-17T01:26:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d53ba670 con 0x5561d5340b60 2023-10-17T01:26:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5cb2f61b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d53ba830 con 0x5561d53419b0 2023-10-17T01:26:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d53786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5cb2f61b80 10 client.0 _ll_register_callbacks cb 0x5561d517ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5c9d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d53786c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45500/0 (socket says 172.21.15.45:45500) 2023-10-17T01:26:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.274+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3088321666 learned_addr learned my addr 172.21.15.45:0/3088321666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.275+0000 7f5c9cb9a700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d5378c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.275+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3088321666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 msgr2=0x5561d5378c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.275+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d5378c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.275+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3088321666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 msgr2=0x5561d5379140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.275+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5379140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.276+0000 7f5c9d39b700 1 -- 172.21.15.45:0/3088321666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561d50c2d80 con 0x5561d5340b60 2023-10-17T01:26:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.276+0000 7f5c9cb9a700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d5378c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.277+0000 7f5c9d39b700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d53786c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5c940a28f0 tx=0x7f5c94009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba397e3140671fd server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.277+0000 7f5c80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:26:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.279+0000 7f5c80ff9700 1 -- 172.21.15.45:0/3088321666 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9409e160 con 0x5561d5340b60 2023-10-17T01:26:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.279+0000 7f5c80ff9700 1 -- 172.21.15.45:0/3088321666 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5c94098890 con 0x5561d5340b60 2023-10-17T01:26:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.279+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c94028340 con 0x5561d5340b60 2023-10-17T01:26:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.280+0000 7f5c80ff9700 1 -- 172.21.15.45:0/3088321666 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9409ae50 con 0x5561d5340b60 2023-10-17T01:26:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.280+0000 7f5cb2f61b80 10 client.26714 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.280+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5561d50c1250 con 0x5561d5340b60 2023-10-17T01:26:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.280+0000 7f5c80ff9700 1 -- 172.21.15.45:0/3088321666 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c94028340 con 0x5561d5340b60 2023-10-17T01:26:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5c80ff9700 1 -- 172.21.15.45:0/3088321666 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f5c940a22a0 con 0x5561d5340b60 2023-10-17T01:26:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5cb2f61b80 10 client.26714 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5cb2f61b80 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5cb2f61b80 20 client.26714 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d53bbc80 0x5561d53be070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5cb2f61b80 4 client.26714 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:26:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5561d528dce0 con 0x5561d53bbc80 2023-10-17T01:26:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.281+0000 7f5c9cb9a700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d53bbc80 0x5561d53be070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.283+0000 7f5c80ff9700 10 client.26714 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.284+0000 7f5c9cb9a700 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d53bbc80 0x5561d53be070 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4275340aaf83b54d server_cookie=4bfb553a1344a54d in_seq=0 out_seq=0 2023-10-17T01:26:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.300+0000 7f5c80ff9700 1 -- 172.21.15.45:0/3088321666 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5561d528dce0 con 0x5561d53bbc80 2023-10-17T01:26:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.300+0000 7f5c80ff9700 10 client.26714 handle_command_reply: tid=0 2023-10-17T01:26:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.300+0000 7f5cb2f61b80 1 client.26714 shutdown 2023-10-17T01:26:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 msgr2=0x5561d53786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d53786c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5c940a28f0 tx=0x7f5c94009310 comp rx=0 tx=0).stop 2023-10-17T01:26:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 shutdown_connections 2023-10-17T01:26:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561d53637e0 0x5561d5379140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5561d53bbc80 0x5561d53be070 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5561d5340b60 0x5561d53786c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 --2- 172.21.15.45:0/3088321666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5561d53419b0 0x5561d5378c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5c80ff9700 0 client.26714 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 >> 172.21.15.45:0/3088321666 conn(0x5561d52a3070 msgr2=0x5561d535cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 shutdown_connections 2023-10-17T01:26:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 1 -- 172.21.15.45:0/3088321666 wait complete. 2023-10-17T01:26:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.301+0000 7f5cb2f61b80 20 client.26714 trim_cache size 0 max 16384 2023-10-17T01:26:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:26:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:26:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:26:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:26:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5c840a0c70 msgr2=0x7f5c840a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5c840a0c70 0x7f5c840a3130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x5561d5340510 tx=0x7f5c9000dd30 comp rx=0 tx=0).stop 2023-10-17T01:26:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 msgr2=0x7f5c98194970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98194970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5c940098e0 tx=0x7f5c9400f690 comp rx=0 tx=0).stop 2023-10-17T01:26:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 shutdown_connections 2023-10-17T01:26:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f5c840a0c70 0x7f5c840a3130 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c981028f0 0x7f5c98195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c98100d60 0x7f5c98194970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 --2- 172.21.15.45:0/3507588708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c98101f50 0x7f5c98194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 >> 172.21.15.45:0/3507588708 conn(0x7f5c980fc290 msgr2=0x7f5c980fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 shutdown_connections 2023-10-17T01:26:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:38.302+0000 7f5c9f5ff700 1 -- 172.21.15.45:0/3507588708 wait complete. 2023-10-17T01:26:38.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:38 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:26:38.835 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:38 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:26:38.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:38 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:26:39.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:39 smithi045 ceph-mon[98474]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 622 op/s 2023-10-17T01:26:39.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:39 smithi060 ceph-mon[134136]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 622 op/s 2023-10-17T01:26:39.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:39 smithi181 ceph-mon[134520]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 622 op/s 2023-10-17T01:26:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 690.0 s, 1376.7 tps, lat 73.288 ms stddev 84.253 2023-10-17T01:26:40.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:26:40.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:26:40.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:26:40.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:26:40.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:26:40.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:26:40.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:26:40.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:26:40.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:26:41.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:41 smithi045 ceph-mon[98474]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 699 op/s 2023-10-17T01:26:41.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:41 smithi060 ceph-mon[134136]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 699 op/s 2023-10-17T01:26:41.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:41 smithi181 ceph-mon[134520]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 699 op/s 2023-10-17T01:26:43.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:42 smithi181 ceph-mon[134520]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 576 op/s 2023-10-17T01:26:43.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:42 smithi045 ceph-mon[98474]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 576 op/s 2023-10-17T01:26:43.336 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:42 smithi060 ceph-mon[134136]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.4 MiB/s wr, 576 op/s 2023-10-17T01:26:45.162 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:44 smithi181 ceph-mon[134520]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 732 op/s 2023-10-17T01:26:45.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:44 smithi045 ceph-mon[98474]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 732 op/s 2023-10-17T01:26:45.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:44 smithi060 ceph-mon[134136]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 732 op/s 2023-10-17T01:26:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 695.0 s, 1363.5 tps, lat 72.301 ms stddev 89.906 2023-10-17T01:26:47.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:46 smithi181 ceph-mon[134520]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 MiB/s wr, 535 op/s 2023-10-17T01:26:47.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:46 smithi045 ceph-mon[98474]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 MiB/s wr, 535 op/s 2023-10-17T01:26:47.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:46 smithi060 ceph-mon[134136]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 MiB/s wr, 535 op/s 2023-10-17T01:26:49.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:48 smithi181 ceph-mon[134520]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 567 op/s 2023-10-17T01:26:49.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:48 smithi045 ceph-mon[98474]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 567 op/s 2023-10-17T01:26:49.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:48 smithi060 ceph-mon[134136]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 567 op/s 2023-10-17T01:26:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 700.0 s, 1456.8 tps, lat 69.577 ms stddev 80.774 2023-10-17T01:26:51.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:50 smithi181 ceph-mon[134520]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 690 op/s 2023-10-17T01:26:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:50 smithi045 ceph-mon[98474]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 690 op/s 2023-10-17T01:26:51.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:50 smithi060 ceph-mon[134136]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 690 op/s 2023-10-17T01:26:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 -- 172.21.15.45:0/3428772298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa650100d60 msgr2=0x7fa650101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 --2- 172.21.15.45:0/3428772298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa650100d60 0x7fa650101180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009990 tx=0x7fa64c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 -- 172.21.15.45:0/3428772298 shutdown_connections 2023-10-17T01:26:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 --2- 172.21.15.45:0/3428772298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6501028f0 0x7fa6501091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 --2- 172.21.15.45:0/3428772298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa650100d60 0x7fa650101180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 --2- 172.21.15.45:0/3428772298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650101f50 0x7fa6501023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 -- 172.21.15.45:0/3428772298 >> 172.21.15.45:0/3428772298 conn(0x7fa6500fc290 msgr2=0x7fa6500fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 -- 172.21.15.45:0/3428772298 shutdown_connections 2023-10-17T01:26:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.534+0000 7fa6583ff700 1 -- 172.21.15.45:0/3428772298 wait complete. 2023-10-17T01:26:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.536+0000 7fa6583ff700 1 Processor -- start 2023-10-17T01:26:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.536+0000 7fa6583ff700 1 -- start start 2023-10-17T01:26:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.537+0000 7fa6583ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650100d60 0x7fa65019d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.537+0000 7fa6583ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 0x7fa65019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.537+0000 7fa6583ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6501028f0 0x7fa65019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa6583ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65019ec00 con 0x7fa650100d60 2023-10-17T01:26:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa6583ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65019ed70 con 0x7fa650101f50 2023-10-17T01:26:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa6583ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65019ef30 con 0x7fa6501028f0 2023-10-17T01:26:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa65599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 0x7fa65019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa65599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 0x7fa65019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50500/0 (socket says 172.21.15.45:50500) 2023-10-17T01:26:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa65599a700 1 -- 172.21.15.45:0/1973219447 learned_addr learned my addr 172.21.15.45:0/1973219447 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa65619b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650100d60 0x7fa65019d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.538+0000 7fa65699c700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6501028f0 0x7fa65019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.539+0000 7fa65599a700 1 -- 172.21.15.45:0/1973219447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6501028f0 msgr2=0x7fa65019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.539+0000 7fa65599a700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6501028f0 0x7fa65019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.539+0000 7fa65599a700 1 -- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650100d60 msgr2=0x7fa65019d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.539+0000 7fa65599a700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650100d60 0x7fa65019d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.539+0000 7fa65599a700 1 -- 172.21.15.45:0/1973219447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0095f0 con 0x7fa650101f50 2023-10-17T01:26:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.539+0000 7fa65619b700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650100d60 0x7fa65019d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.540+0000 7fa65699c700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6501028f0 0x7fa65019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:26:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.540+0000 7fa65599a700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 0x7fa65019d720 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e9a0 tx=0x7fa64000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.540+0000 7fa6477fe700 1 -- 172.21.15.45:0/1973219447 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640009790 con 0x7fa650101f50 2023-10-17T01:26:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.540+0000 7fa6477fe700 1 -- 172.21.15.45:0/1973219447 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64000cbd0 con 0x7fa650101f50 2023-10-17T01:26:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.540+0000 7fa6477fe700 1 -- 172.21.15.45:0/1973219447 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640018680 con 0x7fa650101f50 2023-10-17T01:26:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.540+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65019f210 con 0x7fa650101f50 2023-10-17T01:26:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.541+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501a59d0 con 0x7fa650101f50 2023-10-17T01:26:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.551+0000 7fa6477fe700 1 -- 172.21.15.45:0/1973219447 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa64000cd40 con 0x7fa650101f50 2023-10-17T01:26:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.556+0000 7fa6477fe700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa63c0a0ba0 0x7fa63c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.556+0000 7fa6477fe700 1 -- 172.21.15.45:0/1973219447 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa640014070 con 0x7fa650101f50 2023-10-17T01:26:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa65619b700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa63c0a0ba0 0x7fa63c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa66bc38b80 1 Processor -- start 2023-10-17T01:26:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa66bc38b80 1 -- start start 2023-10-17T01:26:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563838159830 0x563838159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383815ad90 0x56383815b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.560+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638380b19b0 con 0x563838159830 2023-10-17T01:26:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.561+0000 7fa65599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.561+0000 7fa65599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50506/0 (socket says 172.21.15.45:50506) 2023-10-17T01:26:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.561+0000 7fa65599a700 1 -- 172.21.15.45:0/659541277 learned_addr learned my addr 172.21.15.45:0/659541277 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.562+0000 7fa65619b700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa63c0a0ba0 0x7fa63c0a3060 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b2e0 tx=0x7fa64c00bd40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.562+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638380af370 con 0x56383815a190 2023-10-17T01:26:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.562+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638380aac60 con 0x56383815ad90 2023-10-17T01:26:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.562+0000 7fa65619b700 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563838159830 0x563838159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.563+0000 7fa65699c700 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383815ad90 0x56383815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.563+0000 7fa65619b700 1 -- 172.21.15.45:0/659541277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383815ad90 msgr2=0x56383815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.563+0000 7fa65619b700 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383815ad90 0x56383815b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.563+0000 7fa65619b700 1 -- 172.21.15.45:0/659541277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 msgr2=0x56383815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.563+0000 7fa65619b700 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.563+0000 7fa65619b700 1 -- 172.21.15.45:0/659541277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563837ed8d80 con 0x563838159830 2023-10-17T01:26:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.564+0000 7fa65619b700 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563838159830 0x563838159c50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b680 tx=0x7fa64c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ab3957f9da6577 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.565+0000 7fa644bfe700 1 -- 172.21.15.45:0/659541277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c02b040 con 0x563838159830 2023-10-17T01:26:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.565+0000 7fa644bfe700 1 -- 172.21.15.45:0/659541277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64c01e040 con 0x563838159830 2023-10-17T01:26:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.565+0000 7fa644bfe700 1 -- 172.21.15.45:0/659541277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00fbf0 con 0x563838159830 2023-10-17T01:26:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563838159830 msgr2=0x563838159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563838159830 0x563838159c50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b680 tx=0x7fa64c024000 comp rx=0 tx=0).stop 2023-10-17T01:26:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 shutdown_connections 2023-10-17T01:26:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56383815ad90 0x56383815b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563838159830 0x563838159c50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/659541277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.570+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 >> 172.21.15.45:0/659541277 conn(0x5638380b9070 msgr2=0x5638381542c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.571+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 shutdown_connections 2023-10-17T01:26:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.571+0000 7fa66bc38b80 1 -- 172.21.15.45:0/659541277 wait complete. 2023-10-17T01:26:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.572+0000 7fa66bc38b80 1 -- start start 2023-10-17T01:26:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.572+0000 7fa66bc38b80 1 Processor -- start 2023-10-17T01:26:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.574+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.574+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815a190 0x5638381638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.574+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815ad90 0x56383816a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638380474b0 con 0x56383815a190 2023-10-17T01:26:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563838047150 con 0x56383815ad90 2023-10-17T01:26:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa65619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563837f2e8c0 con 0x563838159830 2023-10-17T01:26:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa65619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60004/0 (socket says 172.21.15.45:60004) 2023-10-17T01:26:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa65619b700 1 -- 172.21.15.45:0/1711222595 learned_addr learned my addr 172.21.15.45:0/1711222595 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.575+0000 7fa65699c700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815ad90 0x56383816a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65599a700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815a190 0x5638381638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65619b700 1 -- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815ad90 msgr2=0x56383816a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65619b700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815ad90 0x56383816a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65619b700 1 -- 172.21.15.45:0/1711222595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815a190 msgr2=0x5638381638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65619b700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815a190 0x5638381638f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65619b700 1 -- 172.21.15.45:0/1711222595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa650109af0 con 0x563838159830 2023-10-17T01:26:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa66bc38b80 10 client.0 _ll_register_callbacks cb 0x563837f94a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638380d5cb0 con 0x563838159830 2023-10-17T01:26:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa65619b700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381633b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00f690 tx=0x7fa64c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ae91eed9274a3b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.576+0000 7fa639ffb700 10 client.26683 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:26:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.578+0000 7fa639ffb700 1 -- 172.21.15.45:0/1711222595 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c02b040 con 0x563838159830 2023-10-17T01:26:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.578+0000 7fa639ffb700 1 -- 172.21.15.45:0/1711222595 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa64c004d70 con 0x563838159830 2023-10-17T01:26:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.578+0000 7fa639ffb700 1 -- 172.21.15.45:0/1711222595 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64c00fc90 con 0x563838159830 2023-10-17T01:26:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.578+0000 7fa66bc38b80 10 client.26683 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.578+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x563837ec1c00 con 0x563838159830 2023-10-17T01:26:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.581+0000 7fa639ffb700 1 -- 172.21.15.45:0/1711222595 <== mon.1 v2:172.21.15.181:3300/0 4 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa64c021070 con 0x563838159830 2023-10-17T01:26:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.583+0000 7fa66bc38b80 10 client.26683 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.583+0000 7fa66bc38b80 10 client.26683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.583+0000 7fa66bc38b80 20 client.26683 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.583+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56383816e830 0x563838170cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.583+0000 7fa66bc38b80 4 client.26683 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:26:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.583+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563838128cd0 con 0x56383816e830 2023-10-17T01:26:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.597+0000 7fa65599a700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56383816e830 0x563838170cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.598+0000 7fa65599a700 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56383816e830 0x563838170cd0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da23bf515945584 server_cookie=2c9d17b6c88bef50 in_seq=0 out_seq=0 2023-10-17T01:26:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.598+0000 7fa639ffb700 10 client.26683 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.599+0000 7fa639ffb700 1 -- 172.21.15.45:0/1711222595 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x563838128cd0 con 0x56383816e830 2023-10-17T01:26:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.599+0000 7fa639ffb700 10 client.26683 handle_command_reply: tid=0 2023-10-17T01:26:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.599+0000 7fa66bc38b80 1 client.26683 shutdown 2023-10-17T01:26:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.599+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 msgr2=0x5638381633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.599+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381633b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00f690 tx=0x7fa64c005770 comp rx=0 tx=0).stop 2023-10-17T01:26:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 shutdown_connections 2023-10-17T01:26:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56383816e830 0x563838170cd0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815ad90 0x56383816a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381633b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/1711222595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815a190 0x5638381638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 >> 172.21.15.45:0/1711222595 conn(0x5638380b9070 msgr2=0x563838153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa639ffb700 0 client.26683 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 shutdown_connections 2023-10-17T01:26:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.600+0000 7fa66bc38b80 1 -- 172.21.15.45:0/1711222595 wait complete. 2023-10-17T01:26:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.601+0000 7fa66bc38b80 20 client.26683 trim_cache size 0 max 16384 2023-10-17T01:26:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.624+0000 7fa66bc38b80 1 Processor -- start 2023-10-17T01:26:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.624+0000 7fa66bc38b80 1 -- start start 2023-10-17T01:26:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.624+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x56383815de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.624+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x563838163160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383814e9e0 con 0x56383815ad90 2023-10-17T01:26:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563838154ee0 con 0x56383815a190 2023-10-17T01:26:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563838155d40 con 0x563838159830 2023-10-17T01:26:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa65619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x56383815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa65619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x56383815de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60020/0 (socket says 172.21.15.45:60020) 2023-10-17T01:26:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa65619b700 1 -- 172.21.15.45:0/4194376164 learned_addr learned my addr 172.21.15.45:0/4194376164 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa65699c700 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x563838163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.625+0000 7fa65599a700 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.626+0000 7fa65619b700 1 -- 172.21.15.45:0/4194376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 msgr2=0x56383815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.626+0000 7fa65619b700 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.626+0000 7fa65619b700 1 -- 172.21.15.45:0/4194376164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 msgr2=0x563838163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.626+0000 7fa65619b700 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x563838163160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.626+0000 7fa65619b700 1 -- 172.21.15.45:0/4194376164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563837eff1f0 con 0x563838159830 2023-10-17T01:26:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.627+0000 7fa65619b700 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x56383815de10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0385e0 tx=0x7fa64c021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83294062061ee1b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.627+0000 7fa6453ff700 1 -- 172.21.15.45:0/4194376164 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c02b040 con 0x563838159830 2023-10-17T01:26:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.627+0000 7fa6453ff700 1 -- 172.21.15.45:0/4194376164 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638380d5cb0 con 0x563838159830 2023-10-17T01:26:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.627+0000 7fa6453ff700 1 -- 172.21.15.45:0/4194376164 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c004dd0 con 0x563838159830 2023-10-17T01:26:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 -- 172.21.15.45:0/4194376164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 msgr2=0x56383815de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x56383815de10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0385e0 tx=0x7fa64c021000 comp rx=0 tx=0).stop 2023-10-17T01:26:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 -- 172.21.15.45:0/4194376164 shutdown_connections 2023-10-17T01:26:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x563838163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x56383815de10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/4194376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x56383815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 -- 172.21.15.45:0/4194376164 >> 172.21.15.45:0/4194376164 conn(0x5638380b9070 msgr2=0x563838170840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 -- 172.21.15.45:0/4194376164 shutdown_connections 2023-10-17T01:26:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.628+0000 7fa66bc38b80 1 -- 172.21.15.45:0/4194376164 wait complete. 2023-10-17T01:26:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.629+0000 7fa66bc38b80 1 -- start start 2023-10-17T01:26:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.629+0000 7fa66bc38b80 1 Processor -- start 2023-10-17T01:26:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x5638381b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x5638381bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638381bf800 con 0x56383815ad90 2023-10-17T01:26:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638381bf940 con 0x56383815a190 2023-10-17T01:26:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638381bfb00 con 0x563838159830 2023-10-17T01:26:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.630+0000 7fa66bc38b80 10 client.0 _ll_register_callbacks cb 0x7fa6480091d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:26:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.631+0000 7fa65619b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.631+0000 7fa65599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x5638381b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.631+0000 7fa65699c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x5638381bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.631+0000 7fa65599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x5638381b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50538/0 (socket says 172.21.15.45:50538) 2023-10-17T01:26:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.631+0000 7fa65599a700 1 -- 172.21.15.45:0/3653496784 learned_addr learned my addr 172.21.15.45:0/3653496784 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:26:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65699c700 1 -- 172.21.15.45:0/3653496784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 msgr2=0x5638381b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65699c700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65699c700 1 -- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 msgr2=0x5638381b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:26:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65699c700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x5638381b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65699c700 1 -- 172.21.15.45:0/3653496784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563837ec1c00 con 0x56383815ad90 2023-10-17T01:26:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65599a700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x5638381b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65619b700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:26:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa65699c700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x5638381bf2c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x56383815b750 tx=0x7fa648007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc5e2f2d2585243 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:26:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.632+0000 7fa639ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:26:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.633+0000 7fa639ffb700 1 -- 172.21.15.45:0/3653496784 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648004bb0 con 0x56383815ad90 2023-10-17T01:26:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.633+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa64c01e040 con 0x56383815ad90 2023-10-17T01:26:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.633+0000 7fa639ffb700 1 -- 172.21.15.45:0/3653496784 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa648003a00 con 0x56383815ad90 2023-10-17T01:26:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.633+0000 7fa639ffb700 1 -- 172.21.15.45:0/3653496784 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa648003c00 con 0x56383815ad90 2023-10-17T01:26:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.633+0000 7fa66bc38b80 10 client.16404 fetch_fsmap learned FSMap version 40 2023-10-17T01:26:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.633+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563837ed7250 con 0x56383815ad90 2023-10-17T01:26:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa639ffb700 1 -- 172.21.15.45:0/3653496784 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64c01e040 con 0x56383815ad90 2023-10-17T01:26:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa639ffb700 1 -- 172.21.15.45:0/3653496784 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa64801e030 con 0x56383815ad90 2023-10-17T01:26:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa66bc38b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:26:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa66bc38b80 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:26:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa66bc38b80 20 client.16404 populate_metadata read hostname 'smithi045' 2023-10-17T01:26:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5638381c1590 0x5638381c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:26:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa66bc38b80 4 client.16404 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:26:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.634+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5638380a3ce0 con 0x5638381c1590 2023-10-17T01:26:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.635+0000 7fa65619b700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5638381c1590 0x5638381c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:26:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.635+0000 7fa65619b700 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5638381c1590 0x5638381c39b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89d69374a5b4fa3 server_cookie=c2fabb25aa41a1d4 in_seq=0 out_seq=0 2023-10-17T01:26:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.635+0000 7fa639ffb700 10 client.16404 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.647+0000 7fa639ffb700 1 -- 172.21.15.45:0/3653496784 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5638380a3ce0 con 0x5638381c1590 2023-10-17T01:26:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.647+0000 7fa639ffb700 10 client.16404 handle_command_reply: tid=0 2023-10-17T01:26:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.647+0000 7fa66bc38b80 1 client.16404 shutdown 2023-10-17T01:26:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.647+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 msgr2=0x5638381bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.647+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x5638381bf2c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x56383815b750 tx=0x7fa648007e10 comp rx=0 tx=0).stop 2023-10-17T01:26:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 shutdown_connections 2023-10-17T01:26:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56383815ad90 0x5638381bf2c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563838159830 0x5638381b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5638381c1590 0x5638381c39b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 --2- 172.21.15.45:0/3653496784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56383815a190 0x5638381b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 >> 172.21.15.45:0/3653496784 conn(0x5638380b9070 msgr2=0x56383815de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa639ffb700 0 client.16404 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:26:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 shutdown_connections 2023-10-17T01:26:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 1 -- 172.21.15.45:0/3653496784 wait complete. 2023-10-17T01:26:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.648+0000 7fa66bc38b80 20 client.16404 trim_cache size 0 max 16384 2023-10-17T01:26:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:26:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:26:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:26:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:26:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa63c0a0ba0 msgr2=0x7fa63c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa63c0a0ba0 0x7fa63c0a3060 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b2e0 tx=0x7fa64c00bd40 comp rx=0 tx=0).stop 2023-10-17T01:26:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 msgr2=0x7fa65019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:26:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 0x7fa65019d720 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e9a0 tx=0x7fa64000ecb0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 shutdown_connections 2023-10-17T01:26:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa63c0a0ba0 0x7fa63c0a3060 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6501028f0 0x7fa65019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa650100d60 0x7fa65019d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 --2- 172.21.15.45:0/1973219447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa650101f50 0x7fa65019d720 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:26:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 >> 172.21.15.45:0/1973219447 conn(0x7fa6500fc290 msgr2=0x7fa6500fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:26:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 shutdown_connections 2023-10-17T01:26:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:26:51.649+0000 7fa6583ff700 1 -- 172.21.15.45:0/1973219447 wait complete. 2023-10-17T01:26:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:52 smithi181 ceph-mon[134520]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 610 op/s 2023-10-17T01:26:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:52 smithi045 ceph-mon[98474]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 610 op/s 2023-10-17T01:26:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:52 smithi060 ceph-mon[134136]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 610 op/s 2023-10-17T01:26:55.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:54 smithi181 ceph-mon[134520]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 781 op/s 2023-10-17T01:26:55.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:54 smithi045 ceph-mon[98474]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 781 op/s 2023-10-17T01:26:55.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:54 smithi060 ceph-mon[134136]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 781 op/s 2023-10-17T01:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 705.0 s, 1562.8 tps, lat 64.017 ms stddev 78.196 2023-10-17T01:26:57.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:56 smithi181 ceph-mon[134520]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 600 op/s 2023-10-17T01:26:57.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:56 smithi045 ceph-mon[98474]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 600 op/s 2023-10-17T01:26:57.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:56 smithi060 ceph-mon[134136]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 600 op/s 2023-10-17T01:26:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:26:58 smithi181 ceph-mon[134520]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 654 op/s 2023-10-17T01:26:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:26:58 smithi045 ceph-mon[98474]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 654 op/s 2023-10-17T01:26:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:26:58 smithi060 ceph-mon[134136]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 654 op/s 2023-10-17T01:27:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 710.0 s, 1278.6 tps, lat 77.731 ms stddev 87.760 2023-10-17T01:27:01.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:00 smithi181 ceph-mon[134520]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 762 op/s 2023-10-17T01:27:01.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:00 smithi045 ceph-mon[98474]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 762 op/s 2023-10-17T01:27:01.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:00 smithi060 ceph-mon[134136]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 MiB/s wr, 762 op/s 2023-10-17T01:27:03.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:02 smithi181 ceph-mon[134520]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 646 op/s 2023-10-17T01:27:03.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:02 smithi045 ceph-mon[98474]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 646 op/s 2023-10-17T01:27:03.333 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:02 smithi060 ceph-mon[134136]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 646 op/s 2023-10-17T01:27:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.877+0000 7f9b6ffff700 1 -- 172.21.15.45:0/2954976629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 msgr2=0x7f9b681012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.877+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/2954976629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b681012a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b540099c0 tx=0x7f9b54009cd0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 -- 172.21.15.45:0/2954976629 shutdown_connections 2023-10-17T01:27:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/2954976629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b681017e0 0x7f9b68108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/2954976629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b680ffc50 0x7f9b68100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/2954976629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b681012a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 -- 172.21.15.45:0/2954976629 >> 172.21.15.45:0/2954976629 conn(0x7f9b680fb1a0 msgr2=0x7f9b680fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 -- 172.21.15.45:0/2954976629 shutdown_connections 2023-10-17T01:27:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.878+0000 7f9b6ffff700 1 -- 172.21.15.45:0/2954976629 wait complete. 2023-10-17T01:27:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.879+0000 7f9b6ffff700 1 Processor -- start 2023-10-17T01:27:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.879+0000 7f9b6ffff700 1 -- start start 2023-10-17T01:27:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b680ffc50 0x7f9b68102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b68102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b681017e0 0x7f9b68102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6ffff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b68103dc0 con 0x7f9b681017e0 2023-10-17T01:27:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b68103f30 con 0x7f9b68100e40 2023-10-17T01:27:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b681040f0 con 0x7f9b680ffc50 2023-10-17T01:27:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b68102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b68102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59934/0 (socket says 172.21.15.45:59934) 2023-10-17T01:27:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6d59a700 1 -- 172.21.15.45:0/1074296603 learned_addr learned my addr 172.21.15.45:0/1074296603 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.880+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b680ffc50 0x7f9b68102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b681017e0 0x7f9b68102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6d59a700 1 -- 172.21.15.45:0/1074296603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b680ffc50 msgr2=0x7f9b68102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b680ffc50 0x7f9b68102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6d59a700 1 -- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b681017e0 msgr2=0x7f9b68102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b681017e0 0x7f9b68102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6d59a700 1 -- 172.21.15.45:0/1074296603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b540095f0 con 0x7f9b68100e40 2023-10-17T01:27:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b681017e0 0x7f9b68102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.881+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b68102760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400bf40 tx=0x7f9b5400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.882+0000 7f9b66ffd700 1 -- 172.21.15.45:0/1074296603 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5401d070 con 0x7f9b68100e40 2023-10-17T01:27:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.882+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6819ab30 con 0x7f9b68100e40 2023-10-17T01:27:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.882+0000 7f9b66ffd700 1 -- 172.21.15.45:0/1074296603 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b54005510 con 0x7f9b68100e40 2023-10-17T01:27:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.882+0000 7f9b66ffd700 1 -- 172.21.15.45:0/1074296603 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54004c00 con 0x7f9b68100e40 2023-10-17T01:27:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.882+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6819b030 con 0x7f9b68100e40 2023-10-17T01:27:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.885+0000 7f9b66ffd700 1 -- 172.21.15.45:0/1074296603 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f9b5400fb80 con 0x7f9b68100e40 2023-10-17T01:27:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.888+0000 7f9b66ffd700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9b580a0ba0 0x7f9b580a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.888+0000 7f9b66ffd700 1 -- 172.21.15.45:0/1074296603 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9b540cd060 con 0x7f9b68100e40 2023-10-17T01:27:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.890+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9b580a0ba0 0x7f9b580a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.890+0000 7f9b838adb80 1 Processor -- start 2023-10-17T01:27:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.890+0000 7f9b838adb80 1 -- start start 2023-10-17T01:27:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 0x5598d07a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5190 0x5598d07a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9b580a0ba0 0x7f9b580a3060 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9b68103b30 tx=0x7f9b5c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b838adb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d06fc9b0 con 0x5598d07a5d90 2023-10-17T01:27:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b838adb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d06fa370 con 0x5598d07a5190 2023-10-17T01:27:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b838adb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d06f5c60 con 0x5598d07a4830 2023-10-17T01:27:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b6dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 0x5598d07a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b6dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 0x5598d07a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40862/0 (socket says 172.21.15.45:40862) 2023-10-17T01:27:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/622966480 learned_addr learned my addr 172.21.15.45:0/622966480 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.891+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5190 0x5598d07a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.892+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/622966480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5190 msgr2=0x5598d07a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.892+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5190 0x5598d07a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.892+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/622966480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 msgr2=0x5598d07a6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.892+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.892+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/622966480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598d0523d80 con 0x5598d07a4830 2023-10-17T01:27:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.892+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 0x5598d07a4c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0176b0 tx=0x7f9b5c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b6776cf3dc01c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.893+0000 7f9b53fff700 1 -- 172.21.15.45:0/622966480 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c00e040 con 0x5598d07a4830 2023-10-17T01:27:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.893+0000 7f9b53fff700 1 -- 172.21.15.45:0/622966480 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b5c00f040 con 0x5598d07a4830 2023-10-17T01:27:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.893+0000 7f9b53fff700 1 -- 172.21.15.45:0/622966480 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c005240 con 0x5598d07a4830 2023-10-17T01:27:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.894+0000 7f9b838adb80 1 -- 172.21.15.45:0/622966480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 msgr2=0x5598d07a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.894+0000 7f9b838adb80 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 0x5598d07a4c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0176b0 tx=0x7f9b5c014000 comp rx=0 tx=0).stop 2023-10-17T01:27:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 -- 172.21.15.45:0/622966480 shutdown_connections 2023-10-17T01:27:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a4830 0x5598d07a4c50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 --2- 172.21.15.45:0/622966480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5190 0x5598d07a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 -- 172.21.15.45:0/622966480 >> 172.21.15.45:0/622966480 conn(0x5598d0704070 msgr2=0x5598d079f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 -- 172.21.15.45:0/622966480 shutdown_connections 2023-10-17T01:27:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.895+0000 7f9b838adb80 1 -- 172.21.15.45:0/622966480 wait complete. 2023-10-17T01:27:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 -- start start 2023-10-17T01:27:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 Processor -- start 2023-10-17T01:27:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d07ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d06924b0 con 0x5598d07a5d90 2023-10-17T01:27:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.896+0000 7f9b838adb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d0692150 con 0x5598d07a4830 2023-10-17T01:27:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b838adb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d05798c0 con 0x5598d07a5190 2023-10-17T01:27:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b838adb80 10 client.0 _ll_register_callbacks cb 0x5598d05dfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d07ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d07ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40868/0 (socket says 172.21.15.45:40868) 2023-10-17T01:27:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 -- 172.21.15.45:0/2799188663 learned_addr learned my addr 172.21.15.45:0/2799188663 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 -- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 msgr2=0x5598d07ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 -- 172.21.15.45:0/2799188663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 msgr2=0x5598d07b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 -- 172.21.15.45:0/2799188663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c017440 con 0x5598d07a5190 2023-10-17T01:27:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.897+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d07ae920 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b54096c10 tx=0x7f9b5409ac50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be77dea4e5979b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b517fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:27:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b517fa700 1 -- 172.21.15.45:0/2799188663 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540a2460 con 0x5598d07a5190 2023-10-17T01:27:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b517fa700 1 -- 172.21.15.45:0/2799188663 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9b5409ae10 con 0x5598d07a5190 2023-10-17T01:27:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598d0720cb0 con 0x5598d07a5190 2023-10-17T01:27:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b517fa700 1 -- 172.21.15.45:0/2799188663 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b5409a710 con 0x5598d07a5190 2023-10-17T01:27:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 10 client.26698 fetch_fsmap learned FSMap version 40 2023-10-17T01:27:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5598d050cc00 con 0x5598d07a5190 2023-10-17T01:27:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b517fa700 1 -- 172.21.15.45:0/2799188663 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b540b8020 con 0x5598d07a5190 2023-10-17T01:27:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b517fa700 1 -- 172.21.15.45:0/2799188663 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f9b540a02f0 con 0x5598d07a5190 2023-10-17T01:27:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 10 client.26698 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:27:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 20 client.26698 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d07b9ec0 0x5598d07bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 4 client.26698 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.898+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598d0773cd0 con 0x5598d07b9ec0 2023-10-17T01:27:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.899+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d07b9ec0 0x5598d07bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.899+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d07b9ec0 0x5598d07bc2b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7f7fdc394ae854 server_cookie=4afe8b27eecb0516 in_seq=0 out_seq=0 2023-10-17T01:27:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.899+0000 7f9b517fa700 10 client.26698 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.900+0000 7f9b517fa700 1 -- 172.21.15.45:0/2799188663 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5598d0773cd0 con 0x5598d07b9ec0 2023-10-17T01:27:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.900+0000 7f9b517fa700 10 client.26698 handle_command_reply: tid=0 2023-10-17T01:27:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.900+0000 7f9b838adb80 1 client.26698 shutdown 2023-10-17T01:27:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.900+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 msgr2=0x5598d07ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.900+0000 7f9b838adb80 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d07ae920 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b54096c10 tx=0x7f9b5409ac50 comp rx=0 tx=0).stop 2023-10-17T01:27:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 shutdown_connections 2023-10-17T01:27:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5d90 0x5598d07b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d07b9ec0 0x5598d07bc2b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 --2- 172.21.15.45:0/2799188663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d07ae920 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 >> 172.21.15.45:0/2799188663 conn(0x5598d0704070 msgr2=0x5598d079ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b517fa700 0 client.26698 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 shutdown_connections 2023-10-17T01:27:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 1 -- 172.21.15.45:0/2799188663 wait complete. 2023-10-17T01:27:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.901+0000 7f9b838adb80 20 client.26698 trim_cache size 0 max 16384 2023-10-17T01:27:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 Processor -- start 2023-10-17T01:27:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 -- start start 2023-10-17T01:27:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5190 0x5598d07ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 0x5598d07ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d07af5d0 con 0x5598d07a5190 2023-10-17T01:27:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d07a05c0 con 0x5598d07a4830 2023-10-17T01:27:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b838adb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d07a89a0 con 0x5598d07a5d90 2023-10-17T01:27:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.924+0000 7f9b6e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 0x5598d07ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 0x5598d07ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40876/0 (socket says 172.21.15.45:40876) 2023-10-17T01:27:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 -- 172.21.15.45:0/551300838 learned_addr learned my addr 172.21.15.45:0/551300838 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5190 0x5598d07ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 -- 172.21.15.45:0/551300838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 msgr2=0x5598d07b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 -- 172.21.15.45:0/551300838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5190 msgr2=0x5598d07ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5190 0x5598d07ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 -- 172.21.15.45:0/551300838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598d054a1f0 con 0x5598d07a5d90 2023-10-17T01:27:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 0x5598d07ae920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000cb20 tx=0x7f9b6000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e242bee28efccf32 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b64ff9700 1 -- 172.21.15.45:0/551300838 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6000ecb0 con 0x5598d07a5d90 2023-10-17T01:27:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.925+0000 7f9b64ff9700 1 -- 172.21.15.45:0/551300838 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b6000ee10 con 0x5598d07a5d90 2023-10-17T01:27:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b64ff9700 1 -- 172.21.15.45:0/551300838 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6000f8c0 con 0x5598d07a5d90 2023-10-17T01:27:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 -- 172.21.15.45:0/551300838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 msgr2=0x5598d07ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 0x5598d07ae920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000cb20 tx=0x7f9b6000ce30 comp rx=0 tx=0).stop 2023-10-17T01:27:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 -- 172.21.15.45:0/551300838 shutdown_connections 2023-10-17T01:27:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5d90 0x5598d07ae920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a4830 0x5598d07b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 --2- 172.21.15.45:0/551300838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a5190 0x5598d07ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 -- 172.21.15.45:0/551300838 >> 172.21.15.45:0/551300838 conn(0x5598d0704070 msgr2=0x5598d07ac2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 -- 172.21.15.45:0/551300838 shutdown_connections 2023-10-17T01:27:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.926+0000 7f9b838adb80 1 -- 172.21.15.45:0/551300838 wait complete. 2023-10-17T01:27:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.927+0000 7f9b838adb80 1 -- start start 2023-10-17T01:27:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.927+0000 7f9b838adb80 1 Processor -- start 2023-10-17T01:27:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.927+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a4830 0x5598d0803a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d0803f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b838adb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5d90 0x5598d080aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b838adb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d080b0e0 con 0x5598d07a4830 2023-10-17T01:27:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b838adb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d080b220 con 0x5598d07a5d90 2023-10-17T01:27:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b838adb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d080b3e0 con 0x5598d07a5190 2023-10-17T01:27:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b838adb80 10 client.0 _ll_register_callbacks cb 0x5598d05dfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6d59a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d0803f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a4830 0x5598d0803a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6d59a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d0803f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40882/0 (socket says 172.21.15.45:40882) 2023-10-17T01:27:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6d59a700 1 -- 172.21.15.45:0/3747159914 learned_addr learned my addr 172.21.15.45:0/3747159914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5d90 0x5598d080aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/3747159914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 msgr2=0x5598d0803f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d0803f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5d90 msgr2=0x5598d080aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5d90 0x5598d080aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.928+0000 7f9b6dd9b700 1 -- 172.21.15.45:0/3747159914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6000c750 con 0x5598d07a4830 2023-10-17T01:27:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.929+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d0803f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.929+0000 7f9b6e59c700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5d90 0x5598d080aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.929+0000 7f9b6dd9b700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a4830 0x5598d0803a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009960 tx=0x7f9b5c016fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7271288cfc9ed9a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.929+0000 7f9b517fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:27:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.930+0000 7f9b517fa700 1 -- 172.21.15.45:0/3747159914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c02f040 con 0x5598d07a4830 2023-10-17T01:27:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.930+0000 7f9b517fa700 1 -- 172.21.15.45:0/3747159914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9b5c01c040 con 0x5598d07a4830 2023-10-17T01:27:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.930+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b5c00f040 con 0x5598d07a4830 2023-10-17T01:27:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.931+0000 7f9b517fa700 1 -- 172.21.15.45:0/3747159914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b5c00e040 con 0x5598d07a4830 2023-10-17T01:27:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.931+0000 7f9b838adb80 10 client.16413 fetch_fsmap learned FSMap version 40 2023-10-17T01:27:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.931+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5598d0522250 con 0x5598d07a4830 2023-10-17T01:27:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.932+0000 7f9b517fa700 1 -- 172.21.15.45:0/3747159914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b5c00f040 con 0x5598d07a4830 2023-10-17T01:27:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.932+0000 7f9b517fa700 1 -- 172.21.15.45:0/3747159914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f9b5c027020 con 0x5598d07a4830 2023-10-17T01:27:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.932+0000 7f9b838adb80 10 client.16413 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:27:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.932+0000 7f9b838adb80 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.932+0000 7f9b838adb80 20 client.16413 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.933+0000 7f9b838adb80 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d080cc70 0x5598d080f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.933+0000 7f9b838adb80 4 client.16413 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:27:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.933+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598d06eece0 con 0x5598d080cc70 2023-10-17T01:27:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.933+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d080cc70 0x5598d080f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.934+0000 7f9b6d59a700 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d080cc70 0x5598d080f090 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0db4b4297168ca server_cookie=af4b42962ed9d63 in_seq=0 out_seq=0 2023-10-17T01:27:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:04.934+0000 7f9b517fa700 10 client.16413 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.008+0000 7f9b517fa700 1 -- 172.21.15.45:0/3747159914 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598d06eece0 con 0x5598d080cc70 2023-10-17T01:27:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.008+0000 7f9b517fa700 10 client.16413 handle_command_reply: tid=0 2023-10-17T01:27:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.008+0000 7f9b838adb80 1 client.16413 shutdown 2023-10-17T01:27:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a4830 msgr2=0x5598d0803a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a4830 0x5598d0803a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009960 tx=0x7f9b5c016fd0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 shutdown_connections 2023-10-17T01:27:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5598d080cc70 0x5598d080f090 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5598d07a5d90 0x5598d080aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598d07a4830 0x5598d0803a30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b517fa700 0 client.16413 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 --2- 172.21.15.45:0/3747159914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d07a5190 0x5598d0803f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 >> 172.21.15.45:0/3747159914 conn(0x5598d0704070 msgr2=0x5598d07ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 shutdown_connections 2023-10-17T01:27:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.010+0000 7f9b838adb80 1 -- 172.21.15.45:0/3747159914 wait complete. 2023-10-17T01:27:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:27:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:27:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:27:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:27:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.013+0000 7f9b838adb80 20 client.16413 trim_cache size 0 max 16384 2023-10-17T01:27:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9b580a0ba0 msgr2=0x7f9b580a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9b580a0ba0 0x7f9b580a3060 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9b68103b30 tx=0x7f9b5c00b040 comp rx=0 tx=0).stop 2023-10-17T01:27:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 msgr2=0x7f9b68102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b68102760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400bf40 tx=0x7f9b5400f690 comp rx=0 tx=0).stop 2023-10-17T01:27:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 shutdown_connections 2023-10-17T01:27:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f9b580a0ba0 0x7f9b580a3060 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b681017e0 0x7f9b68102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b680ffc50 0x7f9b68102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 --2- 172.21.15.45:0/1074296603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b68100e40 0x7f9b68102760 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 >> 172.21.15.45:0/1074296603 conn(0x7f9b680fb1a0 msgr2=0x7f9b68068140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 shutdown_connections 2023-10-17T01:27:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:05.014+0000 7f9b6ffff700 1 -- 172.21.15.45:0/1074296603 wait complete. 2023-10-17T01:27:05.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:04 smithi181 ceph-mon[134520]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 MiB/s wr, 789 op/s 2023-10-17T01:27:05.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:04 smithi045 ceph-mon[98474]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 MiB/s wr, 789 op/s 2023-10-17T01:27:05.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:04 smithi060 ceph-mon[134136]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 MiB/s wr, 789 op/s 2023-10-17T01:27:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 715.0 s, 1644.0 tps, lat 60.140 ms stddev 71.703 2023-10-17T01:27:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:06 smithi181 ceph-mon[134520]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 613 op/s 2023-10-17T01:27:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:06 smithi045 ceph-mon[98474]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 613 op/s 2023-10-17T01:27:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:06 smithi060 ceph-mon[134136]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 613 op/s 2023-10-17T01:27:07.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:27:07.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 -- 172.21.15.45:0/3704570500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac108c40 msgr2=0x7ff1ac10b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/3704570500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac108c40 0x7ff1ac10b0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b0a0 tx=0x7ff1a400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 -- 172.21.15.45:0/3704570500 shutdown_connections 2023-10-17T01:27:07.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/3704570500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac108c40 0x7ff1ac10b0b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/3704570500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1ac074d30 0x7ff1ac073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/3704570500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ac073750 0x7ff1ac073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 -- 172.21.15.45:0/3704570500 >> 172.21.15.45:0/3704570500 conn(0x7ff1ac0fc300 msgr2=0x7ff1ac0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:07.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 -- 172.21.15.45:0/3704570500 shutdown_connections 2023-10-17T01:27:07.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.915+0000 7ff1b35ff700 1 -- 172.21.15.45:0/3704570500 wait complete. 2023-10-17T01:27:07.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.917+0000 7ff1b35ff700 1 Processor -- start 2023-10-17T01:27:07.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.917+0000 7ff1b35ff700 1 -- start start 2023-10-17T01:27:07.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b35ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ac073750 0x7ff1ac198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b35ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 0x7ff1ac199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b35ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1ac108c40 0x7ff1ac1a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b35ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1a0900 con 0x7ff1ac073750 2023-10-17T01:27:07.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b35ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1a0a70 con 0x7ff1ac108c40 2023-10-17T01:27:07.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b35ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1a0c30 con 0x7ff1ac074d30 2023-10-17T01:27:07.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.918+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 0x7ff1ac199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.919+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 0x7ff1ac199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45798/0 (socket says 172.21.15.45:45798) 2023-10-17T01:27:07.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.919+0000 7ff1b139b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ac073750 0x7ff1ac198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.919+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/2008019185 learned_addr learned my addr 172.21.15.45:0/2008019185 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.919+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/2008019185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1ac108c40 msgr2=0x7ff1ac1a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.919+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1ac108c40 0x7ff1ac1a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.920+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ac073750 msgr2=0x7ff1ac198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.920+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ac073750 0x7ff1ac198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.920+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/2008019185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4009d00 con 0x7ff1ac074d30 2023-10-17T01:27:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.920+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 0x7ff1ac199300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00d900 tx=0x7ff19c00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.921+0000 7ff1a27fc700 1 -- 172.21.15.45:0/2008019185 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c004a30 con 0x7ff1ac074d30 2023-10-17T01:27:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.921+0000 7ff1a27fc700 1 -- 172.21.15.45:0/2008019185 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0054e0 con 0x7ff1ac074d30 2023-10-17T01:27:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.921+0000 7ff1a27fc700 1 -- 172.21.15.45:0/2008019185 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c009bc0 con 0x7ff1ac074d30 2023-10-17T01:27:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.921+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ac1a0f10 con 0x7ff1ac074d30 2023-10-17T01:27:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.921+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac1a1310 con 0x7ff1ac074d30 2023-10-17T01:27:07.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.926+0000 7ff1a27fc700 1 -- 172.21.15.45:0/2008019185 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff19c009d20 con 0x7ff1ac074d30 2023-10-17T01:27:07.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.928+0000 7ff1c6f66b80 1 Processor -- start 2023-10-17T01:27:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.929+0000 7ff1c6f66b80 1 -- start start 2023-10-17T01:27:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.929+0000 7ff1a27fc700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1980a0bf0 0x7ff1980a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.929+0000 7ff1a27fc700 1 -- 172.21.15.45:0/2008019185 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff19c0cbf80 con 0x7ff1ac074d30 2023-10-17T01:27:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.930+0000 7ff1b1b9c700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1980a0bf0 0x7ff1980a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e8ef820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 0x56515e8f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e8f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e83ea10 con 0x56515e8ef400 2023-10-17T01:27:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e83c3d0 con 0x56515e8f0ae0 2023-10-17T01:27:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e837cc0 con 0x56515e8efd60 2023-10-17T01:27:07.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1b1b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e8f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 0x56515e8f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 0x56515e8f0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:45810/0 (socket says 172.21.15.45:45810) 2023-10-17T01:27:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.931+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3881514300 learned_addr learned my addr 172.21.15.45:0/3881514300 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b139b700 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e8ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3881514300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 msgr2=0x56515e8f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e8f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3881514300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 msgr2=0x56515e8ef820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e8ef820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3881514300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56515e665de0 con 0x56515e8efd60 2023-10-17T01:27:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.932+0000 7ff1b1b9c700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1980a0bf0 0x7ff1980a30b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4007ab0 tx=0x7ff1a4007590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.933+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 0x56515e8f0500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0a10e0 tx=0x7ff19c099020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e5c9d0f0b2670e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.933+0000 7ff1977fe700 1 -- 172.21.15.45:0/3881514300 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c09d3d0 con 0x56515e8efd60 2023-10-17T01:27:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.933+0000 7ff1977fe700 1 -- 172.21.15.45:0/3881514300 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0a1610 con 0x56515e8efd60 2023-10-17T01:27:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.933+0000 7ff1977fe700 1 -- 172.21.15.45:0/3881514300 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0840c0 con 0x56515e8efd60 2023-10-17T01:27:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3881514300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 msgr2=0x56515e8f0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 0x56515e8f0500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0a10e0 tx=0x7ff19c099020 comp rx=0 tx=0).stop 2023-10-17T01:27:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3881514300 shutdown_connections 2023-10-17T01:27:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e8f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e8ef820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3881514300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8efd60 0x56515e8f0500 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3881514300 >> 172.21.15.45:0/3881514300 conn(0x56515e8460d0 msgr2=0x56515e8e1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:07.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3881514300 shutdown_connections 2023-10-17T01:27:07.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.934+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3881514300 wait complete. 2023-10-17T01:27:07.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.935+0000 7ff1c6f66b80 1 -- start start 2023-10-17T01:27:07.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.935+0000 7ff1c6f66b80 1 Processor -- start 2023-10-17T01:27:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8ef400 0x56515e8f9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e8f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e9005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e7d4510 con 0x56515e8efd60 2023-10-17T01:27:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e7d41b0 con 0x56515e8f0ae0 2023-10-17T01:27:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e6bb920 con 0x56515e8ef400 2023-10-17T01:27:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.936+0000 7ff1c6f66b80 10 client.0 _ll_register_callbacks cb 0x56515e721a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e9005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e9005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48578/0 (socket says 172.21.15.45:48578) 2023-10-17T01:27:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 -- 172.21.15.45:0/2649366845 learned_addr learned my addr 172.21.15.45:0/2649366845 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b139b700 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8ef400 0x56515e8f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 -- 172.21.15.45:0/2649366845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8ef400 msgr2=0x56515e8f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8ef400 0x56515e8f9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 -- 172.21.15.45:0/2649366845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 msgr2=0x56515e8f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e8f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.937+0000 7ff1b1b9c700 1 -- 172.21.15.45:0/2649366845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac1a1310 con 0x56515e8f0ae0 2023-10-17T01:27:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.938+0000 7ff1b1b9c700 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e9005f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4003cb0 tx=0x7ff1a401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81a9ba4d55936bb server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.938+0000 7ff194ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:27:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.938+0000 7ff194ff9700 1 -- 172.21.15.45:0/2649366845 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400e070 con 0x56515e8f0ae0 2023-10-17T01:27:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.938+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56515e862d10 con 0x56515e8f0ae0 2023-10-17T01:27:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.938+0000 7ff194ff9700 1 -- 172.21.15.45:0/2649366845 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1a4003e40 con 0x56515e8f0ae0 2023-10-17T01:27:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.938+0000 7ff194ff9700 1 -- 172.21.15.45:0/2649366845 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a40078e0 con 0x56515e8f0ae0 2023-10-17T01:27:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.939+0000 7ff1c6f66b80 10 client.26756 fetch_fsmap learned FSMap version 40 2023-10-17T01:27:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.939+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56515e64ec60 con 0x56515e8f0ae0 2023-10-17T01:27:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.939+0000 7ff194ff9700 1 -- 172.21.15.45:0/2649366845 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56515e862d10 con 0x56515e8f0ae0 2023-10-17T01:27:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.941+0000 7ff194ff9700 1 -- 172.21.15.45:0/2649366845 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff1a4028b80 con 0x56515e8f0ae0 2023-10-17T01:27:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1c6f66b80 10 client.26756 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:27:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1c6f66b80 10 client.26756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1c6f66b80 20 client.26756 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e904f40 0x56515e907360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1c6f66b80 4 client.26756 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56515e8b5d30 con 0x56515e904f40 2023-10-17T01:27:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.942+0000 7ff1b139b700 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e904f40 0x56515e907360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.943+0000 7ff1b139b700 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e904f40 0x56515e907360 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2598e4c89b456552 server_cookie=e6edc464ec3e3335 in_seq=0 out_seq=0 2023-10-17T01:27:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.943+0000 7ff194ff9700 10 client.26756 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.944+0000 7ff194ff9700 1 -- 172.21.15.45:0/2649366845 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x56515e8b5d30 con 0x56515e904f40 2023-10-17T01:27:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.944+0000 7ff194ff9700 10 client.26756 handle_command_reply: tid=0 2023-10-17T01:27:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.944+0000 7ff1c6f66b80 1 client.26756 shutdown 2023-10-17T01:27:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 msgr2=0x56515e9005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e9005f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4003cb0 tx=0x7ff1a401d000 comp rx=0 tx=0).stop 2023-10-17T01:27:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 shutdown_connections 2023-10-17T01:27:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e904f40 0x56515e907360 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8f0ae0 0x56515e9005f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8ef400 0x56515e8f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2649366845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e8f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 >> 172.21.15.45:0/2649366845 conn(0x56515e8460d0 msgr2=0x56515e8e1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff194ff9700 0 client.26756 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 shutdown_connections 2023-10-17T01:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.945+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2649366845 wait complete. 2023-10-17T01:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.946+0000 7ff1c6f66b80 20 client.26756 trim_cache size 0 max 16384 2023-10-17T01:27:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 Processor -- start 2023-10-17T01:27:07.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 -- start start 2023-10-17T01:27:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 0x56515e8f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e900dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e8f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e8e2550 con 0x56515e8efd60 2023-10-17T01:27:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e8f7ae0 con 0x56515e8ef400 2023-10-17T01:27:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e8f0960 con 0x56515e8f0ae0 2023-10-17T01:27:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1b139b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 0x56515e8f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1b1b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e8f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1b139b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 0x56515e8f0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48590/0 (socket says 172.21.15.45:48590) 2023-10-17T01:27:07.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1b139b700 1 -- 172.21.15.45:0/2089524010 learned_addr learned my addr 172.21.15.45:0/2089524010 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.968+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e900dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.969+0000 7ff1b139b700 1 -- 172.21.15.45:0/2089524010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 msgr2=0x56515e8f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.969+0000 7ff1b139b700 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e8f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.969+0000 7ff1b139b700 1 -- 172.21.15.45:0/2089524010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 msgr2=0x56515e900dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.970+0000 7ff1b139b700 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e900dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.970+0000 7ff1b139b700 1 -- 172.21.15.45:0/2089524010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56515e68c250 con 0x56515e8ef400 2023-10-17T01:27:07.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.970+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e900dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:07.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.971+0000 7ff1b139b700 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 0x56515e8f0ec0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x56515e907c40 tx=0x7ff1a800b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ac9e5f04f0fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:07.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.972+0000 7ff197fff700 1 -- 172.21.15.45:0/2089524010 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8017040 con 0x56515e8ef400 2023-10-17T01:27:07.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.972+0000 7ff197fff700 1 -- 172.21.15.45:0/2089524010 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a8019370 con 0x56515e8ef400 2023-10-17T01:27:07.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.972+0000 7ff197fff700 1 -- 172.21.15.45:0/2089524010 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8003e10 con 0x56515e8ef400 2023-10-17T01:27:07.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2089524010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 msgr2=0x56515e8f0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 0x56515e8f0ec0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x56515e907c40 tx=0x7ff1a800b3c0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2089524010 shutdown_connections 2023-10-17T01:27:07.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e8f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8ef400 0x56515e8f0ec0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/2089524010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8efd60 0x56515e900dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2089524010 >> 172.21.15.45:0/2089524010 conn(0x56515e8460d0 msgr2=0x56515e8f7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:07.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2089524010 shutdown_connections 2023-10-17T01:27:07.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/2089524010 wait complete. 2023-10-17T01:27:07.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.973+0000 7ff1c6f66b80 1 -- start start 2023-10-17T01:27:07.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 Processor -- start 2023-10-17T01:27:07.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e94eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 0x56515e94f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e955eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e9563f0 con 0x56515e8ef400 2023-10-17T01:27:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e956530 con 0x56515e8efd60 2023-10-17T01:27:07.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515e9566f0 con 0x56515e8f0ae0 2023-10-17T01:27:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.974+0000 7ff1c6f66b80 10 client.0 _ll_register_callbacks cb 0x7ff1a40093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 0x56515e94f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 0x56515e94f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48606/0 (socket says 172.21.15.45:48606) 2023-10-17T01:27:07.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3170175501 learned_addr learned my addr 172.21.15.45:0/3170175501 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:07.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b1b9c700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e955eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b139b700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e94eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3170175501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 msgr2=0x56515e955eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e955eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3170175501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 msgr2=0x56515e94eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:07.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e94eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b0b9a700 1 -- 172.21.15.45:0/3170175501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56515e665de0 con 0x56515e8efd60 2023-10-17T01:27:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.975+0000 7ff1b139b700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e94eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff1b0b9a700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 0x56515e94f020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff19c01d040 tx=0x7ff19c095220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b357b33db8154f13 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff194ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:27:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff194ff9700 1 -- 172.21.15.45:0/3170175501 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c097940 con 0x56515e8efd60 2023-10-17T01:27:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff194ff9700 1 -- 172.21.15.45:0/3170175501 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff19c005650 con 0x56515e8efd60 2023-10-17T01:27:07.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56515e830d40 con 0x56515e8efd60 2023-10-17T01:27:07.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff194ff9700 1 -- 172.21.15.45:0/3170175501 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19c099800 con 0x56515e8efd60 2023-10-17T01:27:07.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff1c6f66b80 10 client.26762 fetch_fsmap learned FSMap version 40 2023-10-17T01:27:07.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7ff1a800b700 con 0x56515e8efd60 2023-10-17T01:27:07.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff194ff9700 1 -- 172.21.15.45:0/3170175501 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0a5080 con 0x56515e8efd60 2023-10-17T01:27:07.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.976+0000 7ff194ff9700 1 -- 172.21.15.45:0/3170175501 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 40) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff19c084020 con 0x56515e8efd60 2023-10-17T01:27:07.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1c6f66b80 10 client.26762 fetch_fsmap finished waiting for FSMap version 40 2023-10-17T01:27:07.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1c6f66b80 10 client.26762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1c6f66b80 20 client.26762 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e957de0 0x56515e95a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1c6f66b80 4 client.26762 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:27:07.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56515e830a60 con 0x56515e957de0 2023-10-17T01:27:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1b139b700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e957de0 0x56515e95a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff1b139b700 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e957de0 0x56515e95a200 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98484b0d0376797 server_cookie=d99061ff45e58203 in_seq=0 out_seq=0 2023-10-17T01:27:07.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.977+0000 7ff194ff9700 10 client.26762 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.978+0000 7ff194ff9700 1 -- 172.21.15.45:0/3170175501 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56515e830a60 con 0x56515e957de0 2023-10-17T01:27:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.978+0000 7ff194ff9700 10 client.26762 handle_command_reply: tid=0 2023-10-17T01:27:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.979+0000 7ff1c6f66b80 1 client.26762 shutdown 2023-10-17T01:27:07.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.979+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 msgr2=0x56515e94f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.979+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 0x56515e94f020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff19c01d040 tx=0x7ff19c095220 comp rx=0 tx=0).stop 2023-10-17T01:27:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.981+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 shutdown_connections 2023-10-17T01:27:07.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.981+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56515e8f0ae0 0x56515e955eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.981+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56515e8ef400 0x56515e94eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.981+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56515e957de0 0x56515e95a200 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.982+0000 7ff1c6f66b80 1 --2- 172.21.15.45:0/3170175501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56515e8efd60 0x56515e94f020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.982+0000 7ff194ff9700 0 client.26762 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.982+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 >> 172.21.15.45:0/3170175501 conn(0x56515e8460d0 msgr2=0x56515e900dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:07.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.982+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 shutdown_connections 2023-10-17T01:27:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.982+0000 7ff1c6f66b80 1 -- 172.21.15.45:0/3170175501 wait complete. 2023-10-17T01:27:07.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.982+0000 7ff1c6f66b80 20 client.26762 trim_cache size 0 max 16384 2023-10-17T01:27:08.000 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:27:08.000 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-17T01:27:08.001 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:27:08.001 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:27:08.001 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:27:08.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:27:08.002 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:27:08.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:27:08.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:27:08.003 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:27:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1980a0bf0 msgr2=0x7ff1980a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1980a0bf0 0x7ff1980a30b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4007ab0 tx=0x7ff1a4007590 comp rx=0 tx=0).stop 2023-10-17T01:27:08.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 msgr2=0x7ff1ac199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 0x7ff1ac199300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00d900 tx=0x7ff19c00dc10 comp rx=0 tx=0).stop 2023-10-17T01:27:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 shutdown_connections 2023-10-17T01:27:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1980a0bf0 0x7ff1980a30b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1ac108c40 0x7ff1ac1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:08.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ac073750 0x7ff1ac198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 --2- 172.21.15.45:0/2008019185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1ac074d30 0x7ff1ac199300 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 >> 172.21.15.45:0/2008019185 conn(0x7ff1ac0fc300 msgr2=0x7ff1ac0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:08.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 shutdown_connections 2023-10-17T01:27:08.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:07.983+0000 7ff1b35ff700 1 -- 172.21.15.45:0/2008019185 wait complete. 2023-10-17T01:27:08.013 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:27:09.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:08 smithi181 ceph-mon[134520]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 648 op/s 2023-10-17T01:27:09.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:08 smithi045 ceph-mon[98474]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 648 op/s 2023-10-17T01:27:09.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:08 smithi060 ceph-mon[134136]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 648 op/s 2023-10-17T01:27:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 720.0 s, 1476.8 tps, lat 68.284 ms stddev 83.392 2023-10-17T01:27:11.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:10 smithi181 ceph-mon[134520]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 713 op/s 2023-10-17T01:27:11.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:10 smithi045 ceph-mon[98474]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 713 op/s 2023-10-17T01:27:11.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:10 smithi060 ceph-mon[134136]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 713 op/s 2023-10-17T01:27:12.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:11 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:27:12.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:11 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:27:12.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:11 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:27:13.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:12 smithi181 ceph-mon[134520]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2023-10-17T01:27:13.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:12 smithi045 ceph-mon[98474]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2023-10-17T01:27:13.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:12 smithi060 ceph-mon[134136]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 602 op/s 2023-10-17T01:27:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:14 smithi181 ceph-mon[134520]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 784 op/s 2023-10-17T01:27:15.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:14 smithi045 ceph-mon[98474]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 784 op/s 2023-10-17T01:27:15.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:14 smithi060 ceph-mon[134136]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 784 op/s 2023-10-17T01:27:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 725.0 s, 955.6 tps, lat 104.434 ms stddev 122.160 2023-10-17T01:27:17.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:16 smithi045 ceph-mon[98474]: scrub summary: active paths [/] 2023-10-17T01:27:17.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:16 smithi045 ceph-mon[98474]: scrub summary: idle+waiting paths [/] 2023-10-17T01:27:17.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:16 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:27:17.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:16 smithi045 ceph-mon[98474]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.0 MiB/s wr, 620 op/s 2023-10-17T01:27:17.338 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:16 smithi060 ceph-mon[134136]: scrub summary: active paths [/] 2023-10-17T01:27:17.338 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:16 smithi060 ceph-mon[134136]: scrub summary: idle+waiting paths [/] 2023-10-17T01:27:17.338 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:16 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:27:17.339 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:16 smithi060 ceph-mon[134136]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.0 MiB/s wr, 620 op/s 2023-10-17T01:27:17.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:16 smithi181 ceph-mon[134520]: scrub summary: active paths [/] 2023-10-17T01:27:17.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:16 smithi181 ceph-mon[134520]: scrub summary: idle+waiting paths [/] 2023-10-17T01:27:17.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:16 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:27:17.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:16 smithi181 ceph-mon[134520]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.0 MiB/s wr, 620 op/s 2023-10-17T01:27:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 -- 172.21.15.45:0/1895162723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 msgr2=0x7f09c0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 --2- 172.21.15.45:0/1895162723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0108ea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09b800b0a0 tx=0x7f09b800b3b0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 -- 172.21.15.45:0/1895162723 shutdown_connections 2023-10-17T01:27:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 --2- 172.21.15.45:0/1895162723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0108ea0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 --2- 172.21.15.45:0/1895162723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 0x7f09c0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 --2- 172.21.15.45:0/1895162723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 0x7f09c01064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 -- 172.21.15.45:0/1895162723 >> 172.21.15.45:0/1895162723 conn(0x7f09c00fb1a0 msgr2=0x7f09c00fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 -- 172.21.15.45:0/1895162723 shutdown_connections 2023-10-17T01:27:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.256+0000 7f09c67ff700 1 -- 172.21.15.45:0/1895162723 wait complete. 2023-10-17T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.258+0000 7f09c67ff700 1 Processor -- start 2023-10-17T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.258+0000 7f09c67ff700 1 -- start start 2023-10-17T01:27:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c67ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 0x7f09c00ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c67ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 0x7f09c00ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c67ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0100200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c67ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a0b80 con 0x7f09c0104100 2023-10-17T01:27:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c67ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a0cc0 con 0x7f09c01017d0 2023-10-17T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c67ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a0e00 con 0x7f09c0106a30 2023-10-17T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 0x7f09c00ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09c4d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 0x7f09c00ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42872/0 (socket says 172.21.15.45:42872) 2023-10-17T01:27:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09bffff700 1 -- 172.21.15.45:0/887385057 learned_addr learned my addr 172.21.15.45:0/887385057 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.259+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 0x7f09c00ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c4d9c700 1 -- 172.21.15.45:0/887385057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 msgr2=0x7f09c00ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 0x7f09c00ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c4d9c700 1 -- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 msgr2=0x7f09c00ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 0x7f09c00ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c4d9c700 1 -- 172.21.15.45:0/887385057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8009d00 con 0x7f09c0106a30 2023-10-17T01:27:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 0x7f09c00ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0100200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009fd0 tx=0x7f09b8007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09bd7fa700 1 -- 172.21.15.45:0/887385057 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800e040 con 0x7f09c0106a30 2023-10-17T01:27:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09bd7fa700 1 -- 172.21.15.45:0/887385057 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b8004c50 con 0x7f09c0106a30 2023-10-17T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09bd7fa700 1 -- 172.21.15.45:0/887385057 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80052a0 con 0x7f09c0106a30 2023-10-17T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.260+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c01a1030 con 0x7f09c0106a30 2023-10-17T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.261+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01a1460 con 0x7f09c0106a30 2023-10-17T01:27:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.264+0000 7f09bd7fa700 1 -- 172.21.15.45:0/887385057 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f09b8005610 con 0x7f09c0106a30 2023-10-17T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.265+0000 7f09bd7fa700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09a80a0ba0 0x7f09a80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.265+0000 7f09bd7fa700 1 -- 172.21.15.45:0/887385057 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f09b80cda10 con 0x7f09c0106a30 2023-10-17T01:27:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09bffff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09a80a0ba0 0x7f09a80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 Processor -- start 2023-10-17T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 -- start start 2023-10-17T01:27:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 0x55bb89db8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89db93e0 0x55bb89db9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89d089b0 con 0x55bb89db93e0 2023-10-17T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89d06370 con 0x55bb89dba1b0 2023-10-17T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.266+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89d01c60 con 0x55bb89dbab10 2023-10-17T01:27:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 0x55bb89db8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 0x55bb89db8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42124/0 (socket says 172.21.15.45:42124) 2023-10-17T01:27:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bf7fe700 1 -- 172.21.15.45:0/2387644723 learned_addr learned my addr 172.21.15.45:0/2387644723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bf7fe700 1 -- 172.21.15.45:0/2387644723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 msgr2=0x55bb89dba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bffff700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09bf7fe700 1 -- 172.21.15.45:0/2387644723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89db93e0 msgr2=0x55bb89db9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.267+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89db93e0 0x55bb89db9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.268+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89db93e0 0x55bb89db9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.268+0000 7f09bf7fe700 1 -- 172.21.15.45:0/2387644723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb89b2fd80 con 0x55bb89dbab10 2023-10-17T01:27:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.268+0000 7f09bffff700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.268+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89db93e0 0x55bb89db9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.268+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 0x55bb89db8e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009cb0 tx=0x7f09b4010370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187e8c4b565f8b1a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.269+0000 7f09a67fc700 1 -- 172.21.15.45:0/2387644723 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4010ca0 con 0x55bb89dbab10 2023-10-17T01:27:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.270+0000 7f09a67fc700 1 -- 172.21.15.45:0/2387644723 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b4010e00 con 0x55bb89dbab10 2023-10-17T01:27:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.270+0000 7f09a67fc700 1 -- 172.21.15.45:0/2387644723 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011800 con 0x55bb89dbab10 2023-10-17T01:27:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.270+0000 7f09bffff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09a80a0ba0 0x7f09a80a3060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09c0101090 tx=0x7f09b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.271+0000 7f09da0e3b80 1 -- 172.21.15.45:0/2387644723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 msgr2=0x55bb89db8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.271+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 0x55bb89db8e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009cb0 tx=0x7f09b4010370 comp rx=0 tx=0).stop 2023-10-17T01:27:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 -- 172.21.15.45:0/2387644723 shutdown_connections 2023-10-17T01:27:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89db93e0 0x55bb89db9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/2387644723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89dbab10 0x55bb89db8e00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 -- 172.21.15.45:0/2387644723 >> 172.21.15.45:0/2387644723 conn(0x55bb89d10070 msgr2=0x55bb89dab2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 -- 172.21.15.45:0/2387644723 shutdown_connections 2023-10-17T01:27:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 -- 172.21.15.45:0/2387644723 wait complete. 2023-10-17T01:27:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.272+0000 7f09da0e3b80 1 -- start start 2023-10-17T01:27:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 Processor -- start 2023-10-17T01:27:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89dc2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dc3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89dca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89c9e4b0 con 0x55bb89dbab10 2023-10-17T01:27:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89c9e150 con 0x55bb89dba1b0 2023-10-17T01:27:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89b858c0 con 0x55bb89db93e0 2023-10-17T01:27:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09da0e3b80 10 client.0 _ll_register_callbacks cb 0x55bb89beba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.273+0000 7f09bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89dc2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89dc2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42140/0 (socket says 172.21.15.45:42140) 2023-10-17T01:27:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 -- 172.21.15.45:0/3647447765 learned_addr learned my addr 172.21.15.45:0/3647447765 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 -- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 msgr2=0x55bb89dc3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dc3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89dca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 -- 172.21.15.45:0/3647447765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 msgr2=0x55bb89dca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89dca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 -- 172.21.15.45:0/3647447765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b40098e0 con 0x55bb89db93e0 2023-10-17T01:27:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89dca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f09bffff700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89dc2d00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f09b001dcf0 tx=0x7f09b00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf1fa55290c2664b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.274+0000 7f099f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:27:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f099f7fe700 1 -- 172.21.15.45:0/3647447765 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b001b900 con 0x55bb89db93e0 2023-10-17T01:27:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f099f7fe700 1 -- 172.21.15.45:0/3647447765 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f09b001ba60 con 0x55bb89db93e0 2023-10-17T01:27:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb89d2ccb0 con 0x55bb89db93e0 2023-10-17T01:27:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f099f7fe700 1 -- 172.21.15.45:0/3647447765 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b001bc20 con 0x55bb89db93e0 2023-10-17T01:27:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f09da0e3b80 10 client.26734 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bb89b18c00 con 0x55bb89db93e0 2023-10-17T01:27:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f099f7fe700 1 -- 172.21.15.45:0/3647447765 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb89d2ccb0 con 0x55bb89db93e0 2023-10-17T01:27:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.275+0000 7f099f7fe700 1 -- 172.21.15.45:0/3647447765 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f09b001c430 con 0x55bb89db93e0 2023-10-17T01:27:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09da0e3b80 10 client.26734 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09da0e3b80 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09da0e3b80 20 client.26734 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89dcec80 0x55bb89dd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09da0e3b80 4 client.26734 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb89d7fcd0 con 0x55bb89dcec80 2023-10-17T01:27:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.276+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89dcec80 0x55bb89dd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.277+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89dcec80 0x55bb89dd10a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991ab9f6eee58efd server_cookie=92da17878d1785d8 in_seq=0 out_seq=0 2023-10-17T01:27:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.277+0000 7f099f7fe700 10 client.26734 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.277+0000 7f099f7fe700 1 -- 172.21.15.45:0/3647447765 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55bb89d7fcd0 con 0x55bb89dcec80 2023-10-17T01:27:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.277+0000 7f099f7fe700 10 client.26734 handle_command_reply: tid=0 2023-10-17T01:27:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.277+0000 7f09da0e3b80 1 client.26734 shutdown 2023-10-17T01:27:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 msgr2=0x55bb89dc2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89dc2d00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f09b001dcf0 tx=0x7f09b00047c0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 shutdown_connections 2023-10-17T01:27:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89dca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89dc2d00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89dcec80 0x55bb89dd10a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/3647447765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89dc3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 >> 172.21.15.45:0/3647447765 conn(0x55bb89d10070 msgr2=0x55bb89daaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f099f7fe700 0 client.26734 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 shutdown_connections 2023-10-17T01:27:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.278+0000 7f09da0e3b80 1 -- 172.21.15.45:0/3647447765 wait complete. 2023-10-17T01:27:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.279+0000 7f09da0e3b80 20 client.26734 trim_cache size 0 max 16384 2023-10-17T01:27:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 Processor -- start 2023-10-17T01:27:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 -- start start 2023-10-17T01:27:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89db8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dba1b0 0x55bb89dc2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 0x55bb89dc2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89da8010 con 0x55bb89dba1b0 2023-10-17T01:27:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.299+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89dcbea0 con 0x55bb89dbab10 2023-10-17T01:27:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.300+0000 7f09da0e3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89dc16a0 con 0x55bb89db93e0 2023-10-17T01:27:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.300+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dba1b0 0x55bb89dc2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.300+0000 7f09c4d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 0x55bb89dc2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.300+0000 7f09c4d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 0x55bb89dc2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42906/0 (socket says 172.21.15.45:42906) 2023-10-17T01:27:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.300+0000 7f09c4d9c700 1 -- 172.21.15.45:0/1081905269 learned_addr learned my addr 172.21.15.45:0/1081905269 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.301+0000 7f09bffff700 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89db8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.301+0000 7f09c4d9c700 1 -- 172.21.15.45:0/1081905269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 msgr2=0x55bb89db8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.301+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89db8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.301+0000 7f09c4d9c700 1 -- 172.21.15.45:0/1081905269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dba1b0 msgr2=0x55bb89dc2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.301+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dba1b0 0x55bb89dc2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.302+0000 7f09c4d9c700 1 -- 172.21.15.45:0/1081905269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb89b561f0 con 0x55bb89dbab10 2023-10-17T01:27:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.302+0000 7f09bffff700 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89db8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.302+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 0x55bb89dc2ff0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09b809adc0 tx=0x7f09b80097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f2abd084130faa server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.302+0000 7f09a6ffd700 1 -- 172.21.15.45:0/1081905269 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8004dc0 con 0x55bb89dbab10 2023-10-17T01:27:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.303+0000 7f09a6ffd700 1 -- 172.21.15.45:0/1081905269 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b809b0c0 con 0x55bb89dbab10 2023-10-17T01:27:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.303+0000 7f09a6ffd700 1 -- 172.21.15.45:0/1081905269 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8004dc0 con 0x55bb89dbab10 2023-10-17T01:27:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1081905269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 msgr2=0x55bb89dc2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 0x55bb89dc2ff0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09b809adc0 tx=0x7f09b80097e0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1081905269 shutdown_connections 2023-10-17T01:27:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dbab10 0x55bb89dc2ff0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89db8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1081905269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dba1b0 0x55bb89dc2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.304+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1081905269 >> 172.21.15.45:0/1081905269 conn(0x55bb89d10070 msgr2=0x55bb89dbd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.305+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1081905269 shutdown_connections 2023-10-17T01:27:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.305+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1081905269 wait complete. 2023-10-17T01:27:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.305+0000 7f09da0e3b80 1 -- start start 2023-10-17T01:27:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.306+0000 7f09da0e3b80 1 Processor -- start 2023-10-17T01:27:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.306+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89e1c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.306+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89e1cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.306+0000 7f09da0e3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89e1d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89e1c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89e1c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:42150/0 (socket says 172.21.15.45:42150) 2023-10-17T01:27:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89e1cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89e1cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:42916/0 (socket says 172.21.15.45:42916) 2023-10-17T01:27:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 -- 172.21.15.45:0/1742029873 learned_addr learned my addr 172.21.15.45:0/1742029873 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89e1e440 con 0x55bb89dbab10 2023-10-17T01:27:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89dc8020 con 0x55bb89dba1b0 2023-10-17T01:27:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb89dc81e0 con 0x55bb89db93e0 2023-10-17T01:27:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09da0e3b80 10 client.0 _ll_register_callbacks cb 0x7f09b001d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09c4d9c700 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89e1d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 -- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 msgr2=0x55bb89e1cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89e1cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 -- 172.21.15.45:0/1742029873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 msgr2=0x55bb89e1d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89e1d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01a1460 con 0x55bb89db93e0 2023-10-17T01:27:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f09bffff700 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89e1c7c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f09b001c7e0 tx=0x7f09b0005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ecb303b8093dd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.307+0000 7f099f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:27:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.308+0000 7f099f7fe700 1 -- 172.21.15.45:0/1742029873 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b001cc00 con 0x55bb89db93e0 2023-10-17T01:27:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.308+0000 7f099f7fe700 1 -- 172.21.15.45:0/1742029873 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f09b001cd60 con 0x55bb89db93e0 2023-10-17T01:27:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.308+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09b4010e00 con 0x55bb89db93e0 2023-10-17T01:27:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f099f7fe700 1 -- 172.21.15.45:0/1742029873 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b0004830 con 0x55bb89db93e0 2023-10-17T01:27:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 10 client.26737 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bb89b2e250 con 0x55bb89db93e0 2023-10-17T01:27:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f099f7fe700 1 -- 172.21.15.45:0/1742029873 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b4010e00 con 0x55bb89db93e0 2023-10-17T01:27:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f099f7fe700 1 -- 172.21.15.45:0/1742029873 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f09b0039030 con 0x55bb89db93e0 2023-10-17T01:27:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 10 client.26737 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 10 client.26737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 20 client.26737 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89e32d40 0x55bb89e35130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 4 client.26737 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:27:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.309+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bb89cface0 con 0x55bb89e32d40 2023-10-17T01:27:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.310+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89e32d40 0x55bb89e35130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.310+0000 7f09bf7fe700 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89e32d40 0x55bb89e35130 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a5a8520905e8aa server_cookie=a602a492d6fd4152 in_seq=0 out_seq=0 2023-10-17T01:27:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.310+0000 7f099f7fe700 10 client.26737 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.321+0000 7f099f7fe700 1 -- 172.21.15.45:0/1742029873 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bb89cface0 con 0x55bb89e32d40 2023-10-17T01:27:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.321+0000 7f099f7fe700 10 client.26737 handle_command_reply: tid=0 2023-10-17T01:27:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.321+0000 7f09da0e3b80 1 client.26737 shutdown 2023-10-17T01:27:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 msgr2=0x55bb89e1c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89e1c7c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f09b001c7e0 tx=0x7f09b0005df0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 shutdown_connections 2023-10-17T01:27:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb89dbab10 0x55bb89e1d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bb89db93e0 0x55bb89e1c7c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55bb89e32d40 0x55bb89e35130 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 --2- 172.21.15.45:0/1742029873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb89dba1b0 0x55bb89e1cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 >> 172.21.15.45:0/1742029873 conn(0x55bb89d10070 msgr2=0x55bb89dc2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f099f7fe700 0 client.26737 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.322+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 shutdown_connections 2023-10-17T01:27:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.323+0000 7f09da0e3b80 1 -- 172.21.15.45:0/1742029873 wait complete. 2023-10-17T01:27:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.323+0000 7f09da0e3b80 20 client.26737 trim_cache size 0 max 16384 2023-10-17T01:27:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:27:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:27:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:27:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:27:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09a80a0ba0 msgr2=0x7f09a80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09a80a0ba0 0x7f09a80a3060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09c0101090 tx=0x7f09b0009260 comp rx=0 tx=0).stop 2023-10-17T01:27:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 msgr2=0x7f09c0100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0100200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009fd0 tx=0x7f09b8007930 comp rx=0 tx=0).stop 2023-10-17T01:27:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 shutdown_connections 2023-10-17T01:27:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f09a80a0ba0 0x7f09a80a3060 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c0106a30 0x7f09c0100200 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c01017d0 0x7f09c00ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 --2- 172.21.15.45:0/887385057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09c0104100 0x7f09c00ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 >> 172.21.15.45:0/887385057 conn(0x7f09c00fb1a0 msgr2=0x7f09c0102440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 shutdown_connections 2023-10-17T01:27:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:18.324+0000 7f09c67ff700 1 -- 172.21.15.45:0/887385057 wait complete. 2023-10-17T01:27:19.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:18 smithi181 ceph-mon[134520]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 685 op/s 2023-10-17T01:27:19.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:18 smithi045 ceph-mon[98474]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 685 op/s 2023-10-17T01:27:19.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:18 smithi060 ceph-mon[134136]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 685 op/s 2023-10-17T01:27:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 730.0 s, 1130.8 tps, lat 88.171 ms stddev 101.294 2023-10-17T01:27:21.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:20 smithi181 ceph-mon[134520]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 784 op/s 2023-10-17T01:27:21.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:20 smithi045 ceph-mon[98474]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 784 op/s 2023-10-17T01:27:21.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:20 smithi060 ceph-mon[134136]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 784 op/s 2023-10-17T01:27:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:22 smithi181 ceph-mon[134520]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 712 op/s 2023-10-17T01:27:23.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:22 smithi045 ceph-mon[98474]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 712 op/s 2023-10-17T01:27:23.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:22 smithi060 ceph-mon[134136]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 712 op/s 2023-10-17T01:27:25.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:24 smithi181 ceph-mon[134520]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 896 op/s 2023-10-17T01:27:25.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:24 smithi045 ceph-mon[98474]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 896 op/s 2023-10-17T01:27:25.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:24 smithi060 ceph-mon[134136]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 896 op/s 2023-10-17T01:27:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 735.0 s, 1336.8 tps, lat 75.276 ms stddev 88.930 2023-10-17T01:27:27.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:26 smithi181 ceph-mon[134520]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 664 op/s 2023-10-17T01:27:27.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:26 smithi045 ceph-mon[98474]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 664 op/s 2023-10-17T01:27:27.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:26 smithi060 ceph-mon[134136]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 664 op/s 2023-10-17T01:27:29.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:28 smithi181 ceph-mon[134520]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 730 op/s 2023-10-17T01:27:29.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:28 smithi045 ceph-mon[98474]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 730 op/s 2023-10-17T01:27:29.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:28 smithi060 ceph-mon[134136]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 730 op/s 2023-10-17T01:27:30.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 740.0 s, 1333.7 tps, lat 75.394 ms stddev 86.618 2023-10-17T01:27:31.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:30 smithi181 ceph-mon[134520]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 738 op/s 2023-10-17T01:27:31.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:30 smithi045 ceph-mon[98474]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 738 op/s 2023-10-17T01:27:31.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:30 smithi060 ceph-mon[134136]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 738 op/s 2023-10-17T01:27:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.582+0000 7f12c99ff700 1 -- 172.21.15.45:0/3465354201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c4104100 msgr2=0x7f12c41064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.582+0000 7f12c99ff700 1 --2- 172.21.15.45:0/3465354201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c4104100 0x7f12c41064f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f12b400faf0 tx=0x7f12b400fe00 comp rx=0 tx=0).stop 2023-10-17T01:27:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.582+0000 7f12c99ff700 1 -- 172.21.15.45:0/3465354201 shutdown_connections 2023-10-17T01:27:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.582+0000 7f12c99ff700 1 --2- 172.21.15.45:0/3465354201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4106a30 0x7f12c4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.583+0000 7f12c99ff700 1 --2- 172.21.15.45:0/3465354201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c41017d0 0x7f12c4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.583+0000 7f12c99ff700 1 --2- 172.21.15.45:0/3465354201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c4104100 0x7f12c41064f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.583+0000 7f12c99ff700 1 -- 172.21.15.45:0/3465354201 >> 172.21.15.45:0/3465354201 conn(0x7f12c40fb1a0 msgr2=0x7f12c40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.586+0000 7f12c99ff700 1 -- 172.21.15.45:0/3465354201 shutdown_connections 2023-10-17T01:27:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.586+0000 7f12c99ff700 1 -- 172.21.15.45:0/3465354201 wait complete. 2023-10-17T01:27:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.587+0000 7f12c99ff700 1 Processor -- start 2023-10-17T01:27:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 -- start start 2023-10-17T01:27:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 0x7f12c4194990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4104100 0x7f12c4194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c4106a30 0x7f12c4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c41963b0 con 0x7f12c4106a30 2023-10-17T01:27:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4196520 con 0x7f12c4104100 2023-10-17T01:27:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.588+0000 7f12c99ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c41966e0 con 0x7f12c41017d0 2023-10-17T01:27:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 0x7f12c4194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c4106a30 0x7f12c4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 0x7f12c4194990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:49036/0 (socket says 172.21.15.45:49036) 2023-10-17T01:27:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 -- 172.21.15.45:0/499154881 learned_addr learned my addr 172.21.15.45:0/499154881 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c27fc700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4104100 0x7f12c4194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 -- 172.21.15.45:0/499154881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4104100 msgr2=0x7f12c4194ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4104100 0x7f12c4194ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 -- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c4106a30 msgr2=0x7f12c4195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c4106a30 0x7f12c4195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.589+0000 7f12c2ffd700 1 -- 172.21.15.45:0/499154881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b400f750 con 0x7f12c41017d0 2023-10-17T01:27:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.590+0000 7f12c37fe700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c4106a30 0x7f12c4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.590+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 0x7f12c4194990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d950 tx=0x7f12b800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.595+0000 7f12c89fd700 1 -- 172.21.15.45:0/499154881 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8009b40 con 0x7f12c41017d0 2023-10-17T01:27:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.595+0000 7f12c89fd700 1 -- 172.21.15.45:0/499154881 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12b8009ca0 con 0x7f12c41017d0 2023-10-17T01:27:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.595+0000 7f12c89fd700 1 -- 172.21.15.45:0/499154881 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800f620 con 0x7f12c41017d0 2023-10-17T01:27:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.597+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c41969c0 con 0x7f12c41017d0 2023-10-17T01:27:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.597+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c419d180 con 0x7f12c41017d0 2023-10-17T01:27:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.607+0000 7f12c89fd700 1 -- 172.21.15.45:0/499154881 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f12b8010460 con 0x7f12c41017d0 2023-10-17T01:27:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.609+0000 7f12c89fd700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ac0a0c70 0x7f12ac0a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.609+0000 7f12dd307b80 1 Processor -- start 2023-10-17T01:27:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.610+0000 7f12dd307b80 1 -- start start 2023-10-17T01:27:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.610+0000 7f12c89fd700 1 -- 172.21.15.45:0/499154881 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f12b80cc420 con 0x7f12c41017d0 2023-10-17T01:27:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.610+0000 7f12c27fc700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ac0a0c70 0x7f12ac0a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.611+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1e830 0x560aa0c1ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.611+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c1f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.611+0000 7f12c27fc700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ac0a0c70 0x7f12ac0a3130 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009870 tx=0x7f12b4011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.611+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 0x560aa0c20210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12dd307b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0b769b0 con 0x560aa0c1fd90 2023-10-17T01:27:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12dd307b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0b74370 con 0x560aa0c1f190 2023-10-17T01:27:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12dd307b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0b6fc60 con 0x560aa0c1e830 2023-10-17T01:27:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12c27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c1f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 0x560aa0c20210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 0x560aa0c20210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38104/0 (socket says 172.21.15.45:38104) 2023-10-17T01:27:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.612+0000 7f12c37fe700 1 -- 172.21.15.45:0/1023370212 learned_addr learned my addr 172.21.15.45:0/1023370212 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.613+0000 7f12c37fe700 1 -- 172.21.15.45:0/1023370212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1e830 msgr2=0x560aa0c1ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.613+0000 7f12c37fe700 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1e830 0x560aa0c1ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.613+0000 7f12c37fe700 1 -- 172.21.15.45:0/1023370212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 msgr2=0x560aa0c1f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.613+0000 7f12c37fe700 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c1f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.613+0000 7f12c37fe700 1 -- 172.21.15.45:0/1023370212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa099dd80 con 0x560aa0c1fd90 2023-10-17T01:27:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.617+0000 7f12c37fe700 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 0x560aa0c20210 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12bc007f30 tx=0x7f12bc00c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e3d3ec7f4e9520 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.618+0000 7f12a9ffb700 1 -- 172.21.15.45:0/1023370212 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00f640 con 0x560aa0c1fd90 2023-10-17T01:27:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.618+0000 7f12a9ffb700 1 -- 172.21.15.45:0/1023370212 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12bc00fc60 con 0x560aa0c1fd90 2023-10-17T01:27:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.618+0000 7f12a9ffb700 1 -- 172.21.15.45:0/1023370212 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00f640 con 0x560aa0c1fd90 2023-10-17T01:27:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.619+0000 7f12dd307b80 1 -- 172.21.15.45:0/1023370212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 msgr2=0x560aa0c20210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.619+0000 7f12dd307b80 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 0x560aa0c20210 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12bc007f30 tx=0x7f12bc00c5a0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 -- 172.21.15.45:0/1023370212 shutdown_connections 2023-10-17T01:27:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1fd90 0x560aa0c20210 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1e830 0x560aa0c1ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 --2- 172.21.15.45:0/1023370212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c1f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 -- 172.21.15.45:0/1023370212 >> 172.21.15.45:0/1023370212 conn(0x560aa0b7e070 msgr2=0x560aa0c192c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 -- 172.21.15.45:0/1023370212 shutdown_connections 2023-10-17T01:27:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.620+0000 7f12dd307b80 1 -- 172.21.15.45:0/1023370212 wait complete. 2023-10-17T01:27:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.621+0000 7f12dd307b80 1 -- start start 2023-10-17T01:27:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.621+0000 7f12dd307b80 1 Processor -- start 2023-10-17T01:27:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.622+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.622+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1f190 0x560aa0c28910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.622+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1fd90 0x560aa0c2f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.622+0000 7f12dd307b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0b0c4b0 con 0x560aa0c1e830 2023-10-17T01:27:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.622+0000 7f12dd307b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0b0c150 con 0x560aa0c1fd90 2023-10-17T01:27:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.622+0000 7f12dd307b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa09f38c0 con 0x560aa0c1f190 2023-10-17T01:27:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.623+0000 7f12dd307b80 10 client.0 _ll_register_callbacks cb 0x560aa0a59a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.623+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.623+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c283d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38116/0 (socket says 172.21.15.45:38116) 2023-10-17T01:27:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.623+0000 7f12c2ffd700 1 -- 172.21.15.45:0/396393163 learned_addr learned my addr 172.21.15.45:0/396393163 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.624+0000 7f12c2ffd700 1 -- 172.21.15.45:0/396393163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1f190 msgr2=0x560aa0c28910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:27:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.624+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1f190 0x560aa0c28910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.624+0000 7f12c2ffd700 1 -- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1fd90 msgr2=0x560aa0c2f950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:27:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.624+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1fd90 0x560aa0c2f950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.624+0000 7f12c2ffd700 1 -- 172.21.15.45:0/396393163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc007b60 con 0x560aa0c1e830 2023-10-17T01:27:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.625+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c283d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009520 tx=0x7f12b8099530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dfb9c576be2cf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.625+0000 7f12a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:27:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.626+0000 7f12a2ffd700 1 -- 172.21.15.45:0/396393163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80a1b10 con 0x560aa0c1e830 2023-10-17T01:27:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.626+0000 7f12a2ffd700 1 -- 172.21.15.45:0/396393163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f12b8010710 con 0x560aa0c1e830 2023-10-17T01:27:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.626+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560aa0b9acb0 con 0x560aa0c1e830 2023-10-17T01:27:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.627+0000 7f12a2ffd700 1 -- 172.21.15.45:0/396393163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b80108d0 con 0x560aa0c1e830 2023-10-17T01:27:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.627+0000 7f12dd307b80 10 client.16437 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.627+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560aa0986c00 con 0x560aa0c1e830 2023-10-17T01:27:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.627+0000 7f12a2ffd700 1 -- 172.21.15.45:0/396393163 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560aa0b9acb0 con 0x560aa0c1e830 2023-10-17T01:27:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.627+0000 7f12a2ffd700 1 -- 172.21.15.45:0/396393163 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12b8099d00 con 0x560aa0c1e830 2023-10-17T01:27:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12dd307b80 10 client.16437 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12dd307b80 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12dd307b80 20 client.16437 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12dd307b80 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0c45310 0x560aa0c47730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12dd307b80 4 client.16437 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560aa0bedcd0 con 0x560aa0c45310 2023-10-17T01:27:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.628+0000 7f12c27fc700 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0c45310 0x560aa0c47730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.635+0000 7f12a2ffd700 10 client.16437 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.635+0000 7f12c27fc700 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0c45310 0x560aa0c47730 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1bc832d57aab5d server_cookie=843f8c5cbf343b6f in_seq=0 out_seq=0 2023-10-17T01:27:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12a2ffd700 1 -- 172.21.15.45:0/396393163 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x560aa0bedcd0 con 0x560aa0c45310 2023-10-17T01:27:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12a2ffd700 10 client.16437 handle_command_reply: tid=0 2023-10-17T01:27:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 client.16437 shutdown 2023-10-17T01:27:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 msgr2=0x560aa0c283d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c283d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009520 tx=0x7f12b8099530 comp rx=0 tx=0).stop 2023-10-17T01:27:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 shutdown_connections 2023-10-17T01:27:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0c45310 0x560aa0c47730 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1fd90 0x560aa0c2f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12a2ffd700 0 client.16437 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c283d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.637+0000 7f12dd307b80 1 --2- 172.21.15.45:0/396393163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1f190 0x560aa0c28910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.638+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 >> 172.21.15.45:0/396393163 conn(0x560aa0b7e070 msgr2=0x560aa0c18a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.638+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 shutdown_connections 2023-10-17T01:27:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.638+0000 7f12dd307b80 1 -- 172.21.15.45:0/396393163 wait complete. 2023-10-17T01:27:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.638+0000 7f12dd307b80 20 client.16437 trim_cache size 0 max 16384 2023-10-17T01:27:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 Processor -- start 2023-10-17T01:27:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 -- start start 2023-10-17T01:27:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c22d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c23260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0c139e0 con 0x560aa0c1e830 2023-10-17T01:27:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0c425d0 con 0x560aa0c1f190 2023-10-17T01:27:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.660+0000 7f12dd307b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0c1fbf0 con 0x560aa0c1fd90 2023-10-17T01:27:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c22d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c22d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59936/0 (socket says 172.21.15.45:59936) 2023-10-17T01:27:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 -- 172.21.15.45:0/4008271184 learned_addr learned my addr 172.21.15.45:0/4008271184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c37fe700 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c23260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 -- 172.21.15.45:0/4008271184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 msgr2=0x560aa0c23260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c23260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 -- 172.21.15.45:0/4008271184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 msgr2=0x560aa0c261d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c261d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 -- 172.21.15.45:0/4008271184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa09c41f0 con 0x560aa0c1f190 2023-10-17T01:27:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.661+0000 7f12c27fc700 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c22d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12b4005c50 tx=0x7f12b4023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b94a3c908d10e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.662+0000 7f12aa7fc700 1 -- 172.21.15.45:0/4008271184 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40351d0 con 0x560aa0c1f190 2023-10-17T01:27:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.662+0000 7f12aa7fc700 1 -- 172.21.15.45:0/4008271184 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12b4035330 con 0x560aa0c1f190 2023-10-17T01:27:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.662+0000 7f12aa7fc700 1 -- 172.21.15.45:0/4008271184 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4031800 con 0x560aa0c1f190 2023-10-17T01:27:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 -- 172.21.15.45:0/4008271184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 msgr2=0x560aa0c22d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c22d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12b4005c50 tx=0x7f12b4023000 comp rx=0 tx=0).stop 2023-10-17T01:27:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 -- 172.21.15.45:0/4008271184 shutdown_connections 2023-10-17T01:27:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c23260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 --2- 172.21.15.45:0/4008271184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c22d20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 -- 172.21.15.45:0/4008271184 >> 172.21.15.45:0/4008271184 conn(0x560aa0b7e070 msgr2=0x560aa0c27060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 -- 172.21.15.45:0/4008271184 shutdown_connections 2023-10-17T01:27:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 -- 172.21.15.45:0/4008271184 wait complete. 2023-10-17T01:27:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 -- start start 2023-10-17T01:27:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.663+0000 7f12dd307b80 1 Processor -- start 2023-10-17T01:27:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c9e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0c9edb0 con 0x560aa0c1e830 2023-10-17T01:27:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0c9eef0 con 0x560aa0c1f190 2023-10-17T01:27:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa0c9f0b0 con 0x560aa0c1fd90 2023-10-17T01:27:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12dd307b80 10 client.0 _ll_register_callbacks cb 0x7f12b8099870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c9e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c9e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:49070/0 (socket says 172.21.15.45:49070) 2023-10-17T01:27:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12c37fe700 1 -- 172.21.15.45:0/87873972 learned_addr learned my addr 172.21.15.45:0/87873972 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.664+0000 7f12c27fc700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c2ffd700 1 -- 172.21.15.45:0/87873972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 msgr2=0x560aa0c9e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c9e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c2ffd700 1 -- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 msgr2=0x560aa0c97c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c97c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c2ffd700 1 -- 172.21.15.45:0/87873972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c4109a90 con 0x560aa0c1e830 2023-10-17T01:27:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c27fc700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12c2ffd700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c976c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f12b8097df0 tx=0x7f12b801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c50d861ee227da server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:27:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12a2ffd700 1 -- 172.21.15.45:0/87873972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8010910 con 0x560aa0c1e830 2023-10-17T01:27:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.665+0000 7f12a2ffd700 1 -- 172.21.15.45:0/87873972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f12b80a1b10 con 0x560aa0c1e830 2023-10-17T01:27:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12bc00fc60 con 0x560aa0c1e830 2023-10-17T01:27:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12a2ffd700 1 -- 172.21.15.45:0/87873972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b80b90d0 con 0x560aa0c1e830 2023-10-17T01:27:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 10 client.16440 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560aa099c250 con 0x560aa0c1e830 2023-10-17T01:27:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12a2ffd700 1 -- 172.21.15.45:0/87873972 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12bc00fc60 con 0x560aa0c1e830 2023-10-17T01:27:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12a2ffd700 1 -- 172.21.15.45:0/87873972 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12b809aab0 con 0x560aa0c1e830 2023-10-17T01:27:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 20 client.16440 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0ca0760 0x560aa0ca2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 4 client.16440 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:27:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.666+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560aa0b68ce0 con 0x560aa0ca0760 2023-10-17T01:27:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.667+0000 7f12c27fc700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0ca0760 0x560aa0ca2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.668+0000 7f12c27fc700 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0ca0760 0x560aa0ca2b80 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0cdb5963f65f816 server_cookie=48bbec16b0231335 in_seq=0 out_seq=0 2023-10-17T01:27:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.668+0000 7f12a2ffd700 10 client.16440 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.677+0000 7f12a2ffd700 1 -- 172.21.15.45:0/87873972 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560aa0b68ce0 con 0x560aa0ca0760 2023-10-17T01:27:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.677+0000 7f12a2ffd700 10 client.16440 handle_command_reply: tid=0 2023-10-17T01:27:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.677+0000 7f12dd307b80 1 client.16440 shutdown 2023-10-17T01:27:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.677+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 msgr2=0x560aa0c976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.677+0000 7f12dd307b80 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c976c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f12b8097df0 tx=0x7f12b801e000 comp rx=0 tx=0).stop 2023-10-17T01:27:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 shutdown_connections 2023-10-17T01:27:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560aa0c1fd90 0x560aa0c9e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560aa0c1e830 0x560aa0c976c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560aa0ca0760 0x560aa0ca2b80 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 --2- 172.21.15.45:0/87873972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560aa0c1f190 0x560aa0c97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 >> 172.21.15.45:0/87873972 conn(0x560aa0b7e070 msgr2=0x560aa0c27060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12a2ffd700 0 client.16440 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 shutdown_connections 2023-10-17T01:27:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.678+0000 7f12dd307b80 1 -- 172.21.15.45:0/87873972 wait complete. 2023-10-17T01:27:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12dd307b80 20 client.16440 trim_cache size 0 max 16384 2023-10-17T01:27:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:27:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:27:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:27:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:27:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ac0a0c70 msgr2=0x7f12ac0a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ac0a0c70 0x7f12ac0a3130 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009870 tx=0x7f12b4011040 comp rx=0 tx=0).stop 2023-10-17T01:27:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 msgr2=0x7f12c4194990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 0x7f12c4194990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d950 tx=0x7f12b800dd10 comp rx=0 tx=0).stop 2023-10-17T01:27:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 shutdown_connections 2023-10-17T01:27:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f12ac0a0c70 0x7f12ac0a3130 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12c4106a30 0x7f12c4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c41017d0 0x7f12c4194990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.679+0000 7f12c99ff700 1 --2- 172.21.15.45:0/499154881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c4104100 0x7f12c4194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.680+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 >> 172.21.15.45:0/499154881 conn(0x7f12c40fb1a0 msgr2=0x7f12c4102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.680+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 shutdown_connections 2023-10-17T01:27:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:31.680+0000 7f12c99ff700 1 -- 172.21.15.45:0/499154881 wait complete. 2023-10-17T01:27:33.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:32 smithi181 ceph-mon[134520]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 673 op/s 2023-10-17T01:27:33.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:32 smithi045 ceph-mon[98474]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 673 op/s 2023-10-17T01:27:33.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:32 smithi060 ceph-mon[134136]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 673 op/s 2023-10-17T01:27:35.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:34 smithi045 ceph-mon[98474]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 837 op/s 2023-10-17T01:27:35.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:34 smithi060 ceph-mon[134136]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 837 op/s 2023-10-17T01:27:35.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:34 smithi181 ceph-mon[134520]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 837 op/s 2023-10-17T01:27:35.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 745.0 s, 1267.2 tps, lat 78.754 ms stddev 104.982 2023-10-17T01:27:37.164 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:36 smithi181 ceph-mon[134520]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 621 op/s 2023-10-17T01:27:37.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:36 smithi045 ceph-mon[98474]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 621 op/s 2023-10-17T01:27:37.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:36 smithi060 ceph-mon[134136]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 621 op/s 2023-10-17T01:27:38.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:27:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 -- 172.21.15.45:0/2775932144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c107d50 msgr2=0x7fdc1c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 --2- 172.21.15.45:0/2775932144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c107d50 0x7fdc1c108170 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b0a0 tx=0x7fdc0c00b3b0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 -- 172.21.15.45:0/2775932144 shutdown_connections 2023-10-17T01:27:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 --2- 172.21.15.45:0/2775932144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c10f5d0 0x7fdc1c1119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 --2- 172.21.15.45:0/2775932144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c107d50 0x7fdc1c108170 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 --2- 172.21.15.45:0/2775932144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1086b0 0x7fdc1c10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.421+0000 7fdc22bff700 1 -- 172.21.15.45:0/2775932144 >> 172.21.15.45:0/2775932144 conn(0x7fdc1c06db00 msgr2=0x7fdc1c06ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:38.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.422+0000 7fdc22bff700 1 -- 172.21.15.45:0/2775932144 shutdown_connections 2023-10-17T01:27:38.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.422+0000 7fdc22bff700 1 -- 172.21.15.45:0/2775932144 wait complete. 2023-10-17T01:27:38.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.423+0000 7fdc22bff700 1 Processor -- start 2023-10-17T01:27:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.424+0000 7fdc22bff700 1 -- start start 2023-10-17T01:27:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc22bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 0x7fdc1c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc22bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1086b0 0x7fdc1c1a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc22bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c10f5d0 0x7fdc1c1acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc22bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1ad5a0 con 0x7fdc1c1086b0 2023-10-17T01:27:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc22bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1ad6e0 con 0x7fdc1c107d50 2023-10-17T01:27:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc22bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1ad870 con 0x7fdc1c10f5d0 2023-10-17T01:27:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc2099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 0x7fdc1c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc2099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 0x7fdc1c1a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54606/0 (socket says 172.21.15.45:54606) 2023-10-17T01:27:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.425+0000 7fdc2099b700 1 -- 172.21.15.45:0/1031210618 learned_addr learned my addr 172.21.15.45:0/1031210618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:38.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.427+0000 7fdc2099b700 1 -- 172.21.15.45:0/1031210618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c10f5d0 msgr2=0x7fdc1c1acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.427+0000 7fdc2099b700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c10f5d0 0x7fdc1c1acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.427+0000 7fdc2099b700 1 -- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1086b0 msgr2=0x7fdc1c1a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.427+0000 7fdc2099b700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1086b0 0x7fdc1c1a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.427+0000 7fdc2099b700 1 -- 172.21.15.45:0/1031210618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c009d00 con 0x7fdc1c107d50 2023-10-17T01:27:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.428+0000 7fdc2099b700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 0x7fdc1c1a5a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b070 tx=0x7fdc0c004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.429+0000 7fdc1a7fc700 1 -- 172.21.15.45:0/1031210618 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00e040 con 0x7fdc1c107d50 2023-10-17T01:27:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.429+0000 7fdc1a7fc700 1 -- 172.21.15.45:0/1031210618 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0c01bd20 con 0x7fdc1c107d50 2023-10-17T01:27:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.429+0000 7fdc1a7fc700 1 -- 172.21.15.45:0/1031210618 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0059c0 con 0x7fdc1c107d50 2023-10-17T01:27:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.429+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c1adaf0 con 0x7fdc1c107d50 2023-10-17T01:27:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.430+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c1adfa0 con 0x7fdc1c107d50 2023-10-17T01:27:38.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.439+0000 7fdc1a7fc700 1 -- 172.21.15.45:0/1031210618 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdc0c01b860 con 0x7fdc1c107d50 2023-10-17T01:27:38.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.441+0000 7fdc36545b80 1 Processor -- start 2023-10-17T01:27:38.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.441+0000 7fdc36545b80 1 -- start start 2023-10-17T01:27:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e2778bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e2778cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2778d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc36545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e276e3a10 con 0x557e2778bba0 2023-10-17T01:27:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc36545b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e276e13d0 con 0x557e2778c500 2023-10-17T01:27:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc36545b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e276dccc0 con 0x557e2778d2b0 2023-10-17T01:27:38.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc2099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e2778cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc2099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e2778cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54608/0 (socket says 172.21.15.45:54608) 2023-10-17T01:27:38.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc2099b700 1 -- 172.21.15.45:0/3757110082 learned_addr learned my addr 172.21.15.45:0/3757110082 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:38.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.442+0000 7fdc2119c700 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e2778bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.443+0000 7fdc2119c700 1 -- 172.21.15.45:0/3757110082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 msgr2=0x557e2778d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.443+0000 7fdc2119c700 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2778d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.443+0000 7fdc2119c700 1 -- 172.21.15.45:0/3757110082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 msgr2=0x557e2778cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.443+0000 7fdc2119c700 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e2778cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.443+0000 7fdc2119c700 1 -- 172.21.15.45:0/3757110082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e2750ade0 con 0x557e2778bba0 2023-10-17T01:27:38.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.444+0000 7fdc1a7fc700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc080a0bf0 0x7fdc080a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.444+0000 7fdc2119c700 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e2778bfc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400a990 tx=0x7fdc1400aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66db090f50f4fd15 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.444+0000 7fdc1a7fc700 1 -- 172.21.15.45:0/1031210618 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc0c0cde50 con 0x7fdc1c107d50 2023-10-17T01:27:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.445+0000 7fdc2099b700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc080a0bf0 0x7fdc080a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.445+0000 7fdc12ffd700 1 -- 172.21.15.45:0/3757110082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14004440 con 0x557e2778bba0 2023-10-17T01:27:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.445+0000 7fdc12ffd700 1 -- 172.21.15.45:0/3757110082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc140045a0 con 0x557e2778bba0 2023-10-17T01:27:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.445+0000 7fdc12ffd700 1 -- 172.21.15.45:0/3757110082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400cd80 con 0x557e2778bba0 2023-10-17T01:27:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.445+0000 7fdc36545b80 1 -- 172.21.15.45:0/3757110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 msgr2=0x557e2778bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.445+0000 7fdc36545b80 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e2778bfc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400a990 tx=0x7fdc1400aca0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.446+0000 7fdc36545b80 1 -- 172.21.15.45:0/3757110082 shutdown_connections 2023-10-17T01:27:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.446+0000 7fdc36545b80 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2778d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.446+0000 7fdc36545b80 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e2778bfc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.446+0000 7fdc36545b80 1 --2- 172.21.15.45:0/3757110082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e2778cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.446+0000 7fdc36545b80 1 -- 172.21.15.45:0/3757110082 >> 172.21.15.45:0/3757110082 conn(0x557e276eb0d0 msgr2=0x557e27786950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.447+0000 7fdc36545b80 1 -- 172.21.15.45:0/3757110082 shutdown_connections 2023-10-17T01:27:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.447+0000 7fdc36545b80 1 -- 172.21.15.45:0/3757110082 wait complete. 2023-10-17T01:27:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.448+0000 7fdc36545b80 1 -- start start 2023-10-17T01:27:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.448+0000 7fdc36545b80 1 Processor -- start 2023-10-17T01:27:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.448+0000 7fdc2099b700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc080a0bf0 0x7fdc080a30b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009fd0 tx=0x7fdc0c009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:38.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e27795900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e27795e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2779ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc2119c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e27795900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc2119c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e27795900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60928/0 (socket says 172.21.15.45:60928) 2023-10-17T01:27:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc2119c700 1 -- 172.21.15.45:0/4218847256 learned_addr learned my addr 172.21.15.45:0/4218847256 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e27679510 con 0x557e2778bba0 2023-10-17T01:27:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e276791b0 con 0x557e2778c500 2023-10-17T01:27:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.449+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e27560920 con 0x557e2778d2b0 2023-10-17T01:27:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.450+0000 7fdc2119c700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2779ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.450+0000 7fdc36545b80 10 client.0 _ll_register_callbacks cb 0x557e275c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.451+0000 7fdc13fff700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e27795e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.453+0000 7fdc13fff700 1 -- 172.21.15.45:0/4218847256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 msgr2=0x557e2779ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.453+0000 7fdc13fff700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2779ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.453+0000 7fdc13fff700 1 -- 172.21.15.45:0/4218847256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 msgr2=0x557e27795900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.453+0000 7fdc13fff700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e27795900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.453+0000 7fdc13fff700 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1400a5f0 con 0x557e2778c500 2023-10-17T01:27:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.453+0000 7fdc2119c700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e27795900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:27:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.458+0000 7fdc13fff700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e27795e40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009e20 tx=0x7fdc0400b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=825c36705e855da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:38.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.458+0000 7fdbfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:27:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.460+0000 7fdbfbfff700 1 -- 172.21.15.45:0/4218847256 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04011420 con 0x557e2778c500 2023-10-17T01:27:38.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.460+0000 7fdbfbfff700 1 -- 172.21.15.45:0/4218847256 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc04011580 con 0x557e2778c500 2023-10-17T01:27:38.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.462+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e27707d10 con 0x557e2778c500 2023-10-17T01:27:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.463+0000 7fdbfbfff700 1 -- 172.21.15.45:0/4218847256 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc04011740 con 0x557e2778c500 2023-10-17T01:27:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.464+0000 7fdc36545b80 10 client.26795 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.464+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557e274f3c60 con 0x557e2778c500 2023-10-17T01:27:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.464+0000 7fdbfbfff700 1 -- 172.21.15.45:0/4218847256 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e27707d10 con 0x557e2778c500 2023-10-17T01:27:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdbfbfff700 1 -- 172.21.15.45:0/4218847256 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc040118a0 con 0x557e2778c500 2023-10-17T01:27:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdc36545b80 10 client.26795 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdc36545b80 10 client.26795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdc36545b80 20 client.26795 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdc36545b80 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e277a16f0 0x557e277a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdc36545b80 4 client.26795 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.468+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e2775ad30 con 0x557e277a16f0 2023-10-17T01:27:38.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.470+0000 7fdc2119c700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e277a16f0 0x557e277a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.471+0000 7fdc2119c700 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e277a16f0 0x557e277a3b10 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2fcd0a79bddd24c server_cookie=5ea3fd92462aab86 in_seq=0 out_seq=0 2023-10-17T01:27:38.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.471+0000 7fdbfbfff700 10 client.26795 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:38.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.472+0000 7fdbfbfff700 1 -- 172.21.15.45:0/4218847256 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x557e2775ad30 con 0x557e277a16f0 2023-10-17T01:27:38.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.472+0000 7fdbfbfff700 10 client.26795 handle_command_reply: tid=0 2023-10-17T01:27:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.472+0000 7fdc36545b80 1 client.26795 shutdown 2023-10-17T01:27:38.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 msgr2=0x557e27795e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e27795e40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009e20 tx=0x7fdc0400b3c0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 shutdown_connections 2023-10-17T01:27:38.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e2779ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778bba0 0x557e27795900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e277a16f0 0x557e277a3b10 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 --2- 172.21.15.45:0/4218847256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778c500 0x557e27795e40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdbfbfff700 0 client.26795 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:38.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.473+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 >> 172.21.15.45:0/4218847256 conn(0x557e276eb0d0 msgr2=0x557e27786020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.474+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 shutdown_connections 2023-10-17T01:27:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.474+0000 7fdc36545b80 1 -- 172.21.15.45:0/4218847256 wait complete. 2023-10-17T01:27:38.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.475+0000 7fdc36545b80 20 client.26795 trim_cache size 0 max 16384 2023-10-17T01:27:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 Processor -- start 2023-10-17T01:27:38.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 -- start start 2023-10-17T01:27:38.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27787650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e2779d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27793710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2779eaf0 con 0x557e2778c500 2023-10-17T01:27:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e277934c0 con 0x557e2778bba0 2023-10-17T01:27:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.518+0000 7fdc36545b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e27787b90 con 0x557e2778d2b0 2023-10-17T01:27:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.519+0000 7fdc13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e2779d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.519+0000 7fdc13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e2779d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60936/0 (socket says 172.21.15.45:60936) 2023-10-17T01:27:38.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.519+0000 7fdc13fff700 1 -- 172.21.15.45:0/1962069909 learned_addr learned my addr 172.21.15.45:0/1962069909 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:38.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.520+0000 7fdc13fff700 1 -- 172.21.15.45:0/1962069909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 msgr2=0x557e27793710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.520+0000 7fdc13fff700 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27793710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.520+0000 7fdc13fff700 1 -- 172.21.15.45:0/1962069909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 msgr2=0x557e27787650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.520+0000 7fdc13fff700 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27787650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.520+0000 7fdc13fff700 1 -- 172.21.15.45:0/1962069909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e27531250 con 0x557e2778c500 2023-10-17T01:27:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.521+0000 7fdc13fff700 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e2779d6b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc04001eb0 tx=0x7fdc04011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f038cdce81bff1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.522+0000 7fdc137fe700 1 -- 172.21.15.45:0/1962069909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400bd20 con 0x557e2778c500 2023-10-17T01:27:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.522+0000 7fdc137fe700 1 -- 172.21.15.45:0/1962069909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0400be80 con 0x557e2778c500 2023-10-17T01:27:38.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.522+0000 7fdc137fe700 1 -- 172.21.15.45:0/1962069909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0401ac50 con 0x557e2778c500 2023-10-17T01:27:38.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 -- 172.21.15.45:0/1962069909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 msgr2=0x557e2779d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e2779d6b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc04001eb0 tx=0x7fdc04011e70 comp rx=0 tx=0).stop 2023-10-17T01:27:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 -- 172.21.15.45:0/1962069909 shutdown_connections 2023-10-17T01:27:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27793710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27787650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 --2- 172.21.15.45:0/1962069909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e2779d6b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 -- 172.21.15.45:0/1962069909 >> 172.21.15.45:0/1962069909 conn(0x557e276eb0d0 msgr2=0x557e277945c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 -- 172.21.15.45:0/1962069909 shutdown_connections 2023-10-17T01:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.523+0000 7fdc36545b80 1 -- 172.21.15.45:0/1962069909 wait complete. 2023-10-17T01:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.524+0000 7fdc36545b80 1 -- start start 2023-10-17T01:27:38.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.524+0000 7fdc36545b80 1 Processor -- start 2023-10-17T01:27:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.527+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27797c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.527+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e277981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.527+0000 7fdc36545b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27804090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.527+0000 7fdc2119c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27797c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.527+0000 7fdc2119c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27797c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54644/0 (socket says 172.21.15.45:54644) 2023-10-17T01:27:38.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc36545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e277988c0 con 0x557e2778c500 2023-10-17T01:27:38.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc36545b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e278045d0 con 0x557e2778bba0 2023-10-17T01:27:38.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc36545b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e27804790 con 0x557e2778d2b0 2023-10-17T01:27:38.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc36545b80 10 client.0 _ll_register_callbacks cb 0x7fdc04009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:38.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc2119c700 1 -- 172.21.15.45:0/436092314 learned_addr learned my addr 172.21.15.45:0/436092314 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:38.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc2119c700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27804090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc13fff700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e277981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc2119c700 1 -- 172.21.15.45:0/436092314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 msgr2=0x557e27804090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc2119c700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27804090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc2119c700 1 -- 172.21.15.45:0/436092314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 msgr2=0x557e277981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:38.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.528+0000 7fdc2119c700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e277981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.529+0000 7fdc2119c700 1 -- 172.21.15.45:0/436092314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e274f3c60 con 0x557e2778bba0 2023-10-17T01:27:38.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.529+0000 7fdc13fff700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e277981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:27:38.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.529+0000 7fdc2119c700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27797c90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400c680 tx=0x7fdc1400aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2478a9f949dc35af server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:38.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.529+0000 7fdbfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:27:38.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.530+0000 7fdbfbfff700 1 -- 172.21.15.45:0/436092314 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14004a30 con 0x557e2778bba0 2023-10-17T01:27:38.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.530+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc140045a0 con 0x557e2778bba0 2023-10-17T01:27:38.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.530+0000 7fdbfbfff700 1 -- 172.21.15.45:0/436092314 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdc1400c6b0 con 0x557e2778bba0 2023-10-17T01:27:38.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.542+0000 7fdbfbfff700 1 -- 172.21.15.45:0/436092314 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc14004a30 con 0x557e2778bba0 2023-10-17T01:27:38.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.542+0000 7fdc36545b80 10 client.26798 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:38.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.542+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557e275092b0 con 0x557e2778bba0 2023-10-17T01:27:38.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.543+0000 7fdbfbfff700 1 -- 172.21.15.45:0/436092314 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc140045a0 con 0x557e2778bba0 2023-10-17T01:27:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.543+0000 7fdbfbfff700 1 -- 172.21.15.45:0/436092314 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc14003ad0 con 0x557e2778bba0 2023-10-17T01:27:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.544+0000 7fdc36545b80 10 client.26798 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.544+0000 7fdc36545b80 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.544+0000 7fdc36545b80 20 client.26798 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:38.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.544+0000 7fdc36545b80 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e278062f0 0x557e27808710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:38.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.544+0000 7fdc36545b80 4 client.26798 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:27:38.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.544+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e276d5d40 con 0x557e278062f0 2023-10-17T01:27:38.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.547+0000 7fdc13fff700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e278062f0 0x557e27808710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.550+0000 7fdbfbfff700 10 client.26798 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.550+0000 7fdc13fff700 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e278062f0 0x557e27808710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28038a716118fbe server_cookie=640c054e9232630f in_seq=0 out_seq=0 2023-10-17T01:27:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.550+0000 7fdbfbfff700 1 -- 172.21.15.45:0/436092314 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557e276d5d40 con 0x557e278062f0 2023-10-17T01:27:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.550+0000 7fdbfbfff700 10 client.26798 handle_command_reply: tid=0 2023-10-17T01:27:38.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.550+0000 7fdc36545b80 1 client.26798 shutdown 2023-10-17T01:27:38.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 msgr2=0x557e27797c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27797c90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400c680 tx=0x7fdc1400aca0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 shutdown_connections 2023-10-17T01:27:38.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557e2778d2b0 0x557e27804090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x557e278062f0 0x557e27808710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e2778bba0 0x557e27797c90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 --2- 172.21.15.45:0/436092314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e2778c500 0x557e277981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdbfbfff700 0 client.26798 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:38.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 >> 172.21.15.45:0/436092314 conn(0x557e276eb0d0 msgr2=0x557e2779d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:38.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 shutdown_connections 2023-10-17T01:27:38.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.551+0000 7fdc36545b80 1 -- 172.21.15.45:0/436092314 wait complete. 2023-10-17T01:27:38.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.552+0000 7fdc36545b80 20 client.26798 trim_cache size 0 max 16384 2023-10-17T01:27:38.557 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:27:38.557 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:27:38.557 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:27:38.558 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:27:38.558 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:27:38.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:27:38.558 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:27:38.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:27:38.559 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:27:38.559 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:27:38.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.553+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc080a0bf0 msgr2=0x7fdc080a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.553+0000 7fdc22bff700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc080a0bf0 0x7fdc080a30b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009fd0 tx=0x7fdc0c009040 comp rx=0 tx=0).stop 2023-10-17T01:27:38.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.554+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 msgr2=0x7fdc1c1a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:38.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.554+0000 7fdc22bff700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 0x7fdc1c1a5a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b070 tx=0x7fdc0c004d30 comp rx=0 tx=0).stop 2023-10-17T01:27:38.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.555+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 shutdown_connections 2023-10-17T01:27:38.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.555+0000 7fdc22bff700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdc080a0bf0 0x7fdc080a30b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.555+0000 7fdc22bff700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1c10f5d0 0x7fdc1c1acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.555+0000 7fdc22bff700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc1c107d50 0x7fdc1c1a5a20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.555+0000 7fdc22bff700 1 --2- 172.21.15.45:0/1031210618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1c1086b0 0x7fdc1c1a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:38.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.555+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 >> 172.21.15.45:0/1031210618 conn(0x7fdc1c06db00 msgr2=0x7fdc1c06e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:38.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.556+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 shutdown_connections 2023-10-17T01:27:38.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:27:38.556+0000 7fdc22bff700 1 -- 172.21.15.45:0/1031210618 wait complete. 2023-10-17T01:27:38.593 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:27:39.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:38 smithi181 ceph-mon[134520]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 974 op/s 2023-10-17T01:27:39.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:38 smithi045 ceph-mon[98474]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 974 op/s 2023-10-17T01:27:39.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:38 smithi060 ceph-mon[134136]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 974 op/s 2023-10-17T01:27:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 750.0 s, 843.8 tps, lat 118.170 ms stddev 182.929 2023-10-17T01:27:41.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:27:41.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:40 smithi045 ceph-mon[98474]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-10-17T01:27:41.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:40 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.342 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:27:41.342 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.342 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.343 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:40 smithi060 ceph-mon[134136]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-10-17T01:27:41.343 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.343 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:40 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:27:41.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:40 smithi181 ceph-mon[134520]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-10-17T01:27:41.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:41.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:40 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:42.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:42.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:41 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:42.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:42.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:41 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:42.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:42.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:41 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:43.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:42 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:27:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:42 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:27:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:42 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:42 smithi045 ceph-mon[98474]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-10-17T01:27:43.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:42 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:27:43.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:42 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:27:43.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:42 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:43.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:42 smithi060 ceph-mon[134136]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-10-17T01:27:43.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:42 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:27:43.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:42 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:27:43.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:42 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:27:43.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:42 smithi181 ceph-mon[134520]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-10-17T01:27:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 -- 172.21.15.45:0/3228997743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424102910 msgr2=0x7f44241091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 --2- 172.21.15.45:0/3228997743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424102910 0x7f44241091c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f90 tx=0x7f441c00b330 comp rx=0 tx=0).stop 2023-10-17T01:27:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 -- 172.21.15.45:0/3228997743 shutdown_connections 2023-10-17T01:27:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 --2- 172.21.15.45:0/3228997743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424102910 0x7f44241091c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 --2- 172.21.15.45:0/3228997743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f44241011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 --2- 172.21.15.45:0/3228997743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424101f70 0x7f44241023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 -- 172.21.15.45:0/3228997743 >> 172.21.15.45:0/3228997743 conn(0x7f44240fc2d0 msgr2=0x7f44240fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 -- 172.21.15.45:0/3228997743 shutdown_connections 2023-10-17T01:27:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.904+0000 7f4428bff700 1 -- 172.21.15.45:0/3228997743 wait complete. 2023-10-17T01:27:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.905+0000 7f4428bff700 1 Processor -- start 2023-10-17T01:27:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.905+0000 7f4428bff700 1 -- start start 2023-10-17T01:27:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.905+0000 7f4428bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f4424198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.905+0000 7f4428bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424101f70 0x7f4424199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4428bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424102910 0x7f442419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4428bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419a8d0 con 0x7f4424102910 2023-10-17T01:27:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4428bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419aa40 con 0x7f4424100d80 2023-10-17T01:27:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4428bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419ac00 con 0x7f4424101f70 2023-10-17T01:27:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f4424198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4421d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424101f70 0x7f4424199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f4424198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54672/0 (socket says 172.21.15.45:54672) 2023-10-17T01:27:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 -- 172.21.15.45:0/336577297 learned_addr learned my addr 172.21.15.45:0/336577297 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4422d9d700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424102910 0x7f442419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 -- 172.21.15.45:0/336577297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424101f70 msgr2=0x7f4424199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424101f70 0x7f4424199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 -- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424102910 msgr2=0x7f442419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424102910 0x7f442419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f442259c700 1 -- 172.21.15.45:0/336577297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c009c40 con 0x7f4424100d80 2023-10-17T01:27:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4421d9b700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424101f70 0x7f4424199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.906+0000 7f4422d9d700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424102910 0x7f442419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:27:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.907+0000 7f442259c700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f4424198ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f441400e9f0 tx=0x7f441400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.907+0000 7f44137fe700 1 -- 172.21.15.45:0/336577297 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414004220 con 0x7f4424100d80 2023-10-17T01:27:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.907+0000 7f44137fe700 1 -- 172.21.15.45:0/336577297 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4414004380 con 0x7f4424100d80 2023-10-17T01:27:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.907+0000 7f44137fe700 1 -- 172.21.15.45:0/336577297 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414003c00 con 0x7f4424100d80 2023-10-17T01:27:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.907+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442419aee0 con 0x7f4424100d80 2023-10-17T01:27:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.907+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44241a1640 con 0x7f4424100d80 2023-10-17T01:27:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.912+0000 7f44137fe700 1 -- 172.21.15.45:0/336577297 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f441400c7c0 con 0x7f4424100d80 2023-10-17T01:27:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.914+0000 7f44137fe700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f440c0a0ba0 0x7f440c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.914+0000 7f44137fe700 1 -- 172.21.15.45:0/336577297 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4414014070 con 0x7f4424100d80 2023-10-17T01:27:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.916+0000 7f4421d9b700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f440c0a0ba0 0x7f440c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.916+0000 7f443c44cb80 1 Processor -- start 2023-10-17T01:27:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.917+0000 7f443c44cb80 1 -- start start 2023-10-17T01:27:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.917+0000 7f4421d9b700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f440c0a0ba0 0x7f440c0a3060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4418011490 tx=0x7f4418009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d4769040 0x55d7d4769460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4769fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d476aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f443c44cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d46b89b0 con 0x55d7d4769040 2023-10-17T01:27:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f443c44cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d46b6370 con 0x55d7d47699a0 2023-10-17T01:27:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f443c44cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d46b1c60 con 0x55d7d476a5a0 2023-10-17T01:27:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d476aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d476aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43872/0 (socket says 172.21.15.45:43872) 2023-10-17T01:27:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4422d9d700 1 -- 172.21.15.45:0/2327113259 learned_addr learned my addr 172.21.15.45:0/2327113259 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4769fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.918+0000 7f442259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d4769040 0x55d7d4769460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 -- 172.21.15.45:0/2327113259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 msgr2=0x55d7d476aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d476aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 -- 172.21.15.45:0/2327113259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d4769040 msgr2=0x55d7d4769460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d4769040 0x55d7d4769460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 -- 172.21.15.45:0/2327113259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7d44dfd80 con 0x55d7d47699a0 2023-10-17T01:27:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f442259c700 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d4769040 0x55d7d4769460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.919+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4769fa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44180042f0 tx=0x7f4418014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1efb1ecf773d93 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.920+0000 7f4410ff9700 1 -- 172.21.15.45:0/2327113259 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418003d10 con 0x55d7d47699a0 2023-10-17T01:27:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.920+0000 7f4410ff9700 1 -- 172.21.15.45:0/2327113259 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4418003e70 con 0x55d7d47699a0 2023-10-17T01:27:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.920+0000 7f4410ff9700 1 -- 172.21.15.45:0/2327113259 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441801e7f0 con 0x55d7d47699a0 2023-10-17T01:27:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 -- 172.21.15.45:0/2327113259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 msgr2=0x55d7d4769fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4769fa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44180042f0 tx=0x7f4418014000 comp rx=0 tx=0).stop 2023-10-17T01:27:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 -- 172.21.15.45:0/2327113259 shutdown_connections 2023-10-17T01:27:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d476aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d4769040 0x55d7d4769460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2327113259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4769fa0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.921+0000 7f443c44cb80 1 -- 172.21.15.45:0/2327113259 >> 172.21.15.45:0/2327113259 conn(0x55d7d46c0070 msgr2=0x55d7d475b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.922+0000 7f443c44cb80 1 -- 172.21.15.45:0/2327113259 shutdown_connections 2023-10-17T01:27:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.922+0000 7f443c44cb80 1 -- 172.21.15.45:0/2327113259 wait complete. 2023-10-17T01:27:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.923+0000 7f443c44cb80 1 -- start start 2023-10-17T01:27:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.923+0000 7f443c44cb80 1 Processor -- start 2023-10-17T01:27:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4761e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4762350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d464e4b0 con 0x55d7d476a5a0 2023-10-17T01:27:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d464e150 con 0x55d7d47699a0 2023-10-17T01:27:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d45358c0 con 0x55d7d4769040 2023-10-17T01:27:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f443c44cb80 10 client.0 _ll_register_callbacks cb 0x55d7d459ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4761e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4761e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43874/0 (socket says 172.21.15.45:43874) 2023-10-17T01:27:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 -- 172.21.15.45:0/2829088250 learned_addr learned my addr 172.21.15.45:0/2829088250 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4762350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f4422d9d700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 -- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 msgr2=0x55d7d4762350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4762350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 -- 172.21.15.45:0/2829088250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 msgr2=0x55d7d4767380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.924+0000 7f442259c700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f442259c700 1 -- 172.21.15.45:0/2829088250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44241a1640 con 0x55d7d4769040 2023-10-17T01:27:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f4422d9d700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f442259c700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4761e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f44140c9fe0 tx=0x7f4414004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a260e5f0897e7a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f4409ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:27:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f4409ffb700 1 -- 172.21.15.45:0/2829088250 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140cc2d0 con 0x55d7d4769040 2023-10-17T01:27:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f4409ffb700 1 -- 172.21.15.45:0/2829088250 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f441409d960 con 0x55d7d4769040 2023-10-17T01:27:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7d46dccb0 con 0x55d7d4769040 2023-10-17T01:27:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f4409ffb700 1 -- 172.21.15.45:0/2829088250 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44140a1ba0 con 0x55d7d4769040 2023-10-17T01:27:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f443c44cb80 10 client.26770 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.925+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f4418017070 con 0x55d7d4769040 2023-10-17T01:27:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f4409ffb700 1 -- 172.21.15.45:0/2829088250 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7d46dccb0 con 0x55d7d4769040 2023-10-17T01:27:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f4409ffb700 1 -- 172.21.15.45:0/2829088250 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f44140b90e0 con 0x55d7d4769040 2023-10-17T01:27:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f443c44cb80 10 client.26770 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f443c44cb80 10 client.26770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f443c44cb80 20 client.26770 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d4787290 0x55d7d47896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f443c44cb80 4 client.26770 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7d472fcd0 con 0x55d7d4787290 2023-10-17T01:27:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.926+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d4787290 0x55d7d47896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.927+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d4787290 0x55d7d47896b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d275411a5638c987 server_cookie=23b5d8fd609ad80f in_seq=0 out_seq=0 2023-10-17T01:27:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.927+0000 7f4409ffb700 10 client.26770 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.927+0000 7f4409ffb700 1 -- 172.21.15.45:0/2829088250 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d7d472fcd0 con 0x55d7d4787290 2023-10-17T01:27:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.927+0000 7f4409ffb700 10 client.26770 handle_command_reply: tid=0 2023-10-17T01:27:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.927+0000 7f443c44cb80 1 client.26770 shutdown 2023-10-17T01:27:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 msgr2=0x55d7d4761e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4761e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f44140c9fe0 tx=0x7f4414004570 comp rx=0 tx=0).stop 2023-10-17T01:27:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 shutdown_connections 2023-10-17T01:27:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4761e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d4787290 0x55d7d47896b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2829088250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d4762350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f4409ffb700 0 client.26770 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 >> 172.21.15.45:0/2829088250 conn(0x55d7d46c0070 msgr2=0x55d7d475aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 shutdown_connections 2023-10-17T01:27:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.928+0000 7f443c44cb80 1 -- 172.21.15.45:0/2829088250 wait complete. 2023-10-17T01:27:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.929+0000 7f443c44cb80 20 client.26770 trim_cache size 0 max 16384 2023-10-17T01:27:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.953+0000 7f443c44cb80 1 Processor -- start 2023-10-17T01:27:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.953+0000 7f443c44cb80 1 -- start start 2023-10-17T01:27:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.953+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4767340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.953+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d475aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f443c44cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d4763000 con 0x55d7d476a5a0 2023-10-17T01:27:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f443c44cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d475c5c0 con 0x55d7d47699a0 2023-10-17T01:27:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f443c44cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d4758010 con 0x55d7d4769040 2023-10-17T01:27:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4767340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f4421d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d475aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4767340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43886/0 (socket says 172.21.15.45:43886) 2023-10-17T01:27:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 -- 172.21.15.45:0/2942472342 learned_addr learned my addr 172.21.15.45:0/2942472342 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f4422d9d700 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 -- 172.21.15.45:0/2942472342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 msgr2=0x55d7d475aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d475aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 -- 172.21.15.45:0/2942472342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 msgr2=0x55d7d4767bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f442259c700 1 -- 172.21.15.45:0/2942472342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7d44c8c00 con 0x55d7d4769040 2023-10-17T01:27:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.954+0000 7f4421d9b700 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d475aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.955+0000 7f442259c700 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4767340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f441400c5b0 tx=0x7f4414009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d1b8ca2831214d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.955+0000 7f44117fa700 1 -- 172.21.15.45:0/2942472342 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140a1710 con 0x55d7d4769040 2023-10-17T01:27:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.955+0000 7f44117fa700 1 -- 172.21.15.45:0/2942472342 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441409d960 con 0x55d7d4769040 2023-10-17T01:27:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.955+0000 7f44117fa700 1 -- 172.21.15.45:0/2942472342 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140cc020 con 0x55d7d4769040 2023-10-17T01:27:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.956+0000 7f443c44cb80 1 -- 172.21.15.45:0/2942472342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 msgr2=0x55d7d4767340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.956+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4767340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f441400c5b0 tx=0x7f4414009530 comp rx=0 tx=0).stop 2023-10-17T01:27:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 -- 172.21.15.45:0/2942472342 shutdown_connections 2023-10-17T01:27:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d476a5a0 0x55d7d4767bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d4769040 0x55d7d4767340 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f441400c5b0 tx=0x7f4414009530 comp rx=0 tx=0).stop 2023-10-17T01:27:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 --2- 172.21.15.45:0/2942472342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47699a0 0x55d7d475aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 -- 172.21.15.45:0/2942472342 >> 172.21.15.45:0/2942472342 conn(0x55d7d46c0070 msgr2=0x55d7d4760ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 -- 172.21.15.45:0/2942472342 shutdown_connections 2023-10-17T01:27:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.957+0000 7f443c44cb80 1 -- 172.21.15.45:0/2942472342 wait complete. 2023-10-17T01:27:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.958+0000 7f443c44cb80 1 -- start start 2023-10-17T01:27:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.958+0000 7f443c44cb80 1 Processor -- start 2023-10-17T01:27:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d47699a0 0x55d7d47d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d47d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47d5830 0x55d7d47d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d47d8310 con 0x55d7d47699a0 2023-10-17T01:27:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d47d8480 con 0x55d7d47d5830 2023-10-17T01:27:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d47d8640 con 0x55d7d476a5a0 2023-10-17T01:27:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f443c44cb80 10 client.0 _ll_register_callbacks cb 0x7f4414099740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f4421d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d47d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f442259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d47699a0 0x55d7d47d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f4421d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d47d52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:43896/0 (socket says 172.21.15.45:43896) 2023-10-17T01:27:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f4421d9b700 1 -- 172.21.15.45:0/1584084329 learned_addr learned my addr 172.21.15.45:0/1584084329 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.959+0000 7f4422d9d700 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47d5830 0x55d7d47d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4421d9b700 1 -- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47d5830 msgr2=0x55d7d47d7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4421d9b700 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47d5830 0x55d7d47d7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4421d9b700 1 -- 172.21.15.45:0/1584084329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d47699a0 msgr2=0x55d7d47d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4421d9b700 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d47699a0 0x55d7d47d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4421d9b700 1 -- 172.21.15.45:0/1584084329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7d44dfd80 con 0x55d7d476a5a0 2023-10-17T01:27:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4421d9b700 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d47d52f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f44180042c0 tx=0x7f4418003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53468aed2b7d4ee server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4409ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:27:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4409ffb700 1 -- 172.21.15.45:0/1584084329 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441802a040 con 0x55d7d476a5a0 2023-10-17T01:27:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f4409ffb700 1 -- 172.21.15.45:0/1584084329 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4418034890 con 0x55d7d476a5a0 2023-10-17T01:27:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.960+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4418003e70 con 0x55d7d476a5a0 2023-10-17T01:27:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f4409ffb700 1 -- 172.21.15.45:0/1584084329 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4418034a50 con 0x55d7d476a5a0 2023-10-17T01:27:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 10 client.26776 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f4418017070 con 0x55d7d476a5a0 2023-10-17T01:27:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f4409ffb700 1 -- 172.21.15.45:0/1584084329 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4418003e70 con 0x55d7d476a5a0 2023-10-17T01:27:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f4409ffb700 1 -- 172.21.15.45:0/1584084329 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4418024020 con 0x55d7d476a5a0 2023-10-17T01:27:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 10 client.26776 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 10 client.26776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 20 client.26776 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d47da060 0x55d7d47dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 4 client.26776 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:27:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7d46aace0 con 0x55d7d47da060 2023-10-17T01:27:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.961+0000 7f442259c700 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d47da060 0x55d7d47dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.962+0000 7f4409ffb700 10 client.26776 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:44.962+0000 7f442259c700 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d47da060 0x55d7d47dc480 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b035a158e113f0aa server_cookie=ab7d383ca5fbf415 in_seq=0 out_seq=0 2023-10-17T01:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.011+0000 7f4409ffb700 1 -- 172.21.15.45:0/1584084329 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7d46aace0 con 0x55d7d47da060 2023-10-17T01:27:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.011+0000 7f4409ffb700 10 client.26776 handle_command_reply: tid=0 2023-10-17T01:27:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.012+0000 7f443c44cb80 1 client.26776 shutdown 2023-10-17T01:27:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 msgr2=0x55d7d47d52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d47d52f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f44180042c0 tx=0x7f4418003bf0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 shutdown_connections 2023-10-17T01:27:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d7d47da060 0x55d7d47dc480 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d7d47d5830 0x55d7d47d7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7d47699a0 0x55d7d47d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 --2- 172.21.15.45:0/1584084329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d7d476a5a0 0x55d7d47d52f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f4409ffb700 0 client.26776 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 >> 172.21.15.45:0/1584084329 conn(0x55d7d46c0070 msgr2=0x55d7d4759f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.013+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 shutdown_connections 2023-10-17T01:27:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.014+0000 7f443c44cb80 1 -- 172.21.15.45:0/1584084329 wait complete. 2023-10-17T01:27:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.015+0000 7f443c44cb80 20 client.26776 trim_cache size 0 max 16384 2023-10-17T01:27:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:27:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:27:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:27:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:27:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.016+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f440c0a0ba0 msgr2=0x7f440c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.017+0000 7f4428bff700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f440c0a0ba0 0x7f440c0a3060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4418011490 tx=0x7f4418009260 comp rx=0 tx=0).stop 2023-10-17T01:27:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.017+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 msgr2=0x7f4424198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.017+0000 7f4428bff700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f4424198ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f441400e9f0 tx=0x7f441400ed00 comp rx=0 tx=0).stop 2023-10-17T01:27:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 shutdown_connections 2023-10-17T01:27:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f440c0a0ba0 0x7f440c0a3060 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4424102910 0x7f442419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4424100d80 0x7f4424198ee0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 --2- 172.21.15.45:0/336577297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4424101f70 0x7f4424199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 >> 172.21.15.45:0/336577297 conn(0x7f44240fc2d0 msgr2=0x7f44240fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 shutdown_connections 2023-10-17T01:27:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:45.018+0000 7f4428bff700 1 -- 172.21.15.45:0/336577297 wait complete. 2023-10-17T01:27:45.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:44 smithi181 ceph-mon[134520]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 30 MiB/s wr, 1.90k op/s 2023-10-17T01:27:45.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:44 smithi045 ceph-mon[98474]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 30 MiB/s wr, 1.90k op/s 2023-10-17T01:27:45.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:44 smithi060 ceph-mon[134136]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 30 MiB/s wr, 1.90k op/s 2023-10-17T01:27:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 755.0 s, 1363.0 tps, lat 73.409 ms stddev 86.614 2023-10-17T01:27:47.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:46 smithi045 ceph-mon[98474]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.70k op/s 2023-10-17T01:27:47.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:46 smithi060 ceph-mon[134136]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.70k op/s 2023-10-17T01:27:47.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:46 smithi181 ceph-mon[134520]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 MiB/s wr, 1.70k op/s 2023-10-17T01:27:49.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:48 smithi181 ceph-mon[134520]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2023-10-17T01:27:49.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:48 smithi045 ceph-mon[98474]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2023-10-17T01:27:49.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:48 smithi060 ceph-mon[134136]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2023-10-17T01:27:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 760.0 s, 1349.2 tps, lat 73.004 ms stddev 87.090 2023-10-17T01:27:51.164 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:50 smithi181 ceph-mon[134520]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2023-10-17T01:27:51.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:50 smithi045 ceph-mon[98474]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2023-10-17T01:27:51.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:50 smithi060 ceph-mon[134136]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2023-10-17T01:27:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:52 smithi181 ceph-mon[134520]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 846 op/s 2023-10-17T01:27:53.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:52 smithi045 ceph-mon[98474]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 846 op/s 2023-10-17T01:27:53.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:52 smithi060 ceph-mon[134136]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 846 op/s 2023-10-17T01:27:55.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:54 smithi181 ceph-mon[134520]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 805 op/s 2023-10-17T01:27:55.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:54 smithi045 ceph-mon[98474]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 805 op/s 2023-10-17T01:27:55.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:54 smithi060 ceph-mon[134136]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 805 op/s 2023-10-17T01:27:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 765.0 s, 1569.4 tps, lat 65.203 ms stddev 84.491 2023-10-17T01:27:57.166 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:56 smithi181 ceph-mon[134520]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 614 op/s 2023-10-17T01:27:57.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:56 smithi045 ceph-mon[98474]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 614 op/s 2023-10-17T01:27:57.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:56 smithi060 ceph-mon[134136]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 614 op/s 2023-10-17T01:27:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 -- 172.21.15.45:0/154161295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8106a90 msgr2=0x7f4df8108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/154161295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8106a90 0x7f4df8108ea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009f90 tx=0x7f4df000b330 comp rx=0 tx=0).stop 2023-10-17T01:27:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 -- 172.21.15.45:0/154161295 shutdown_connections 2023-10-17T01:27:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/154161295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8106a90 0x7f4df8108ea0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/154161295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df8101830 0x7f4df8103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/154161295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8104160 0x7f4df8106550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 -- 172.21.15.45:0/154161295 >> 172.21.15.45:0/154161295 conn(0x7f4df80fb1a0 msgr2=0x7f4df80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 -- 172.21.15.45:0/154161295 shutdown_connections 2023-10-17T01:27:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.252+0000 7f4dff9ff700 1 -- 172.21.15.45:0/154161295 wait complete. 2023-10-17T01:27:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 Processor -- start 2023-10-17T01:27:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 -- start start 2023-10-17T01:27:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8101830 0x7f4df8194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df8104160 0x7f4df8194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8106a90 0x7f4df8195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8196290 con 0x7f4df8104160 2023-10-17T01:27:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8196400 con 0x7f4df8101830 2023-10-17T01:27:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dff9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81965c0 con 0x7f4df8106a90 2023-10-17T01:27:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dfd79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8101830 0x7f4df8194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dfd79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8101830 0x7f4df8194820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:34210/0 (socket says 172.21.15.45:34210) 2023-10-17T01:27:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dfd79b700 1 -- 172.21.15.45:0/3006485819 learned_addr learned my addr 172.21.15.45:0/3006485819 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8106a90 0x7f4df8195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.254+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df8104160 0x7f4df8194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dfdf9c700 1 -- 172.21.15.45:0/3006485819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8101830 msgr2=0x7f4df8194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8101830 0x7f4df8194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dfdf9c700 1 -- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df8104160 msgr2=0x7f4df8194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df8104160 0x7f4df8194d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dfdf9c700 1 -- 172.21.15.45:0/3006485819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0009c40 con 0x7f4df8106a90 2023-10-17T01:27:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8106a90 0x7f4df8195c30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009f90 tx=0x7f4df0004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dee7fc700 1 -- 172.21.15.45:0/3006485819 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000e040 con 0x7f4df8106a90 2023-10-17T01:27:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df8196840 con 0x7f4df8106a90 2023-10-17T01:27:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dee7fc700 1 -- 172.21.15.45:0/3006485819 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df000be00 con 0x7f4df8106a90 2023-10-17T01:27:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.255+0000 7f4dee7fc700 1 -- 172.21.15.45:0/3006485819 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df00052a0 con 0x7f4df8106a90 2023-10-17T01:27:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.256+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df819d080 con 0x7f4df8106a90 2023-10-17T01:27:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.263+0000 7f4dee7fc700 1 -- 172.21.15.45:0/3006485819 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f4df001d470 con 0x7f4df8106a90 2023-10-17T01:27:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.264+0000 7f4dee7fc700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4de40a0b20 0x7f4de40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4dee7fc700 1 -- 172.21.15.45:0/3006485819 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4df00cd8f0 con 0x7f4df8106a90 2023-10-17T01:27:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4de40a0b20 0x7f4de40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 Processor -- start 2023-10-17T01:27:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 -- start start 2023-10-17T01:27:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e39c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4de40a0b20 0x7f4de40a2fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4df8194590 tx=0x7f4df400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2f39b0 con 0x558e1e39b830 2023-10-17T01:27:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2f1370 con 0x558e1e39c190 2023-10-17T01:27:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.265+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2ecc60 con 0x558e1e39cd90 2023-10-17T01:27:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfcf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e39c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfdf9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfcf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e39c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:34212/0 (socket says 172.21.15.45:34212) 2023-10-17T01:27:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfcf9a700 1 -- 172.21.15.45:0/3774017045 learned_addr learned my addr 172.21.15.45:0/3774017045 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 -- 172.21.15.45:0/3774017045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 msgr2=0x558e1e39d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 -- 172.21.15.45:0/3774017045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 msgr2=0x558e1e39c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e39c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 -- 172.21.15.45:0/3774017045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e1e11ad80 con 0x558e1e39b830 2023-10-17T01:27:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e39c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.266+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39bc50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4df4010f40 tx=0x7f4df4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8443aa6adc82d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.267+0000 7f4de37fe700 1 -- 172.21.15.45:0/3774017045 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4011db0 con 0x558e1e39b830 2023-10-17T01:27:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.267+0000 7f4de37fe700 1 -- 172.21.15.45:0/3774017045 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df4015070 con 0x558e1e39b830 2023-10-17T01:27:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.267+0000 7f4de37fe700 1 -- 172.21.15.45:0/3774017045 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df401c5f0 con 0x558e1e39b830 2023-10-17T01:27:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 -- 172.21.15.45:0/3774017045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 msgr2=0x558e1e39bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39bc50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4df4010f40 tx=0x7f4df4013000 comp rx=0 tx=0).stop 2023-10-17T01:27:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 -- 172.21.15.45:0/3774017045 shutdown_connections 2023-10-17T01:27:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39bc50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/3774017045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e39c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 -- 172.21.15.45:0/3774017045 >> 172.21.15.45:0/3774017045 conn(0x558e1e2fb070 msgr2=0x558e1e3962c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 -- 172.21.15.45:0/3774017045 shutdown_connections 2023-10-17T01:27:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 -- 172.21.15.45:0/3774017045 wait complete. 2023-10-17T01:27:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.268+0000 7f4e133a8b80 1 -- start start 2023-10-17T01:27:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 Processor -- start 2023-10-17T01:27:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39b830 0x558e1e3a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39c190 0x558e1e3a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39cd90 0x558e1e3ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2894b0 con 0x558e1e39cd90 2023-10-17T01:27:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e289150 con 0x558e1e39b830 2023-10-17T01:27:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e1708c0 con 0x558e1e39c190 2023-10-17T01:27:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4e133a8b80 10 client.0 _ll_register_callbacks cb 0x558e1e1d6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4dfcf9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39c190 0x558e1e3a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4dfd79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39b830 0x558e1e3a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4dfcf9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39c190 0x558e1e3a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47758/0 (socket says 172.21.15.45:47758) 2023-10-17T01:27:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4dfcf9a700 1 -- 172.21.15.45:0/1608358803 learned_addr learned my addr 172.21.15.45:0/1608358803 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.269+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39cd90 0x558e1e3ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4dfd79b700 1 -- 172.21.15.45:0/1608358803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39c190 msgr2=0x558e1e3a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39c190 0x558e1e3a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4dfd79b700 1 -- 172.21.15.45:0/1608358803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39cd90 msgr2=0x558e1e3ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39cd90 0x558e1e3ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4dfd79b700 1 -- 172.21.15.45:0/1608358803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de8009960 con 0x558e1e39b830 2023-10-17T01:27:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39b830 0x558e1e3a53b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4df4011a00 tx=0x7f4df4011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=236163330fea6d55 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4de0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:27:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.270+0000 7f4de0ff9700 1 -- 172.21.15.45:0/1608358803 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df402d070 con 0x558e1e39b830 2023-10-17T01:27:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.271+0000 7f4de0ff9700 1 -- 172.21.15.45:0/1608358803 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4df4017070 con 0x558e1e39b830 2023-10-17T01:27:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.271+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e1e317cb0 con 0x558e1e39b830 2023-10-17T01:27:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.272+0000 7f4de0ff9700 1 -- 172.21.15.45:0/1608358803 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df401e630 con 0x558e1e39b830 2023-10-17T01:27:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.273+0000 7f4e133a8b80 10 client.26828 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.273+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f4df819d080 con 0x558e1e39b830 2023-10-17T01:27:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.273+0000 7f4de0ff9700 1 -- 172.21.15.45:0/1608358803 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df403a030 con 0x558e1e39b830 2023-10-17T01:27:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.273+0000 7f4de0ff9700 1 -- 172.21.15.45:0/1608358803 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4df4003960 con 0x558e1e39b830 2023-10-17T01:27:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.274+0000 7f4e133a8b80 10 client.26828 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.274+0000 7f4e133a8b80 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.274+0000 7f4e133a8b80 20 client.26828 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.274+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e3b1120 0x558e1e3b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.274+0000 7f4e133a8b80 4 client.26828 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:27:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.274+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e1e36acd0 con 0x558e1e3b1120 2023-10-17T01:27:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.275+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e3b1120 0x558e1e3b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.276+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e3b1120 0x558e1e3b3540 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e570eac71ca722 server_cookie=da824de7d78badb0 in_seq=0 out_seq=0 2023-10-17T01:27:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.276+0000 7f4de0ff9700 10 client.26828 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.276+0000 7f4de0ff9700 1 -- 172.21.15.45:0/1608358803 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x558e1e36acd0 con 0x558e1e3b1120 2023-10-17T01:27:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.276+0000 7f4de0ff9700 10 client.26828 handle_command_reply: tid=0 2023-10-17T01:27:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.276+0000 7f4e133a8b80 1 client.26828 shutdown 2023-10-17T01:27:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39b830 msgr2=0x558e1e3a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39b830 0x558e1e3a53b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4df4011a00 tx=0x7f4df4011bb0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 shutdown_connections 2023-10-17T01:27:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39cd90 0x558e1e3ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e3b1120 0x558e1e3b3540 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39b830 0x558e1e3a53b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1608358803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39c190 0x558e1e3a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4de0ff9700 0 client.26828 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 >> 172.21.15.45:0/1608358803 conn(0x558e1e2fb070 msgr2=0x558e1e395a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 shutdown_connections 2023-10-17T01:27:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1608358803 wait complete. 2023-10-17T01:27:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.277+0000 7f4e133a8b80 20 client.26828 trim_cache size 0 max 16384 2023-10-17T01:27:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.296+0000 7f4e133a8b80 1 Processor -- start 2023-10-17T01:27:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.297+0000 7f4e133a8b80 1 -- start start 2023-10-17T01:27:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e3a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e3b0cb0 con 0x558e1e39b830 2023-10-17T01:27:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e3a4860 con 0x558e1e39c190 2023-10-17T01:27:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e3a4f20 con 0x558e1e39cd90 2023-10-17T01:27:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4dfcf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e3a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4dfcf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e3a31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:34228/0 (socket says 172.21.15.45:34228) 2023-10-17T01:27:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.298+0000 7f4dfcf9a700 1 -- 172.21.15.45:0/1382016532 learned_addr learned my addr 172.21.15.45:0/1382016532 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfcf9a700 1 -- 172.21.15.45:0/1382016532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 msgr2=0x558e1e39fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfcf9a700 1 -- 172.21.15.45:0/1382016532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 msgr2=0x558e1e39d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfcf9a700 1 -- 172.21.15.45:0/1382016532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4010c20 con 0x558e1e39c190 2023-10-17T01:27:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.299+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:27:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.300+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e3a31d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x558e1e3b3e10 tx=0x7f4de8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c8ccc3f6d634ed server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.300+0000 7f4de3fff700 1 -- 172.21.15.45:0/1382016532 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8017070 con 0x558e1e39c190 2023-10-17T01:27:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.300+0000 7f4de3fff700 1 -- 172.21.15.45:0/1382016532 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de8018040 con 0x558e1e39c190 2023-10-17T01:27:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.300+0000 7f4de3fff700 1 -- 172.21.15.45:0/1382016532 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8021ab0 con 0x558e1e39c190 2023-10-17T01:27:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.301+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1382016532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 msgr2=0x558e1e3a31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.301+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e3a31d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x558e1e3b3e10 tx=0x7f4de8009310 comp rx=0 tx=0).stop 2023-10-17T01:27:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1382016532 shutdown_connections 2023-10-17T01:27:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39cd90 0x558e1e39fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39b830 0x558e1e39d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/1382016532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39c190 0x558e1e3a31d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1382016532 >> 172.21.15.45:0/1382016532 conn(0x558e1e2fb070 msgr2=0x558e1e3a4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1382016532 shutdown_connections 2023-10-17T01:27:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 -- 172.21.15.45:0/1382016532 wait complete. 2023-10-17T01:27:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 -- start start 2023-10-17T01:27:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.302+0000 7f4e133a8b80 1 Processor -- start 2023-10-17T01:27:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 0x558e1e3fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39c190 0x558e1e3ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39cd90 0x558e1e4017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e401f00 con 0x558e1e39c190 2023-10-17T01:27:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e402070 con 0x558e1e39cd90 2023-10-17T01:27:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e402230 con 0x558e1e39b830 2023-10-17T01:27:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4e133a8b80 10 client.0 _ll_register_callbacks cb 0x7f4de8009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:27:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4dfd79b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 0x558e1e3fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4dfdf9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39cd90 0x558e1e4017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4dfd79b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 0x558e1e3fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47782/0 (socket says 172.21.15.45:47782) 2023-10-17T01:27:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4dfd79b700 1 -- 172.21.15.45:0/4036343526 learned_addr learned my addr 172.21.15.45:0/4036343526 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:27:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39c190 0x558e1e3ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.303+0000 7f4dfd79b700 1 -- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39cd90 msgr2=0x558e1e4017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39cd90 0x558e1e4017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4dfd79b700 1 -- 172.21.15.45:0/4036343526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39c190 msgr2=0x558e1e3ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:27:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39c190 0x558e1e3ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4dfd79b700 1 -- 172.21.15.45:0/4036343526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de8004040 con 0x558e1e39b830 2023-10-17T01:27:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4dfdf9c700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39cd90 0x558e1e4017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:27:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4dfd79b700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 0x558e1e3fcc30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4df400bf20 tx=0x7f4df4003930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a5ff3ce9985577 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:27:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4de0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:27:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4de0ff9700 1 -- 172.21.15.45:0/4036343526 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df402d060 con 0x558e1e39b830 2023-10-17T01:27:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4df4015070 con 0x558e1e39b830 2023-10-17T01:27:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4de0ff9700 1 -- 172.21.15.45:0/4036343526 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4df4017070 con 0x558e1e39b830 2023-10-17T01:27:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4de0ff9700 1 -- 172.21.15.45:0/4036343526 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df401ecb0 con 0x558e1e39b830 2023-10-17T01:27:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4e133a8b80 10 client.26794 fetch_fsmap learned FSMap version 42 2023-10-17T01:27:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.304+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558e1e103c00 con 0x558e1e39b830 2023-10-17T01:27:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4de0ff9700 1 -- 172.21.15.45:0/4036343526 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df4015070 con 0x558e1e39b830 2023-10-17T01:27:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4de0ff9700 1 -- 172.21.15.45:0/4036343526 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4df4026c80 con 0x558e1e39b830 2023-10-17T01:27:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4e133a8b80 10 client.26794 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:27:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4e133a8b80 10 client.26794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:27:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4e133a8b80 20 client.26794 populate_metadata read hostname 'smithi045' 2023-10-17T01:27:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e403840 0x558e1e405c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:27:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4e133a8b80 4 client.26794 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:27:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e1e2e5ce0 con 0x558e1e403840 2023-10-17T01:27:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.305+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e403840 0x558e1e405c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:27:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.306+0000 7f4dfcf9a700 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e403840 0x558e1e405c60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc77278133ac2291 server_cookie=3e5591b34e4bbba0 in_seq=0 out_seq=0 2023-10-17T01:27:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.306+0000 7f4de0ff9700 10 client.26794 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.316+0000 7f4de0ff9700 1 -- 172.21.15.45:0/4036343526 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e1e2e5ce0 con 0x558e1e403840 2023-10-17T01:27:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.316+0000 7f4de0ff9700 10 client.26794 handle_command_reply: tid=0 2023-10-17T01:27:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.316+0000 7f4e133a8b80 1 client.26794 shutdown 2023-10-17T01:27:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.316+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 msgr2=0x558e1e3fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.316+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 0x558e1e3fcc30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4df400bf20 tx=0x7f4df4003930 comp rx=0 tx=0).stop 2023-10-17T01:27:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 shutdown_connections 2023-10-17T01:27:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558e1e403840 0x558e1e405c60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e1e39cd90 0x558e1e4017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e1e39b830 0x558e1e3fcc30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 --2- 172.21.15.45:0/4036343526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e1e39c190 0x558e1e3ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4de0ff9700 0 client.26794 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:27:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 >> 172.21.15.45:0/4036343526 conn(0x558e1e2fb070 msgr2=0x558e1e3a31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 shutdown_connections 2023-10-17T01:27:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 1 -- 172.21.15.45:0/4036343526 wait complete. 2023-10-17T01:27:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.317+0000 7f4e133a8b80 20 client.26794 trim_cache size 0 max 16384 2023-10-17T01:27:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:27:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:27:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:27:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:27:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4de40a0b20 msgr2=0x7f4de40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4de40a0b20 0x7f4de40a2fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4df8194590 tx=0x7f4df400b500 comp rx=0 tx=0).stop 2023-10-17T01:27:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8106a90 msgr2=0x7f4df8195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:27:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8106a90 0x7f4df8195c30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009f90 tx=0x7f4df0004c60 comp rx=0 tx=0).stop 2023-10-17T01:27:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 shutdown_connections 2023-10-17T01:27:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f4de40a0b20 0x7f4de40a2fe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df8106a90 0x7f4df8195c30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df8101830 0x7f4df8194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 --2- 172.21.15.45:0/3006485819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df8104160 0x7f4df8194d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:27:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 >> 172.21.15.45:0/3006485819 conn(0x7f4df80fb1a0 msgr2=0x7f4df80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:27:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 shutdown_connections 2023-10-17T01:27:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:27:58.318+0000 7f4dff9ff700 1 -- 172.21.15.45:0/3006485819 wait complete. 2023-10-17T01:27:59.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:27:58 smithi181 ceph-mon[134520]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 MiB/s wr, 656 op/s 2023-10-17T01:27:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:27:58 smithi045 ceph-mon[98474]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 MiB/s wr, 656 op/s 2023-10-17T01:27:59.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:27:58 smithi060 ceph-mon[134136]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.4 MiB/s wr, 656 op/s 2023-10-17T01:28:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 770.0 s, 1426.6 tps, lat 68.353 ms stddev 87.347 2023-10-17T01:28:01.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:00 smithi181 ceph-mon[134520]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 MiB/s wr, 699 op/s 2023-10-17T01:28:01.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:00 smithi045 ceph-mon[98474]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 MiB/s wr, 699 op/s 2023-10-17T01:28:01.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:00 smithi060 ceph-mon[134136]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 MiB/s wr, 699 op/s 2023-10-17T01:28:03.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:02 smithi181 ceph-mon[134520]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 672 op/s 2023-10-17T01:28:03.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:02 smithi045 ceph-mon[98474]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 672 op/s 2023-10-17T01:28:03.333 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:02 smithi060 ceph-mon[134136]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 672 op/s 2023-10-17T01:28:05.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:04 smithi181 ceph-mon[134520]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 806 op/s 2023-10-17T01:28:05.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:04 smithi045 ceph-mon[98474]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 806 op/s 2023-10-17T01:28:05.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:04 smithi060 ceph-mon[134136]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 806 op/s 2023-10-17T01:28:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 775.0 s, 1497.0 tps, lat 68.015 ms stddev 80.152 2023-10-17T01:28:07.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:06 smithi181 ceph-mon[134520]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-10-17T01:28:07.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:06 smithi045 ceph-mon[98474]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-10-17T01:28:07.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:06 smithi060 ceph-mon[134136]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2023-10-17T01:28:08.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:28:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 -- 172.21.15.45:0/1441621647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818108560 msgr2=0x7f28181089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 --2- 172.21.15.45:0/1441621647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818108560 0x7f28181089c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28000099c0 tx=0x7f2800009cd0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 -- 172.21.15.45:0/1441621647 shutdown_connections 2023-10-17T01:28:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 --2- 172.21.15.45:0/1441621647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2818108f00 0x7f281810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 --2- 172.21.15.45:0/1441621647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818107370 0x7f2818107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 --2- 172.21.15.45:0/1441621647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818108560 0x7f28181089c0 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28000099c0 tx=0x7f2800009cd0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 -- 172.21.15.45:0/1441621647 >> 172.21.15.45:0/1441621647 conn(0x7f28180762f0 msgr2=0x7f2818078750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 -- 172.21.15.45:0/1441621647 shutdown_connections 2023-10-17T01:28:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.956+0000 7f281efff700 1 -- 172.21.15.45:0/1441621647 wait complete. 2023-10-17T01:28:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.958+0000 7f281efff700 1 Processor -- start 2023-10-17T01:28:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.958+0000 7f281efff700 1 -- start start 2023-10-17T01:28:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.960+0000 7f281efff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 0x7f281819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.960+0000 7f281efff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818108f00 0x7f281819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.960+0000 7f281efff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28181a2890 0x7f28181a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 0x7f281819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f280ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818108f00 0x7f281819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 0x7f281819d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33482/0 (socket says 172.21.15.45:33482) 2023-10-17T01:28:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281efff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28181a5250 con 0x7f2818108f00 2023-10-17T01:28:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 -- 172.21.15.45:0/3812926381 learned_addr learned my addr 172.21.15.45:0/3812926381 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28181a5390 con 0x7f2818107370 2023-10-17T01:28:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28181a5520 con 0x7f28181a2890 2023-10-17T01:28:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281d59c700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28181a2890 0x7f28181a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 -- 172.21.15.45:0/3812926381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28181a2890 msgr2=0x7f28181a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28181a2890 0x7f28181a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 -- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818108f00 msgr2=0x7f281819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818108f00 0x7f281819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f281cd9b700 1 -- 172.21.15.45:0/3812926381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000095f0 con 0x7f2818107370 2023-10-17T01:28:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.961+0000 7f280ffff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818108f00 0x7f281819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.962+0000 7f281cd9b700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 0x7f281819d2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2808007f10 tx=0x7f280800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.962+0000 7f28167fc700 1 -- 172.21.15.45:0/3812926381 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808017070 con 0x7f2818107370 2023-10-17T01:28:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.962+0000 7f28167fc700 1 -- 172.21.15.45:0/3812926381 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f280800f040 con 0x7f2818107370 2023-10-17T01:28:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.962+0000 7f28167fc700 1 -- 172.21.15.45:0/3812926381 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808004630 con 0x7f2818107370 2023-10-17T01:28:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.962+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28181a5800 con 0x7f2818107370 2023-10-17T01:28:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.962+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28181a5cc0 con 0x7f2818107370 2023-10-17T01:28:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.969+0000 7f28167fc700 1 -- 172.21.15.45:0/3812926381 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2808004790 con 0x7f2818107370 2023-10-17T01:28:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.971+0000 7f28167fc700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f28040a0c70 0x7f28040a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.971+0000 7f283296ab80 1 Processor -- start 2023-10-17T01:28:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.971+0000 7f28167fc700 1 -- 172.21.15.45:0/3812926381 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f28080cb770 con 0x7f2818107370 2023-10-17T01:28:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.972+0000 7f283296ab80 1 -- start start 2023-10-17T01:28:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.972+0000 7f280ffff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f28040a0c70 0x7f28040a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.972+0000 7f283296ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5879fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.972+0000 7f283296ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f587ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.972+0000 7f283296ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f587b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.972+0000 7f281cd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5879fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f281cd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5879fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33488/0 (socket says 172.21.15.45:33488) 2023-10-17T01:28:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f280ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f587ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f280ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f587ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49230/0 (socket says 172.21.15.45:49230) 2023-10-17T01:28:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f281cd9b700 1 -- 172.21.15.45:0/1884324516 learned_addr learned my addr 172.21.15.45:0/1884324516 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f57d1a10 con 0x5624f587a500 2023-10-17T01:28:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f281d59c700 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f587b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f57cf3d0 con 0x5624f5879ba0 2023-10-17T01:28:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.973+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f57cacc0 con 0x5624f587b2b0 2023-10-17T01:28:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f280ffff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f28040a0c70 0x7f28040a3130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f280000b580 tx=0x7f28000058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f281d59c700 1 -- 172.21.15.45:0/1884324516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 msgr2=0x5624f5879fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f281d59c700 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5879fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f281d59c700 1 -- 172.21.15.45:0/1884324516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 msgr2=0x5624f587ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f281d59c700 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f587ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f281d59c700 1 -- 172.21.15.45:0/1884324516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f55f8de0 con 0x5624f587b2b0 2023-10-17T01:28:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.974+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5879fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.975+0000 7f281d59c700 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f587b730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2810009b80 tx=0x7f2810009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59332bb0f8279642 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.976+0000 7f280effd700 1 -- 172.21.15.45:0/1884324516 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810011ad0 con 0x5624f587b2b0 2023-10-17T01:28:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.976+0000 7f280effd700 1 -- 172.21.15.45:0/1884324516 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2810011c30 con 0x5624f587b2b0 2023-10-17T01:28:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.976+0000 7f280effd700 1 -- 172.21.15.45:0/1884324516 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000f6a0 con 0x5624f587b2b0 2023-10-17T01:28:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.977+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 msgr2=0x5624f587b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.977+0000 7f283296ab80 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f587b730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2810009b80 tx=0x7f2810009e90 comp rx=0 tx=0).stop 2023-10-17T01:28:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 shutdown_connections 2023-10-17T01:28:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f587b730 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5879fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 --2- 172.21.15.45:0/1884324516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f587ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 >> 172.21.15.45:0/1884324516 conn(0x5624f57d90d0 msgr2=0x5624f5874950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 shutdown_connections 2023-10-17T01:28:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.978+0000 7f283296ab80 1 -- 172.21.15.45:0/1884324516 wait complete. 2023-10-17T01:28:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.979+0000 7f283296ab80 1 -- start start 2023-10-17T01:28:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.979+0000 7f283296ab80 1 Processor -- start 2023-10-17T01:28:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5883880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 0x5624f5883dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f588ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f5767510 con 0x5624f5879ba0 2023-10-17T01:28:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f57671b0 con 0x5624f587a500 2023-10-17T01:28:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f564e920 con 0x5624f587b2b0 2023-10-17T01:28:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f283296ab80 10 client.0 _ll_register_callbacks cb 0x5624f56b4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f280ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 0x5624f5883dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f280ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 0x5624f5883dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33504/0 (socket says 172.21.15.45:33504) 2023-10-17T01:28:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.980+0000 7f280ffff700 1 -- 172.21.15.45:0/1595457502 learned_addr learned my addr 172.21.15.45:0/1595457502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.981+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5883880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.981+0000 7f280ffff700 1 -- 172.21.15.45:0/1595457502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 msgr2=0x5624f588ae00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:28:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.981+0000 7f280ffff700 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f588ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.981+0000 7f280ffff700 1 -- 172.21.15.45:0/1595457502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 msgr2=0x5624f5883880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.981+0000 7f280ffff700 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5883880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.981+0000 7f280ffff700 1 -- 172.21.15.45:0/1595457502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28181a5cc0 con 0x5624f587a500 2023-10-17T01:28:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.982+0000 7f280ffff700 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 0x5624f5883dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28000099c0 tx=0x7f2800005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442bc9e71cfade4c server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.982+0000 7f27f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:28:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.982+0000 7f27f7fff700 1 -- 172.21.15.45:0/1595457502 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28000249c0 con 0x5624f587a500 2023-10-17T01:28:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.982+0000 7f27f7fff700 1 -- 172.21.15.45:0/1595457502 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2800024b20 con 0x5624f587a500 2023-10-17T01:28:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.982+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624f57f5d10 con 0x5624f587a500 2023-10-17T01:28:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.983+0000 7f27f7fff700 1 -- 172.21.15.45:0/1595457502 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2800024ce0 con 0x5624f587a500 2023-10-17T01:28:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.983+0000 7f283296ab80 10 client.26846 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.983+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f2810009830 con 0x5624f587a500 2023-10-17T01:28:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.983+0000 7f27f7fff700 1 -- 172.21.15.45:0/1595457502 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624f57f5d10 con 0x5624f587a500 2023-10-17T01:28:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f27f7fff700 1 -- 172.21.15.45:0/1595457502 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2800010470 con 0x5624f587a500 2023-10-17T01:28:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f283296ab80 10 client.26846 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f283296ab80 10 client.26846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f283296ab80 20 client.26846 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f283296ab80 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f588f700 0x5624f5891b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f283296ab80 4 client.26846 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:28:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.984+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f5848d30 con 0x5624f588f700 2023-10-17T01:28:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.985+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f588f700 0x5624f5891b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.986+0000 7f27f7fff700 10 client.26846 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.987+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f588f700 0x5624f5891b20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73441ab92b876c6 server_cookie=db8ee7c417ebfe01 in_seq=0 out_seq=0 2023-10-17T01:28:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.988+0000 7f27f7fff700 1 -- 172.21.15.45:0/1595457502 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5624f5848d30 con 0x5624f588f700 2023-10-17T01:28:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.988+0000 7f27f7fff700 10 client.26846 handle_command_reply: tid=0 2023-10-17T01:28:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.988+0000 7f283296ab80 1 client.26846 shutdown 2023-10-17T01:28:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 msgr2=0x5624f5883dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 0x5624f5883dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28000099c0 tx=0x7f2800005050 comp rx=0 tx=0).stop 2023-10-17T01:28:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 shutdown_connections 2023-10-17T01:28:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f588ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5883880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f588f700 0x5624f5891b20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 --2- 172.21.15.45:0/1595457502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587a500 0x5624f5883dc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 >> 172.21.15.45:0/1595457502 conn(0x5624f57d90d0 msgr2=0x5624f5874020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f27f7fff700 0 client.26846 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 shutdown_connections 2023-10-17T01:28:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.989+0000 7f283296ab80 1 -- 172.21.15.45:0/1595457502 wait complete. 2023-10-17T01:28:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:08.990+0000 7f283296ab80 20 client.26846 trim_cache size 0 max 16384 2023-10-17T01:28:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.025+0000 7f283296ab80 1 Processor -- start 2023-10-17T01:28:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 -- start start 2023-10-17T01:28:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5882590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587a500 0x5624f5875650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 0x5624f5874020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f58833f0 con 0x5624f5879ba0 2023-10-17T01:28:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f5871420 con 0x5624f587b2b0 2023-10-17T01:28:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.026+0000 7f283296ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f588f400 con 0x5624f587a500 2023-10-17T01:28:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 0x5624f5874020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 0x5624f5874020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33510/0 (socket says 172.21.15.45:33510) 2023-10-17T01:28:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 -- 172.21.15.45:0/577068295 learned_addr learned my addr 172.21.15.45:0/577068295 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f280ffff700 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587a500 0x5624f5875650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 -- 172.21.15.45:0/577068295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587a500 msgr2=0x5624f5875650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587a500 0x5624f5875650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 -- 172.21.15.45:0/577068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 msgr2=0x5624f5882590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5882590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.027+0000 7f281d59c700 1 -- 172.21.15.45:0/577068295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f55e1c60 con 0x5624f587b2b0 2023-10-17T01:28:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.028+0000 7f281d59c700 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 0x5624f5874020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2810009b80 tx=0x7f2810005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681ed0045d5be033 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.028+0000 7f280f7fe700 1 -- 172.21.15.45:0/577068295 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810011da0 con 0x5624f587b2b0 2023-10-17T01:28:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.028+0000 7f280f7fe700 1 -- 172.21.15.45:0/577068295 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2810005580 con 0x5624f587b2b0 2023-10-17T01:28:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.028+0000 7f280f7fe700 1 -- 172.21.15.45:0/577068295 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810012770 con 0x5624f587b2b0 2023-10-17T01:28:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 -- 172.21.15.45:0/577068295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 msgr2=0x5624f5874020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 0x5624f5874020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2810009b80 tx=0x7f2810005e20 comp rx=0 tx=0).stop 2023-10-17T01:28:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 -- 172.21.15.45:0/577068295 shutdown_connections 2023-10-17T01:28:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f587b2b0 0x5624f5874020 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f5879ba0 0x5624f5882590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 --2- 172.21.15.45:0/577068295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587a500 0x5624f5875650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 -- 172.21.15.45:0/577068295 >> 172.21.15.45:0/577068295 conn(0x5624f57d90d0 msgr2=0x5624f58748c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.029+0000 7f283296ab80 1 -- 172.21.15.45:0/577068295 shutdown_connections 2023-10-17T01:28:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.030+0000 7f283296ab80 1 -- 172.21.15.45:0/577068295 wait complete. 2023-10-17T01:28:09.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.030+0000 7f283296ab80 1 -- start start 2023-10-17T01:28:09.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.032+0000 7f283296ab80 1 Processor -- start 2023-10-17T01:28:09.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.033+0000 7f283296ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5886b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.033+0000 7f283296ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f5887040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f283296ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f5888050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f283296ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f5888590 con 0x5624f587a500 2023-10-17T01:28:09.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f283296ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f5888700 con 0x5624f5879ba0 2023-10-17T01:28:09.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f283296ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f58888c0 con 0x5624f587b2b0 2023-10-17T01:28:09.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f283296ab80 10 client.0 _ll_register_callbacks cb 0x7f2800005480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:09.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f5888050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f5888050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46066/0 (socket says 172.21.15.45:46066) 2023-10-17T01:28:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 -- 172.21.15.45:0/1860922109 learned_addr learned my addr 172.21.15.45:0/1860922109 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f280ffff700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f5887040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5886b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 -- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 msgr2=0x5624f5886b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5886b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 -- 172.21.15.45:0/1860922109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 msgr2=0x5624f5887040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f5887040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281d59c700 1 -- 172.21.15.45:0/1860922109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624f55f8de0 con 0x5624f587b2b0 2023-10-17T01:28:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.034+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5886b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:28:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.035+0000 7f281d59c700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f5888050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2810005e90 tx=0x7f2810011fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22b5ef85225b358 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.035+0000 7f27f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:28:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.035+0000 7f27f7fff700 1 -- 172.21.15.45:0/1860922109 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810003a90 con 0x5624f587b2b0 2023-10-17T01:28:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.035+0000 7f27f7fff700 1 -- 172.21.15.45:0/1860922109 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f28100118d0 con 0x5624f587b2b0 2023-10-17T01:28:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.035+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2810011c30 con 0x5624f587b2b0 2023-10-17T01:28:09.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.036+0000 7f27f7fff700 1 -- 172.21.15.45:0/1860922109 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2810011ac0 con 0x5624f587b2b0 2023-10-17T01:28:09.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.036+0000 7f283296ab80 10 client.26809 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:09.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.036+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5624f561f250 con 0x5624f587b2b0 2023-10-17T01:28:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f27f7fff700 1 -- 172.21.15.45:0/1860922109 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2810011c30 con 0x5624f587b2b0 2023-10-17T01:28:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f27f7fff700 1 -- 172.21.15.45:0/1860922109 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2810022070 con 0x5624f587b2b0 2023-10-17T01:28:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f283296ab80 10 client.26809 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f283296ab80 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f283296ab80 20 client.26809 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f283296ab80 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f58e1ce0 0x5624f58e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f283296ab80 4 client.26809 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:28:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.037+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624f57c3d40 con 0x5624f58e1ce0 2023-10-17T01:28:09.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.038+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f58e1ce0 0x5624f58e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:09.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.038+0000 7f281cd9b700 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f58e1ce0 0x5624f58e4100 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2a0437ce503c88 server_cookie=6ce98712fba2c4c in_seq=0 out_seq=0 2023-10-17T01:28:09.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.038+0000 7f27f7fff700 10 client.26809 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:09.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.039+0000 7f27f7fff700 1 -- 172.21.15.45:0/1860922109 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5624f57c3d40 con 0x5624f58e1ce0 2023-10-17T01:28:09.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.039+0000 7f27f7fff700 10 client.26809 handle_command_reply: tid=0 2023-10-17T01:28:09.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.039+0000 7f283296ab80 1 client.26809 shutdown 2023-10-17T01:28:09.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.040+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 msgr2=0x5624f5888050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:09.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.040+0000 7f283296ab80 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f5888050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2810005e90 tx=0x7f2810011fd0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 shutdown_connections 2023-10-17T01:28:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f283296ab80 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5624f587b2b0 0x5624f5888050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f283296ab80 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5624f58e1ce0 0x5624f58e4100 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f283296ab80 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624f5879ba0 0x5624f5886b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f283296ab80 1 --2- 172.21.15.45:0/1860922109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5624f587a500 0x5624f5887040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 >> 172.21.15.45:0/1860922109 conn(0x5624f57d90d0 msgr2=0x5624f5881680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.041+0000 7f27f7fff700 0 client.26809 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.042+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 shutdown_connections 2023-10-17T01:28:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.042+0000 7f283296ab80 1 -- 172.21.15.45:0/1860922109 wait complete. 2023-10-17T01:28:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.042+0000 7f283296ab80 20 client.26809 trim_cache size 0 max 16384 2023-10-17T01:28:09.055 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:28:09.056 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:28:09.056 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:28:09.056 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:28:09.057 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:28:09.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:28:09.057 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:28:09.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:28:09.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:28:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:28:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f28040a0c70 msgr2=0x7f28040a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f28040a0c70 0x7f28040a3130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f280000b580 tx=0x7f28000058e0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 msgr2=0x7f281819d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 0x7f281819d2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2808007f10 tx=0x7f280800d3b0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 shutdown_connections 2023-10-17T01:28:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f28040a0c70 0x7f28040a3130 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28181a2890 0x7f28181a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818107370 0x7f281819d2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 --2- 172.21.15.45:0/3812926381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2818108f00 0x7f281819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.043+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 >> 172.21.15.45:0/3812926381 conn(0x7f28180762f0 msgr2=0x7f2818078660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:09.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.044+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 shutdown_connections 2023-10-17T01:28:09.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:09.044+0000 7f281efff700 1 -- 172.21.15.45:0/3812926381 wait complete. 2023-10-17T01:28:09.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:08 smithi045 ceph-mon[98474]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 664 op/s 2023-10-17T01:28:09.081 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:28:09.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:08 smithi181 ceph-mon[134520]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 664 op/s 2023-10-17T01:28:09.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:08 smithi060 ceph-mon[134136]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 664 op/s 2023-10-17T01:28:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 780.0 s, 1593.4 tps, lat 62.341 ms stddev 68.479 2023-10-17T01:28:11.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:10 smithi181 ceph-mon[134520]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-17T01:28:11.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:10 smithi045 ceph-mon[98474]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-17T01:28:11.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:10 smithi060 ceph-mon[134136]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-17T01:28:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 -- 172.21.15.45:0/1574348236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600102fb0 msgr2=0x7f86001033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 --2- 172.21.15.45:0/1574348236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600102fb0 0x7f86001033d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85f40099c0 tx=0x7f85f4009cd0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 -- 172.21.15.45:0/1574348236 shutdown_connections 2023-10-17T01:28:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 --2- 172.21.15.45:0/1574348236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8600104b40 0x7f860010b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 --2- 172.21.15.45:0/1574348236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600102fb0 0x7f86001033d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 --2- 172.21.15.45:0/1574348236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86001041a0 0x7f8600104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 -- 172.21.15.45:0/1574348236 >> 172.21.15.45:0/1574348236 conn(0x7f86000fe500 msgr2=0x7f8600100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 -- 172.21.15.45:0/1574348236 shutdown_connections 2023-10-17T01:28:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.535+0000 7f8606fff700 1 -- 172.21.15.45:0/1574348236 wait complete. 2023-10-17T01:28:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.536+0000 7f8606fff700 1 Processor -- start 2023-10-17T01:28:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.538+0000 7f8606fff700 1 -- start start 2023-10-17T01:28:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f8606fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8600102fb0 0x7f8600198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f8606fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86001041a0 0x7f86001993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f8606fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600104b40 0x7f860019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f8606fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019a870 con 0x7f8600102fb0 2023-10-17T01:28:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f8606fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019a9e0 con 0x7f8600104b40 2023-10-17T01:28:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f8606fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019aba0 con 0x7f86001041a0 2023-10-17T01:28:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f860559c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600104b40 0x7f860019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f85fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86001041a0 0x7f86001993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f860559c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600104b40 0x7f860019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33528/0 (socket says 172.21.15.45:33528) 2023-10-17T01:28:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.539+0000 7f860559c700 1 -- 172.21.15.45:0/3336994149 learned_addr learned my addr 172.21.15.45:0/3336994149 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.540+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8600102fb0 0x7f8600198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.540+0000 7f85fffff700 1 -- 172.21.15.45:0/3336994149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600104b40 msgr2=0x7f860019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.540+0000 7f85fffff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600104b40 0x7f860019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.540+0000 7f85fffff700 1 -- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8600102fb0 msgr2=0x7f8600198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.540+0000 7f85fffff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8600102fb0 0x7f8600198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.540+0000 7f85fffff700 1 -- 172.21.15.45:0/3336994149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40095f0 con 0x7f86001041a0 2023-10-17T01:28:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.541+0000 7f85fffff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86001041a0 0x7f86001993c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00e9f0 tx=0x7f85ec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.541+0000 7f85fdffb700 1 -- 172.21.15.45:0/3336994149 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0096d0 con 0x7f86001041a0 2023-10-17T01:28:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.541+0000 7f85fdffb700 1 -- 172.21.15.45:0/3336994149 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85ec00cbd0 con 0x7f86001041a0 2023-10-17T01:28:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.541+0000 7f85fdffb700 1 -- 172.21.15.45:0/3336994149 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec018680 con 0x7f86001041a0 2023-10-17T01:28:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.541+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860019ae80 con 0x7f86001041a0 2023-10-17T01:28:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.542+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86001a1590 con 0x7f86001041a0 2023-10-17T01:28:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.547+0000 7f85fdffb700 1 -- 172.21.15.45:0/3336994149 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f85ec00cd40 con 0x7f86001041a0 2023-10-17T01:28:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.547+0000 7f861a8a7b80 1 Processor -- start 2023-10-17T01:28:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.548+0000 7f861a8a7b80 1 -- start start 2023-10-17T01:28:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.550+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.550+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b98227790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.550+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98228210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.550+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9817e9b0 con 0x558b98227d90 2023-10-17T01:28:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.550+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9817c370 con 0x558b98227190 2023-10-17T01:28:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.550+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b98177c60 con 0x558b98226830 2023-10-17T01:28:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.551+0000 7f85fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98228210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.551+0000 7f85fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98228210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49308/0 (socket says 172.21.15.45:49308) 2023-10-17T01:28:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.551+0000 7f85fffff700 1 -- 172.21.15.45:0/417110372 learned_addr learned my addr 172.21.15.45:0/417110372 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.551+0000 7f85fdffb700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85f00a0cf0 0x7f85f00a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.551+0000 7f85fffff700 1 -- 172.21.15.45:0/417110372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 msgr2=0x558b98226c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:28:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.552+0000 7f85fffff700 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.552+0000 7f85fffff700 1 -- 172.21.15.45:0/417110372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 msgr2=0x558b98227790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.552+0000 7f85fffff700 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b98227790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.552+0000 7f85fffff700 1 -- 172.21.15.45:0/417110372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b97fa5d80 con 0x558b98227d90 2023-10-17T01:28:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.552+0000 7f85fdffb700 1 -- 172.21.15.45:0/3336994149 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f85ec014070 con 0x7f86001041a0 2023-10-17T01:28:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.552+0000 7f85fffff700 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98228210 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f85ec07bb10 tx=0x7f85ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d7aa56008408a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.553+0000 7f85eaffd700 1 -- 172.21.15.45:0/417110372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0993e0 con 0x558b98227d90 2023-10-17T01:28:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.553+0000 7f85eaffd700 1 -- 172.21.15.45:0/417110372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85ec099200 con 0x558b98227d90 2023-10-17T01:28:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.553+0000 7f85eaffd700 1 -- 172.21.15.45:0/417110372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec099930 con 0x558b98227d90 2023-10-17T01:28:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.554+0000 7f861a8a7b80 1 -- 172.21.15.45:0/417110372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 msgr2=0x558b98228210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.554+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98228210 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f85ec07bb10 tx=0x7f85ec005880 comp rx=0 tx=0).stop 2023-10-17T01:28:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.557+0000 7f860559c700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85f00a0cf0 0x7f85f00a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.558+0000 7f861a8a7b80 1 -- 172.21.15.45:0/417110372 shutdown_connections 2023-10-17T01:28:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.558+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98228210 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.558+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.558+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/417110372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b98227790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.558+0000 7f861a8a7b80 1 -- 172.21.15.45:0/417110372 >> 172.21.15.45:0/417110372 conn(0x558b98186070 msgr2=0x558b982212c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.560+0000 7f860559c700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85f00a0cf0 0x7f85f00a31b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009c10 tx=0x7f85f800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.560+0000 7f861a8a7b80 1 -- 172.21.15.45:0/417110372 shutdown_connections 2023-10-17T01:28:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.560+0000 7f861a8a7b80 1 -- 172.21.15.45:0/417110372 wait complete. 2023-10-17T01:28:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.561+0000 7f861a8a7b80 1 -- start start 2023-10-17T01:28:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.561+0000 7f861a8a7b80 1 Processor -- start 2023-10-17T01:28:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b98249a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b98249f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98250fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b981144b0 con 0x558b98227d90 2023-10-17T01:28:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b98114150 con 0x558b98226830 2023-10-17T01:28:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b97ffb8c0 con 0x558b98227190 2023-10-17T01:28:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.562+0000 7f861a8a7b80 10 client.0 _ll_register_callbacks cb 0x558b98061a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.563+0000 7f8604d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b98249a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.563+0000 7f8604d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b98249a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33540/0 (socket says 172.21.15.45:33540) 2023-10-17T01:28:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.563+0000 7f8604d9b700 1 -- 172.21.15.45:0/3204729008 learned_addr learned my addr 172.21.15.45:0/3204729008 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.563+0000 7f860559c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98250fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.564+0000 7f8604d9b700 1 -- 172.21.15.45:0/3204729008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 msgr2=0x558b98249f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.564+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b98249f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.564+0000 7f8604d9b700 1 -- 172.21.15.45:0/3204729008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 msgr2=0x558b98250fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.564+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98250fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.564+0000 7f8604d9b700 1 -- 172.21.15.45:0/3204729008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86001a1590 con 0x558b98226830 2023-10-17T01:28:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.570+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b98249a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006010 tx=0x7f85f400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5f79091a89465d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.570+0000 7f85dffff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:28:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.573+0000 7f85dffff700 1 -- 172.21.15.45:0/3204729008 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f401d040 con 0x558b98226830 2023-10-17T01:28:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.573+0000 7f85dffff700 1 -- 172.21.15.45:0/3204729008 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f85f40054e0 con 0x558b98226830 2023-10-17T01:28:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.573+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b981a2cb0 con 0x558b98226830 2023-10-17T01:28:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.582+0000 7f85dffff700 1 -- 172.21.15.45:0/3204729008 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f40056a0 con 0x558b98226830 2023-10-17T01:28:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.582+0000 7f861a8a7b80 10 client.26858 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.582+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558b97f8ec00 con 0x558b98226830 2023-10-17T01:28:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.587+0000 7f85dffff700 1 -- 172.21.15.45:0/3204729008 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b981a2cb0 con 0x558b98226830 2023-10-17T01:28:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f85dffff700 1 -- 172.21.15.45:0/3204729008 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85f400f960 con 0x558b98226830 2023-10-17T01:28:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f861a8a7b80 10 client.26858 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f861a8a7b80 10 client.26858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f861a8a7b80 20 client.26858 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982558e0 0x558b98257d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f861a8a7b80 4 client.26858 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:28:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b981f5cd0 con 0x558b982558e0 2023-10-17T01:28:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.588+0000 7f85fffff700 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982558e0 0x558b98257d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.590+0000 7f85fffff700 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982558e0 0x558b98257d00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d836ca328962639 server_cookie=7b3c71f48dcad4d4 in_seq=0 out_seq=0 2023-10-17T01:28:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.590+0000 7f85dffff700 10 client.26858 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.591+0000 7f85dffff700 1 -- 172.21.15.45:0/3204729008 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x558b981f5cd0 con 0x558b982558e0 2023-10-17T01:28:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.591+0000 7f85dffff700 10 client.26858 handle_command_reply: tid=0 2023-10-17T01:28:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.592+0000 7f861a8a7b80 1 client.26858 shutdown 2023-10-17T01:28:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.592+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 msgr2=0x558b98249a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b98249a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006010 tx=0x7f85f400bea0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 shutdown_connections 2023-10-17T01:28:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b98250fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982558e0 0x558b98257d00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b98249a30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3204729008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b98249f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f85dffff700 0 client.26858 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.595+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 >> 172.21.15.45:0/3204729008 conn(0x558b98186070 msgr2=0x558b98220a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.596+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 shutdown_connections 2023-10-17T01:28:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.596+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3204729008 wait complete. 2023-10-17T01:28:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.596+0000 7f861a8a7b80 20 client.26858 trim_cache size 0 max 16384 2023-10-17T01:28:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 Processor -- start 2023-10-17T01:28:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 -- start start 2023-10-17T01:28:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b9823ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b9822acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9821e010 con 0x558b98227d90 2023-10-17T01:28:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b982406d0 con 0x558b98227190 2023-10-17T01:28:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.627+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b9823fb90 con 0x558b98226830 2023-10-17T01:28:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b9823ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b9823ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33556/0 (socket says 172.21.15.45:33556) 2023-10-17T01:28:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 -- 172.21.15.45:0/2895606743 learned_addr learned my addr 172.21.15.45:0/2895606743 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f8604d9b700 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f860559c700 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b9822acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 -- 172.21.15.45:0/2895606743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 msgr2=0x558b98226c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 -- 172.21.15.45:0/2895606743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 msgr2=0x558b9822acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b9822acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 -- 172.21.15.45:0/2895606743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b97fcc1f0 con 0x558b98227190 2023-10-17T01:28:11.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f860559c700 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b9822acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:28:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85fffff700 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b9823ffa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0c9da0 tx=0x7f85ec0b9110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6208494f13ea4aa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85eb7fe700 1 -- 172.21.15.45:0/2895606743 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec099420 con 0x558b98227190 2023-10-17T01:28:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85eb7fe700 1 -- 172.21.15.45:0/2895606743 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85ec0b9180 con 0x558b98227190 2023-10-17T01:28:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.628+0000 7f85eb7fe700 1 -- 172.21.15.45:0/2895606743 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0998a0 con 0x558b98227190 2023-10-17T01:28:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 -- 172.21.15.45:0/2895606743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 msgr2=0x558b9823ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b9823ffa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0c9da0 tx=0x7f85ec0b9110 comp rx=0 tx=0).stop 2023-10-17T01:28:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 -- 172.21.15.45:0/2895606743 shutdown_connections 2023-10-17T01:28:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b9822acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98226830 0x558b98226c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/2895606743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98227190 0x558b9823ffa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.629+0000 7f861a8a7b80 1 -- 172.21.15.45:0/2895606743 >> 172.21.15.45:0/2895606743 conn(0x558b98186070 msgr2=0x558b9822e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.630+0000 7f861a8a7b80 1 -- 172.21.15.45:0/2895606743 shutdown_connections 2023-10-17T01:28:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.630+0000 7f861a8a7b80 1 -- 172.21.15.45:0/2895606743 wait complete. 2023-10-17T01:28:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.630+0000 7f861a8a7b80 1 -- start start 2023-10-17T01:28:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.630+0000 7f861a8a7b80 1 Processor -- start 2023-10-17T01:28:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b982a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b982a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b982a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b982a65a0 con 0x558b98227d90 2023-10-17T01:28:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b982a6710 con 0x558b98226830 2023-10-17T01:28:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b982a68d0 con 0x558b98227190 2023-10-17T01:28:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f861a8a7b80 10 client.0 _ll_register_callbacks cb 0x7f85ec005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f85fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b982a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f85fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b982a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:46106/0 (socket says 172.21.15.45:46106) 2023-10-17T01:28:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f85fffff700 1 -- 172.21.15.45:0/3137460392 learned_addr learned my addr 172.21.15.45:0/3137460392 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.632+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b982a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f860559c700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b982a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f85fffff700 1 -- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 msgr2=0x558b982a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f85fffff700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b982a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f85fffff700 1 -- 172.21.15.45:0/3137460392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 msgr2=0x558b982a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f85fffff700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b982a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f85fffff700 1 -- 172.21.15.45:0/3137460392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b97fa5d80 con 0x558b98227190 2023-10-17T01:28:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.633+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b982a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f85fffff700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b982a37b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c5b0 tx=0x7f85ec099470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c48bea5e697733e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f85dffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:28:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f85dffff700 1 -- 172.21.15.45:0/3137460392 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec099fa0 con 0x558b98227190 2023-10-17T01:28:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f85dffff700 1 -- 172.21.15.45:0/3137460392 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f85ec0998a0 con 0x558b98227190 2023-10-17T01:28:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85ec099200 con 0x558b98227190 2023-10-17T01:28:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f85dffff700 1 -- 172.21.15.45:0/3137460392 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85ec0b5080 con 0x558b98227190 2023-10-17T01:28:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f861a8a7b80 10 client.26821 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558b97fa4250 con 0x558b98227190 2023-10-17T01:28:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.634+0000 7f85dffff700 1 -- 172.21.15.45:0/3137460392 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85ec099200 con 0x558b98227190 2023-10-17T01:28:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f85dffff700 1 -- 172.21.15.45:0/3137460392 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85ec099b00 con 0x558b98227190 2023-10-17T01:28:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f861a8a7b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f861a8a7b80 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f861a8a7b80 20 client.26821 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982a7e00 0x558b982aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f861a8a7b80 4 client.26821 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:28:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b98170ce0 con 0x558b982a7e00 2023-10-17T01:28:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.635+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982a7e00 0x558b982aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.636+0000 7f8604d9b700 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982a7e00 0x558b982aa220 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7c29d03b4c9546 server_cookie=78186ac735b4b79a in_seq=0 out_seq=0 2023-10-17T01:28:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.636+0000 7f85dffff700 10 client.26821 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.645+0000 7f85dffff700 1 -- 172.21.15.45:0/3137460392 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b98170ce0 con 0x558b982a7e00 2023-10-17T01:28:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.645+0000 7f85dffff700 10 client.26821 handle_command_reply: tid=0 2023-10-17T01:28:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.646+0000 7f861a8a7b80 1 client.26821 shutdown 2023-10-17T01:28:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 msgr2=0x558b982a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b982a37b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c5b0 tx=0x7f85ec099470 comp rx=0 tx=0).stop 2023-10-17T01:28:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 shutdown_connections 2023-10-17T01:28:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b98227d90 0x558b982a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x558b982a7e00 0x558b982aa220 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558b98226830 0x558b982a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 --2- 172.21.15.45:0/3137460392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b98227190 0x558b982a37b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.647+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 >> 172.21.15.45:0/3137460392 conn(0x558b98186070 msgr2=0x558b9822e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.648+0000 7f85dffff700 0 client.26821 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.648+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 shutdown_connections 2023-10-17T01:28:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.648+0000 7f861a8a7b80 1 -- 172.21.15.45:0/3137460392 wait complete. 2023-10-17T01:28:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.648+0000 7f861a8a7b80 20 client.26821 trim_cache size 0 max 16384 2023-10-17T01:28:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:28:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:28:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:28:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:28:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85f00a0cf0 msgr2=0x7f85f00a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85f00a0cf0 0x7f85f00a31b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009c10 tx=0x7f85f800b040 comp rx=0 tx=0).stop 2023-10-17T01:28:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86001041a0 msgr2=0x7f86001993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86001041a0 0x7f86001993c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00e9f0 tx=0x7f85ec00ed00 comp rx=0 tx=0).stop 2023-10-17T01:28:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 shutdown_connections 2023-10-17T01:28:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f85f00a0cf0 0x7f85f00a31b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8600104b40 0x7f860019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8600102fb0 0x7f8600198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 --2- 172.21.15.45:0/3336994149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86001041a0 0x7f86001993c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 >> 172.21.15.45:0/3336994149 conn(0x7f86000fe500 msgr2=0x7f86001076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 shutdown_connections 2023-10-17T01:28:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:11.649+0000 7f8606fff700 1 -- 172.21.15.45:0/3336994149 wait complete. 2023-10-17T01:28:13.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:12 smithi181 ceph-mon[134520]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 629 op/s 2023-10-17T01:28:13.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:12 smithi045 ceph-mon[98474]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 629 op/s 2023-10-17T01:28:13.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:12 smithi060 ceph-mon[134136]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 MiB/s wr, 629 op/s 2023-10-17T01:28:15.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:14 smithi181 ceph-mon[134520]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 MiB/s wr, 728 op/s 2023-10-17T01:28:15.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:14 smithi045 ceph-mon[98474]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 MiB/s wr, 728 op/s 2023-10-17T01:28:15.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:14 smithi060 ceph-mon[134136]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 MiB/s wr, 728 op/s 2023-10-17T01:28:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 785.0 s, 1415.2 tps, lat 71.439 ms stddev 100.150 2023-10-17T01:28:17.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:16 smithi045 ceph-mon[98474]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 560 op/s 2023-10-17T01:28:17.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:16 smithi060 ceph-mon[134136]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 560 op/s 2023-10-17T01:28:17.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:16 smithi181 ceph-mon[134520]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 560 op/s 2023-10-17T01:28:19.169 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:18 smithi181 ceph-mon[134520]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 639 op/s 2023-10-17T01:28:19.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:18 smithi045 ceph-mon[98474]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 639 op/s 2023-10-17T01:28:19.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:18 smithi060 ceph-mon[134136]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 639 op/s 2023-10-17T01:28:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 790.0 s, 1492.8 tps, lat 66.112 ms stddev 77.416 2023-10-17T01:28:21.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:20 smithi181 ceph-mon[134520]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 620 op/s 2023-10-17T01:28:21.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:20 smithi045 ceph-mon[98474]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 620 op/s 2023-10-17T01:28:21.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:20 smithi060 ceph-mon[134136]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 620 op/s 2023-10-17T01:28:23.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:22 smithi181 ceph-mon[134520]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 569 op/s 2023-10-17T01:28:23.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:22 smithi045 ceph-mon[98474]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 569 op/s 2023-10-17T01:28:23.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:22 smithi060 ceph-mon[134136]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 MiB/s wr, 569 op/s 2023-10-17T01:28:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.849+0000 7f10f11ff700 1 -- 172.21.15.45:0/3024940430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec074d90 msgr2=0x7f10ec0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.849+0000 7f10f11ff700 1 --2- 172.21.15.45:0/3024940430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec074d90 0x7f10ec0731e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009990 tx=0x7f10dc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.850+0000 7f10f11ff700 1 -- 172.21.15.45:0/3024940430 shutdown_connections 2023-10-17T01:28:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.850+0000 7f10f11ff700 1 --2- 172.21.15.45:0/3024940430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ec108c20 0x7f10ec10b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.850+0000 7f10f11ff700 1 --2- 172.21.15.45:0/3024940430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec074d90 0x7f10ec0731e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.850+0000 7f10f11ff700 1 --2- 172.21.15.45:0/3024940430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec0737b0 0x7f10ec073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.850+0000 7f10f11ff700 1 -- 172.21.15.45:0/3024940430 >> 172.21.15.45:0/3024940430 conn(0x7f10ec0fc290 msgr2=0x7f10ec0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.850+0000 7f10f11ff700 1 -- 172.21.15.45:0/3024940430 shutdown_connections 2023-10-17T01:28:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.851+0000 7f10f11ff700 1 -- 172.21.15.45:0/3024940430 wait complete. 2023-10-17T01:28:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 Processor -- start 2023-10-17T01:28:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 -- start start 2023-10-17T01:28:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ec0737b0 0x7f10ec198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 0x7f10ec199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec108c20 0x7f10ec19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec19a7b0 con 0x7f10ec0737b0 2023-10-17T01:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec19a920 con 0x7f10ec108c20 2023-10-17T01:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.852+0000 7f10f11ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec19aae0 con 0x7f10ec074d90 2023-10-17T01:28:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 0x7f10ec199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10eb59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec108c20 0x7f10ec19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 0x7f10ec199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36230/0 (socket says 172.21.15.45:36230) 2023-10-17T01:28:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 -- 172.21.15.45:0/1449291915 learned_addr learned my addr 172.21.15.45:0/1449291915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ead9d700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ec0737b0 0x7f10ec198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 -- 172.21.15.45:0/1449291915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec108c20 msgr2=0x7f10ec19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec108c20 0x7f10ec19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 -- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ec0737b0 msgr2=0x7f10ec198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ec0737b0 0x7f10ec198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10ea59c700 1 -- 172.21.15.45:0/1449291915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc0095f0 con 0x7f10ec074d90 2023-10-17T01:28:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.853+0000 7f10eb59e700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec108c20 0x7f10ec19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.854+0000 7f10ea59c700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 0x7f10ec199300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f10e000e970 tx=0x7f10e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.854+0000 7f10d3fff700 1 -- 172.21.15.45:0/1449291915 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0009790 con 0x7f10ec074d90 2023-10-17T01:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.854+0000 7f10d3fff700 1 -- 172.21.15.45:0/1449291915 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e000c760 con 0x7f10ec074d90 2023-10-17T01:28:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.854+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec19adc0 con 0x7f10ec074d90 2023-10-17T01:28:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.855+0000 7f10d3fff700 1 -- 172.21.15.45:0/1449291915 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0003bb0 con 0x7f10ec074d90 2023-10-17T01:28:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.855+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec1a1550 con 0x7f10ec074d90 2023-10-17T01:28:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.858+0000 7f10d3fff700 1 -- 172.21.15.45:0/1449291915 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f10e0003d50 con 0x7f10ec074d90 2023-10-17T01:28:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.860+0000 7f10d3fff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f10d40a0ca0 0x7f10d40a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.860+0000 7f10d3fff700 1 -- 172.21.15.45:0/1449291915 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f10e0014070 con 0x7f10ec074d90 2023-10-17T01:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.861+0000 7f1104b1bb80 1 Processor -- start 2023-10-17T01:28:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.861+0000 7f10ead9d700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f10d40a0ca0 0x7f10d40a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.862+0000 7f1104b1bb80 1 -- start start 2023-10-17T01:28:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f10ead9d700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f10d40a0ca0 0x7f10d40a3160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000c00 tx=0x7f10dc00b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f44bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f44bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bfd90 0x55d5f44c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f44169b0 con 0x55d5f44be830 2023-10-17T01:28:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f4414370 con 0x55d5f44bfd90 2023-10-17T01:28:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.863+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f440fc60 con 0x55d5f44bf190 2023-10-17T01:28:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f44bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f44bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f44bf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36246/0 (socket says 172.21.15.45:36246) 2023-10-17T01:28:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 -- 172.21.15.45:0/988784951 learned_addr learned my addr 172.21.15.45:0/988784951 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 -- 172.21.15.45:0/988784951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bfd90 msgr2=0x55d5f44c0210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:28:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bfd90 0x55d5f44c0210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 -- 172.21.15.45:0/988784951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 msgr2=0x55d5f44bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f44bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.864+0000 7f10ea59c700 1 -- 172.21.15.45:0/988784951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5f423dd80 con 0x55d5f44bf190 2023-10-17T01:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.865+0000 7f10ead9d700 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f44bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.865+0000 7f10ea59c700 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f44bf790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f10e0096240 tx=0x7f10e00991b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99816d0d9a74d29 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.865+0000 7f10d17fa700 1 -- 172.21.15.45:0/988784951 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e009d600 con 0x55d5f44bf190 2023-10-17T01:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.865+0000 7f10d17fa700 1 -- 172.21.15.45:0/988784951 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e00cb040 con 0x55d5f44bf190 2023-10-17T01:28:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.865+0000 7f10d17fa700 1 -- 172.21.15.45:0/988784951 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0095ee0 con 0x55d5f44bf190 2023-10-17T01:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 -- 172.21.15.45:0/988784951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 msgr2=0x55d5f44bf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f44bf790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f10e0096240 tx=0x7f10e00991b0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 -- 172.21.15.45:0/988784951 shutdown_connections 2023-10-17T01:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bfd90 0x55d5f44c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f44bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/988784951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f44bf790 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 -- 172.21.15.45:0/988784951 >> 172.21.15.45:0/988784951 conn(0x55d5f441e070 msgr2=0x55d5f44b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.866+0000 7f1104b1bb80 1 -- 172.21.15.45:0/988784951 shutdown_connections 2023-10-17T01:28:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.867+0000 7f1104b1bb80 1 -- 172.21.15.45:0/988784951 wait complete. 2023-10-17T01:28:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.867+0000 7f1104b1bb80 1 -- start start 2023-10-17T01:28:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.867+0000 7f1104b1bb80 1 Processor -- start 2023-10-17T01:28:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44bfd90 0x55d5f44cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f43ac4b0 con 0x55d5f44bfd90 2023-10-17T01:28:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f43ac150 con 0x55d5f44bf190 2023-10-17T01:28:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f42938c0 con 0x55d5f44be830 2023-10-17T01:28:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:36250/0 (socket says 172.21.15.45:36250) 2023-10-17T01:28:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f10ead9d700 1 -- 172.21.15.45:0/2036508589 learned_addr learned my addr 172.21.15.45:0/2036508589 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f10ea59c700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f1104b1bb80 10 client.0 _ll_register_callbacks cb 0x55d5f42f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.868+0000 7f10eb59e700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44bfd90 0x55d5f44cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.869+0000 7f10ea59c700 1 -- 172.21.15.45:0/2036508589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 msgr2=0x55d5f44c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.869+0000 7f10ea59c700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.869+0000 7f10ea59c700 1 -- 172.21.15.45:0/2036508589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44bfd90 msgr2=0x55d5f44cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.869+0000 7f10ea59c700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44bfd90 0x55d5f44cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.869+0000 7f10ea59c700 1 -- 172.21.15.45:0/2036508589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec10bcd0 con 0x55d5f44bf190 2023-10-17T01:28:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.869+0000 7f10eb59e700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44bfd90 0x55d5f44cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:28:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f10ea59c700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c8940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55d5f44be5d0 tx=0x7f10e00b9180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45af0716b56ff9f server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f10ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:28:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f10ceff5700 1 -- 172.21.15.45:0/2036508589 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0099c30 con 0x55d5f44bf190 2023-10-17T01:28:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f10ceff5700 1 -- 172.21.15.45:0/2036508589 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f10e009d600 con 0x55d5f44bf190 2023-10-17T01:28:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5f443acb0 con 0x55d5f44bf190 2023-10-17T01:28:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f10ceff5700 1 -- 172.21.15.45:0/2036508589 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e0098cc0 con 0x55d5f44bf190 2023-10-17T01:28:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f1104b1bb80 10 client.26873 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.871+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f10ec1a1550 con 0x55d5f44bf190 2023-10-17T01:28:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.872+0000 7f10ceff5700 1 -- 172.21.15.45:0/2036508589 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5f443acb0 con 0x55d5f44bf190 2023-10-17T01:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.872+0000 7f10ceff5700 1 -- 172.21.15.45:0/2036508589 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10e00b7bc0 con 0x55d5f44bf190 2023-10-17T01:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f1104b1bb80 10 client.26873 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f1104b1bb80 10 client.26873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f1104b1bb80 20 client.26873 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44d3ed0 0x55d5f44d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f1104b1bb80 4 client.26873 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:28:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5f448dcd0 con 0x55d5f44d3ed0 2023-10-17T01:28:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f10ead9d700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44d3ed0 0x55d5f44d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f10ead9d700 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44d3ed0 0x55d5f44d62f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bab333b1ab49fbc server_cookie=2596e26613784d57 in_seq=0 out_seq=0 2023-10-17T01:28:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.873+0000 7f10ceff5700 10 client.26873 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.874+0000 7f10ceff5700 1 -- 172.21.15.45:0/2036508589 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55d5f448dcd0 con 0x55d5f44d3ed0 2023-10-17T01:28:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.874+0000 7f10ceff5700 10 client.26873 handle_command_reply: tid=0 2023-10-17T01:28:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.874+0000 7f1104b1bb80 1 client.26873 shutdown 2023-10-17T01:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.875+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 msgr2=0x55d5f44c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.875+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c8940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55d5f44be5d0 tx=0x7f10e00b9180 comp rx=0 tx=0).stop 2023-10-17T01:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.875+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 shutdown_connections 2023-10-17T01:28:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.876+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44bfd90 0x55d5f44cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.876+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.876+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44d3ed0 0x55d5f44d62f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.876+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/2036508589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c8940 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.876+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 >> 172.21.15.45:0/2036508589 conn(0x55d5f441e070 msgr2=0x55d5f44b8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.876+0000 7f10ceff5700 0 client.26873 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.877+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 shutdown_connections 2023-10-17T01:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.877+0000 7f1104b1bb80 1 -- 172.21.15.45:0/2036508589 wait complete. 2023-10-17T01:28:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.878+0000 7f1104b1bb80 20 client.26873 trim_cache size 0 max 16384 2023-10-17T01:28:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.914+0000 7f1104b1bb80 1 Processor -- start 2023-10-17T01:28:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.914+0000 7f1104b1bb80 1 -- start start 2023-10-17T01:28:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.914+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 0x55d5f44d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.914+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.914+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.914+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f44bad40 con 0x55d5f441e070 2023-10-17T01:28:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.915+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f44b39e0 con 0x55d5f44bf190 2023-10-17T01:28:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.915+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f44b6010 con 0x55d5f44be830 2023-10-17T01:28:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.916+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 0x55d5f44d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.916+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 0x55d5f44d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36934/0 (socket says 172.21.15.45:36934) 2023-10-17T01:28:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.916+0000 7f10ead9d700 1 -- 172.21.15.45:0/22904145 learned_addr learned my addr 172.21.15.45:0/22904145 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.918+0000 7f10ead9d700 1 -- 172.21.15.45:0/22904145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 msgr2=0x55d5f44c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.918+0000 7f10ead9d700 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.918+0000 7f10ead9d700 1 -- 172.21.15.45:0/22904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 msgr2=0x55d5f44c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.918+0000 7f10ead9d700 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.918+0000 7f10ead9d700 1 -- 172.21.15.45:0/22904145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5f4226c00 con 0x55d5f441e070 2023-10-17T01:28:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.918+0000 7f10ead9d700 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 0x55d5f44d01b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0095c0 tx=0x7f10dc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97f6f4c54d1378 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.920+0000 7f10d1ffb700 1 -- 172.21.15.45:0/22904145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc02b040 con 0x55d5f441e070 2023-10-17T01:28:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.920+0000 7f10d1ffb700 1 -- 172.21.15.45:0/22904145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10dc02c040 con 0x55d5f441e070 2023-10-17T01:28:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.920+0000 7f10d1ffb700 1 -- 172.21.15.45:0/22904145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc004d40 con 0x55d5f441e070 2023-10-17T01:28:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.921+0000 7f1104b1bb80 1 -- 172.21.15.45:0/22904145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 msgr2=0x55d5f44d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.921+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 0x55d5f44d01b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0095c0 tx=0x7f10dc020000 comp rx=0 tx=0).stop 2023-10-17T01:28:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 -- 172.21.15.45:0/22904145 shutdown_connections 2023-10-17T01:28:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f44bf190 0x55d5f44c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f441e070 0x55d5f44d01b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/22904145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44be830 0x55d5f44c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 -- 172.21.15.45:0/22904145 >> 172.21.15.45:0/22904145 conn(0x55d5f44b95f0 msgr2=0x55d5f44d5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 -- 172.21.15.45:0/22904145 shutdown_connections 2023-10-17T01:28:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 -- 172.21.15.45:0/22904145 wait complete. 2023-10-17T01:28:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.922+0000 7f1104b1bb80 1 -- start start 2023-10-17T01:28:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.923+0000 7f1104b1bb80 1 Processor -- start 2023-10-17T01:28:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.923+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f441e070 0x55d5f4530800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.923+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f4532d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.923+0000 7f1104b1bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f45352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.924+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f441e070 0x55d5f4530800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.924+0000 7f10ea59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f4532d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.924+0000 7f10ead9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f441e070 0x55d5f4530800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:54232/0 (socket says 172.21.15.45:54232) 2023-10-17T01:28:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.924+0000 7f10ea59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f4532d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36944/0 (socket says 172.21.15.45:36944) 2023-10-17T01:28:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f4535960 con 0x55d5f44be830 2023-10-17T01:28:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f4535ad0 con 0x55d5f441e070 2023-10-17T01:28:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f1104b1bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f4535c90 con 0x55d5f44bf190 2023-10-17T01:28:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f1104b1bb80 10 client.0 _ll_register_callbacks cb 0x7f10dc009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f10ead9d700 1 -- 172.21.15.45:0/3432100698 learned_addr learned my addr 172.21.15.45:0/3432100698 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f10ea59c700 1 -- 172.21.15.45:0/3432100698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 msgr2=0x55d5f45352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f10ea59c700 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f45352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f10ea59c700 1 -- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f441e070 msgr2=0x55d5f4530800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f10ea59c700 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f441e070 0x55d5f4530800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.925+0000 7f10ea59c700 1 -- 172.21.15.45:0/3432100698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc0051d0 con 0x55d5f44be830 2023-10-17T01:28:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.926+0000 7f10ea59c700 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f4532d50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10e00961f0 tx=0x7f10e001ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f17096746790ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.926+0000 7f10ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:28:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.926+0000 7f10ceff5700 1 -- 172.21.15.45:0/3432100698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e009d600 con 0x55d5f44be830 2023-10-17T01:28:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.927+0000 7f10ceff5700 1 -- 172.21.15.45:0/3432100698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f10e0099a30 con 0x55d5f44be830 2023-10-17T01:28:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.927+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10e00cb040 con 0x55d5f44be830 2023-10-17T01:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.928+0000 7f10ceff5700 1 -- 172.21.15.45:0/3432100698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e0099bf0 con 0x55d5f44be830 2023-10-17T01:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.928+0000 7f1104b1bb80 10 client.16521 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.928+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d5f42641f0 con 0x55d5f44be830 2023-10-17T01:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f10ceff5700 1 -- 172.21.15.45:0/3432100698 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e00a6020 con 0x55d5f44be830 2023-10-17T01:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f10ceff5700 1 -- 172.21.15.45:0/3432100698 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10e009d250 con 0x55d5f44be830 2023-10-17T01:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f1104b1bb80 10 client.16521 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f1104b1bb80 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f1104b1bb80 20 client.16521 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44bfd90 0x55d5f4539240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f1104b1bb80 4 client.16521 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.929+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5f4408ce0 con 0x55d5f44bfd90 2023-10-17T01:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.930+0000 7f10ead9d700 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44bfd90 0x55d5f4539240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.965+0000 7f10ead9d700 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44bfd90 0x55d5f4539240 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f704219c04685a9b server_cookie=110b37962ab0267e in_seq=0 out_seq=0 2023-10-17T01:28:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.965+0000 7f10ceff5700 10 client.16521 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.998+0000 7f10ceff5700 1 -- 172.21.15.45:0/3432100698 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5f4408ce0 con 0x55d5f44bfd90 2023-10-17T01:28:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.998+0000 7f10ceff5700 10 client.16521 handle_command_reply: tid=0 2023-10-17T01:28:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.998+0000 7f1104b1bb80 1 client.16521 shutdown 2023-10-17T01:28:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 msgr2=0x55d5f4532d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f4532d50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10e00961f0 tx=0x7f10e001ffe0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 shutdown_connections 2023-10-17T01:28:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d5f44bf190 0x55d5f45352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55d5f44bfd90 0x55d5f4539240 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f441e070 0x55d5f4530800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 --2- 172.21.15.45:0/3432100698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5f44be830 0x55d5f4532d50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 >> 172.21.15.45:0/3432100698 conn(0x55d5f44b95f0 msgr2=0x55d5f44d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f10ceff5700 0 client.16521 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 shutdown_connections 2023-10-17T01:28:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:24.999+0000 7f1104b1bb80 1 -- 172.21.15.45:0/3432100698 wait complete. 2023-10-17T01:28:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f1104b1bb80 20 client.16521 trim_cache size 0 max 16384 2023-10-17T01:28:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:28:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:28:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:28:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:28:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f10d40a0ca0 msgr2=0x7f10d40a3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f10d40a0ca0 0x7f10d40a3160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000c00 tx=0x7f10dc00b280 comp rx=0 tx=0).stop 2023-10-17T01:28:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 msgr2=0x7f10ec199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 0x7f10ec199300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f10e000e970 tx=0x7f10e000ec80 comp rx=0 tx=0).stop 2023-10-17T01:28:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 shutdown_connections 2023-10-17T01:28:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f10d40a0ca0 0x7f10d40a3160 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10ec108c20 0x7f10ec19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10ec0737b0 0x7f10ec198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 --2- 172.21.15.45:0/1449291915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec074d90 0x7f10ec199300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.000+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 >> 172.21.15.45:0/1449291915 conn(0x7f10ec0fc290 msgr2=0x7f10ec0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.001+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 shutdown_connections 2023-10-17T01:28:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:25.001+0000 7f10f11ff700 1 -- 172.21.15.45:0/1449291915 wait complete. 2023-10-17T01:28:25.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:24 smithi181 ceph-mon[134520]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 MiB/s wr, 727 op/s 2023-10-17T01:28:25.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:24 smithi045 ceph-mon[98474]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 MiB/s wr, 727 op/s 2023-10-17T01:28:25.335 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:24 smithi060 ceph-mon[134136]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 MiB/s wr, 727 op/s 2023-10-17T01:28:25.649 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 795.0 s, 1412.0 tps, lat 70.866 ms stddev 87.041 2023-10-17T01:28:28.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:27 smithi181 ceph-mon[134520]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 567 op/s 2023-10-17T01:28:28.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:27 smithi045 ceph-mon[98474]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 567 op/s 2023-10-17T01:28:28.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:27 smithi060 ceph-mon[134136]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 567 op/s 2023-10-17T01:28:30.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:29 smithi181 ceph-mon[134520]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.5 MiB/s wr, 604 op/s 2023-10-17T01:28:30.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:29 smithi045 ceph-mon[98474]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.5 MiB/s wr, 604 op/s 2023-10-17T01:28:30.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:29 smithi060 ceph-mon[134136]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.5 MiB/s wr, 604 op/s 2023-10-17T01:28:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 800.0 s, 1240.2 tps, lat 80.303 ms stddev 110.975 2023-10-17T01:28:31.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:30 smithi045 ceph-mon[98474]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.3 MiB/s wr, 666 op/s 2023-10-17T01:28:31.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:30 smithi060 ceph-mon[134136]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.3 MiB/s wr, 666 op/s 2023-10-17T01:28:31.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:30 smithi181 ceph-mon[134520]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.3 MiB/s wr, 666 op/s 2023-10-17T01:28:33.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:32 smithi181 ceph-mon[134520]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 662 op/s 2023-10-17T01:28:33.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:32 smithi045 ceph-mon[98474]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 662 op/s 2023-10-17T01:28:33.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:32 smithi060 ceph-mon[134136]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 662 op/s 2023-10-17T01:28:35.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:34 smithi181 ceph-mon[134520]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 13 MiB/s wr, 894 op/s 2023-10-17T01:28:35.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:34 smithi045 ceph-mon[98474]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 13 MiB/s wr, 894 op/s 2023-10-17T01:28:35.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:34 smithi060 ceph-mon[134136]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 13 MiB/s wr, 894 op/s 2023-10-17T01:28:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 805.0 s, 936.8 tps, lat 107.029 ms stddev 119.494 2023-10-17T01:28:37.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:36 smithi045 ceph-mon[98474]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-17T01:28:37.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:36 smithi060 ceph-mon[134136]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-17T01:28:37.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:36 smithi181 ceph-mon[134520]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-17T01:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 -- 172.21.15.45:0/3643257902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 msgr2=0x7f67c0103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3643257902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0103c70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009990 tx=0x7f67b0009ca0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 -- 172.21.15.45:0/3643257902 shutdown_connections 2023-10-17T01:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3643257902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c0106ae0 0x7f67c0108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3643257902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0103c70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3643257902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c01041b0 0x7f67c01065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 -- 172.21.15.45:0/3643257902 >> 172.21.15.45:0/3643257902 conn(0x7f67c00fb210 msgr2=0x7f67c00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 -- 172.21.15.45:0/3643257902 shutdown_connections 2023-10-17T01:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.243+0000 7f67c5dff700 1 -- 172.21.15.45:0/3643257902 wait complete. 2023-10-17T01:28:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.248+0000 7f67c5dff700 1 Processor -- start 2023-10-17T01:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.248+0000 7f67c5dff700 1 -- start start 2023-10-17T01:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.248+0000 7f67c5dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0192090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.248+0000 7f67c5dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c01041b0 0x7f67c01925d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.248+0000 7f67c5dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c0106ae0 0x7f67c01934a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0192090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0192090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59396/0 (socket says 172.21.15.45:59396) 2023-10-17T01:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67c5dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0193b00 con 0x7f67c01041b0 2023-10-17T01:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0193c70 con 0x7f67c0101880 2023-10-17T01:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0193e30 con 0x7f67c0106ae0 2023-10-17T01:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 -- 172.21.15.45:0/3941637981 learned_addr learned my addr 172.21.15.45:0/3941637981 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67beffd700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c01041b0 0x7f67c01925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bffff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c0106ae0 0x7f67c01934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 -- 172.21.15.45:0/3941637981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c0106ae0 msgr2=0x7f67c01934a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c0106ae0 0x7f67c01934a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 -- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c01041b0 msgr2=0x7f67c01925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c01041b0 0x7f67c01925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bf7fe700 1 -- 172.21.15.45:0/3941637981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4009520 con 0x7f67c0101880 2023-10-17T01:28:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67bffff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c0106ae0 0x7f67c01934a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:28:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.249+0000 7f67beffd700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c01041b0 0x7f67c01925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:28:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.250+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0192090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f67b000b680 tx=0x7f67b0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.250+0000 7f67bcff9700 1 -- 172.21.15.45:0/3941637981 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b001d070 con 0x7f67c0101880 2023-10-17T01:28:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.250+0000 7f67bcff9700 1 -- 172.21.15.45:0/3941637981 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b000fbc0 con 0x7f67c0101880 2023-10-17T01:28:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.250+0000 7f67bcff9700 1 -- 172.21.15.45:0/3941637981 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0017650 con 0x7f67c0101880 2023-10-17T01:28:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.250+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00095f0 con 0x7f67c0101880 2023-10-17T01:28:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.250+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c01942f0 con 0x7f67c0101880 2023-10-17T01:28:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.256+0000 7f67bcff9700 1 -- 172.21.15.45:0/3941637981 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f67b0022470 con 0x7f67c0101880 2023-10-17T01:28:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.258+0000 7f67d9644b80 1 Processor -- start 2023-10-17T01:28:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.258+0000 7f67bcff9700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f67a80a0ba0 0x7f67a80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.258+0000 7f67d9644b80 1 -- start start 2023-10-17T01:28:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.258+0000 7f67bcff9700 1 -- 172.21.15.45:0/3941637981 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67b00cd820 con 0x7f67c0101880 2023-10-17T01:28:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bd0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67d9644b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9b289b0 con 0x55a7d9bd1190 2023-10-17T01:28:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67d9644b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9b26370 con 0x55a7d9bd0830 2023-10-17T01:28:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67d9644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9b21c60 con 0x55a7d9bd1d90 2023-10-17T01:28:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bffff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f67a80a0ba0 0x7f67a80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bd0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bd0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59404/0 (socket says 172.21.15.45:59404) 2023-10-17T01:28:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bf7fe700 1 -- 172.21.15.45:0/954543492 learned_addr learned my addr 172.21.15.45:0/954543492 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bf7fe700 1 -- 172.21.15.45:0/954543492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1d90 msgr2=0x55a7d9bd2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.259+0000 7f67beffd700 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67bf7fe700 1 -- 172.21.15.45:0/954543492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1190 msgr2=0x55a7d9bd1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67bf7fe700 1 -- 172.21.15.45:0/954543492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d994fd80 con 0x55a7d9bd0830 2023-10-17T01:28:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bd0c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67b00979a0 tx=0x7f67b0005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c3a22ffc8dc59a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67beffd700 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:28:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67a5ffb700 1 -- 172.21.15.45:0/954543492 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b009ed60 con 0x55a7d9bd0830 2023-10-17T01:28:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67a5ffb700 1 -- 172.21.15.45:0/954543492 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b00a2f10 con 0x55a7d9bd0830 2023-10-17T01:28:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.260+0000 7f67a5ffb700 1 -- 172.21.15.45:0/954543492 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00a2a80 con 0x55a7d9bd0830 2023-10-17T01:28:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.261+0000 7f67d9644b80 1 -- 172.21.15.45:0/954543492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 msgr2=0x55a7d9bd0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.261+0000 7f67d9644b80 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bd0c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67b00979a0 tx=0x7f67b0005ad0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.262+0000 7f67bffff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f67a80a0ba0 0x7f67a80a3060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f67b8015e20 tx=0x7f67b8015420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.262+0000 7f67d9644b80 1 -- 172.21.15.45:0/954543492 shutdown_connections 2023-10-17T01:28:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.262+0000 7f67d9644b80 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.262+0000 7f67d9644b80 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bd0c50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.262+0000 7f67d9644b80 1 --2- 172.21.15.45:0/954543492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.262+0000 7f67d9644b80 1 -- 172.21.15.45:0/954543492 >> 172.21.15.45:0/954543492 conn(0x55a7d9b30070 msgr2=0x55a7d9bcb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.263+0000 7f67d9644b80 1 -- 172.21.15.45:0/954543492 shutdown_connections 2023-10-17T01:28:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.264+0000 7f67d9644b80 1 -- 172.21.15.45:0/954543492 wait complete. 2023-10-17T01:28:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.265+0000 7f67d9644b80 1 -- start start 2023-10-17T01:28:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.265+0000 7f67d9644b80 1 Processor -- start 2023-10-17T01:28:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9bda070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bda5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9be15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9abe4b0 con 0x55a7d9bd0830 2023-10-17T01:28:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9abe150 con 0x55a7d9bd1d90 2023-10-17T01:28:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d99a58c0 con 0x55a7d9bd1190 2023-10-17T01:28:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67d9644b80 10 client.0 _ll_register_callbacks cb 0x55a7d9a0ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9be15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9bda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9be15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59420/0 (socket says 172.21.15.45:59420) 2023-10-17T01:28:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67bffff700 1 -- 172.21.15.45:0/522606496 learned_addr learned my addr 172.21.15.45:0/522606496 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.266+0000 7f67beffd700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bda5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bf7fe700 1 -- 172.21.15.45:0/522606496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 msgr2=0x55a7d9bda5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bda5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bf7fe700 1 -- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 msgr2=0x55a7d9be15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9be15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bf7fe700 1 -- 172.21.15.45:0/522606496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c01942f0 con 0x55a7d9bd0830 2023-10-17T01:28:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bffff700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9be15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:28:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9bda070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005910 tx=0x7f67b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3d7acee99dcc7b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.267+0000 7f679effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:28:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.268+0000 7f679effd700 1 -- 172.21.15.45:0/522606496 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00a2c90 con 0x55a7d9bd0830 2023-10-17T01:28:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.268+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7d9b4ccb0 con 0x55a7d9bd0830 2023-10-17T01:28:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.268+0000 7f679effd700 1 -- 172.21.15.45:0/522606496 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67b00b3170 con 0x55a7d9bd0830 2023-10-17T01:28:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.268+0000 7f679effd700 1 -- 172.21.15.45:0/522606496 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b00b3330 con 0x55a7d9bd0830 2023-10-17T01:28:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.268+0000 7f67d9644b80 10 client.16530 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.268+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a7d9938c00 con 0x55a7d9bd0830 2023-10-17T01:28:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f679effd700 1 -- 172.21.15.45:0/522606496 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7d9b4ccb0 con 0x55a7d9bd0830 2023-10-17T01:28:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f679effd700 1 -- 172.21.15.45:0/522606496 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67b0096eb0 con 0x55a7d9bd0830 2023-10-17T01:28:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f67d9644b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f67d9644b80 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f67d9644b80 20 client.16530 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f67d9644b80 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9bf6990 0x55a7d9bf8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f67d9644b80 4 client.16530 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:28:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.269+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7d9b9fcd0 con 0x55a7d9bf6990 2023-10-17T01:28:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.270+0000 7f67beffd700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9bf6990 0x55a7d9bf8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.271+0000 7f67beffd700 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9bf6990 0x55a7d9bf8db0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6496ab3a2fca66 server_cookie=1351e42efd70580e in_seq=0 out_seq=0 2023-10-17T01:28:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.271+0000 7f679effd700 10 client.16530 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.272+0000 7f679effd700 1 -- 172.21.15.45:0/522606496 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55a7d9b9fcd0 con 0x55a7d9bf6990 2023-10-17T01:28:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.272+0000 7f679effd700 10 client.16530 handle_command_reply: tid=0 2023-10-17T01:28:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.272+0000 7f67d9644b80 1 client.16530 shutdown 2023-10-17T01:28:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 msgr2=0x55a7d9bda070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9bda070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005910 tx=0x7f67b00057f0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 shutdown_connections 2023-10-17T01:28:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9bf6990 0x55a7d9bf8db0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9be15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9bda070 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 --2- 172.21.15.45:0/522606496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 >> 172.21.15.45:0/522606496 conn(0x55a7d9b30070 msgr2=0x55a7d9bcaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.273+0000 7f679effd700 0 client.16530 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.274+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 shutdown_connections 2023-10-17T01:28:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.274+0000 7f67d9644b80 1 -- 172.21.15.45:0/522606496 wait complete. 2023-10-17T01:28:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.274+0000 7f67d9644b80 20 client.16530 trim_cache size 0 max 16384 2023-10-17T01:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.311+0000 7f67d9644b80 1 Processor -- start 2023-10-17T01:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.311+0000 7f67d9644b80 1 -- start start 2023-10-17T01:28:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bf2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67d9644b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9bcc1d0 con 0x55a7d9bd1d90 2023-10-17T01:28:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67d9644b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9b9fcd0 con 0x55a7d9bd0830 2023-10-17T01:28:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67d9644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9b4ccb0 con 0x55a7d9bd1190 2023-10-17T01:28:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bf2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67bffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.312+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bf2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59428/0 (socket says 172.21.15.45:59428) 2023-10-17T01:28:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.313+0000 7f67bf7fe700 1 -- 172.21.15.45:0/553332261 learned_addr learned my addr 172.21.15.45:0/553332261 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.313+0000 7f67bffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50620/0 (socket says 172.21.15.45:50620) 2023-10-17T01:28:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.314+0000 7f67bffff700 1 -- 172.21.15.45:0/553332261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 msgr2=0x55a7d9bd8200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-17T01:28:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.314+0000 7f67bffff700 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd8200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.314+0000 7f67bffff700 1 -- 172.21.15.45:0/553332261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 msgr2=0x55a7d9bf2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.314+0000 7f67bffff700 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bf2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.314+0000 7f67bffff700 1 -- 172.21.15.45:0/553332261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d99761f0 con 0x55a7d9bd1d90 2023-10-17T01:28:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.315+0000 7f67bffff700 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd4d50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67b8015940 tx=0x7f67b801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1c0a3fcd5d8d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.316+0000 7f67a67fc700 1 -- 172.21.15.45:0/553332261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800e040 con 0x55a7d9bd1d90 2023-10-17T01:28:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.316+0000 7f67a67fc700 1 -- 172.21.15.45:0/553332261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b8003c00 con 0x55a7d9bd1d90 2023-10-17T01:28:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.316+0000 7f67a67fc700 1 -- 172.21.15.45:0/553332261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8007760 con 0x55a7d9bd1d90 2023-10-17T01:28:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.317+0000 7f67d9644b80 1 -- 172.21.15.45:0/553332261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 msgr2=0x55a7d9bd4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.317+0000 7f67d9644b80 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd4d50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67b8015940 tx=0x7f67b801c000 comp rx=0 tx=0).stop 2023-10-17T01:28:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 -- 172.21.15.45:0/553332261 shutdown_connections 2023-10-17T01:28:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd1d90 0x55a7d9bd4d50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd0830 0x55a7d9bf2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 --2- 172.21.15.45:0/553332261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9bd8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 -- 172.21.15.45:0/553332261 >> 172.21.15.45:0/553332261 conn(0x55a7d9b30070 msgr2=0x55a7d9bd8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 -- 172.21.15.45:0/553332261 shutdown_connections 2023-10-17T01:28:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.319+0000 7f67d9644b80 1 -- 172.21.15.45:0/553332261 wait complete. 2023-10-17T01:28:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.320+0000 7f67d9644b80 1 -- start start 2023-10-17T01:28:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.320+0000 7f67d9644b80 1 Processor -- start 2023-10-17T01:28:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9c422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9c44840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9c46d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9c47480 con 0x55a7d9bd0830 2023-10-17T01:28:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9c475f0 con 0x55a7d9bd1d90 2023-10-17T01:28:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d9c477b0 con 0x55a7d9bd1190 2023-10-17T01:28:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67d9644b80 10 client.0 _ll_register_callbacks cb 0x7f67b8015970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9c46d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67beffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9c44840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67beffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9c44840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37438/0 (socket says 172.21.15.45:37438) 2023-10-17T01:28:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.321+0000 7f67beffd700 1 -- 172.21.15.45:0/1862718641 learned_addr learned my addr 172.21.15.45:0/1862718641 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9c422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67beffd700 1 -- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 msgr2=0x55a7d9c46d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67beffd700 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9c46d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67beffd700 1 -- 172.21.15.45:0/1862718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 msgr2=0x55a7d9c422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67beffd700 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9c422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67beffd700 1 -- 172.21.15.45:0/1862718641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b8009dc0 con 0x55a7d9bd1190 2023-10-17T01:28:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.322+0000 7f67beffd700 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9c44840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f67b40058f0 tx=0x7f67b4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984fdf9e66872718 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.323+0000 7f679effd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:28:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.323+0000 7f679effd700 1 -- 172.21.15.45:0/1862718641 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4017040 con 0x55a7d9bd1190 2023-10-17T01:28:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.324+0000 7f679effd700 1 -- 172.21.15.45:0/1862718641 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67b4011e30 con 0x55a7d9bd1190 2023-10-17T01:28:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.324+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67b00a2f10 con 0x55a7d9bd1190 2023-10-17T01:28:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.324+0000 7f679effd700 1 -- 172.21.15.45:0/1862718641 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b401c8c0 con 0x55a7d9bd1190 2023-10-17T01:28:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.325+0000 7f67d9644b80 10 client.26848 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.325+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a7d994e250 con 0x55a7d9bd1190 2023-10-17T01:28:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.325+0000 7f679effd700 1 -- 172.21.15.45:0/1862718641 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b00a2f10 con 0x55a7d9bd1190 2023-10-17T01:28:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f679effd700 1 -- 172.21.15.45:0/1862718641 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67b400fbc0 con 0x55a7d9bd1190 2023-10-17T01:28:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f67d9644b80 10 client.26848 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f67d9644b80 10 client.26848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f67d9644b80 20 client.26848 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f67d9644b80 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9c492b0 0x55a7d9c4b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f67d9644b80 4 client.26848 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:28:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.326+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7d9b1ace0 con 0x55a7d9c492b0 2023-10-17T01:28:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.327+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9c492b0 0x55a7d9c4b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.327+0000 7f679effd700 10 client.26848 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.327+0000 7f67bf7fe700 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9c492b0 0x55a7d9c4b6d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880d155c85b8d956 server_cookie=af1656df0aebfa3f in_seq=0 out_seq=0 2023-10-17T01:28:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.340+0000 7f679effd700 1 -- 172.21.15.45:0/1862718641 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7d9b1ace0 con 0x55a7d9c492b0 2023-10-17T01:28:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f679effd700 10 client.26848 handle_command_reply: tid=0 2023-10-17T01:28:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 client.26848 shutdown 2023-10-17T01:28:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 msgr2=0x55a7d9c44840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9c44840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f67b40058f0 tx=0x7f67b4011720 comp rx=0 tx=0).stop 2023-10-17T01:28:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 shutdown_connections 2023-10-17T01:28:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55a7d9c492b0 0x55a7d9c4b6d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a7d9bd1d90 0x55a7d9c46d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7d9bd0830 0x55a7d9c422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 --2- 172.21.15.45:0/1862718641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a7d9bd1190 0x55a7d9c44840 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f679effd700 0 client.26848 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.341+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 >> 172.21.15.45:0/1862718641 conn(0x55a7d9b30070 msgr2=0x55a7d9bd8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.342+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 shutdown_connections 2023-10-17T01:28:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.342+0000 7f67d9644b80 1 -- 172.21.15.45:0/1862718641 wait complete. 2023-10-17T01:28:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.342+0000 7f67d9644b80 20 client.26848 trim_cache size 0 max 16384 2023-10-17T01:28:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:28:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:28:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:28:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:28:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f67a80a0ba0 msgr2=0x7f67a80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f67a80a0ba0 0x7f67a80a3060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f67b8015e20 tx=0x7f67b8015420 comp rx=0 tx=0).stop 2023-10-17T01:28:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 msgr2=0x7f67c0192090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0192090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f67b000b680 tx=0x7f67b0005ef0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 shutdown_connections 2023-10-17T01:28:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f67a80a0ba0 0x7f67a80a3060 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c0106ae0 0x7f67c01934a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c0101880 0x7f67c0192090 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 --2- 172.21.15.45:0/3941637981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67c01041b0 0x7f67c01925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.344+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 >> 172.21.15.45:0/3941637981 conn(0x7f67c00fb210 msgr2=0x7f67c00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.346+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 shutdown_connections 2023-10-17T01:28:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:38.346+0000 7f67c5dff700 1 -- 172.21.15.45:0/3941637981 wait complete. 2023-10-17T01:28:39.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:28:39.158 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:38 smithi181 ceph-mon[134520]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 12 MiB/s wr, 768 op/s 2023-10-17T01:28:39.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:38 smithi045 ceph-mon[98474]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 12 MiB/s wr, 768 op/s 2023-10-17T01:28:39.334 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:38 smithi060 ceph-mon[134136]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 12 MiB/s wr, 768 op/s 2023-10-17T01:28:39.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 -- 172.21.15.45:0/1258852998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35241017d0 msgr2=0x7f3524108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 --2- 172.21.15.45:0/1258852998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35241017d0 0x7f3524108080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3514009f90 tx=0x7f351400b330 comp rx=0 tx=0).stop 2023-10-17T01:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 -- 172.21.15.45:0/1258852998 shutdown_connections 2023-10-17T01:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 --2- 172.21.15.45:0/1258852998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35241017d0 0x7f3524108080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 --2- 172.21.15.45:0/1258852998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35240ffc40 0x7f3524100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 --2- 172.21.15.45:0/1258852998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3524100e30 0x7f3524101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 -- 172.21.15.45:0/1258852998 >> 172.21.15.45:0/1258852998 conn(0x7f35240fb1b0 msgr2=0x7f35240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 -- 172.21.15.45:0/1258852998 shutdown_connections 2023-10-17T01:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.424+0000 7f352b1ff700 1 -- 172.21.15.45:0/1258852998 wait complete. 2023-10-17T01:28:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.425+0000 7f352b1ff700 1 Processor -- start 2023-10-17T01:28:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 -- start start 2023-10-17T01:28:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35240ffc40 0x7f35241059c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 0x7f3524105f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35241017d0 0x7f3524102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3524102cf0 con 0x7f35240ffc40 2023-10-17T01:28:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3524102e60 con 0x7f35241017d0 2023-10-17T01:28:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.426+0000 7f352b1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3524103020 con 0x7f3524100e30 2023-10-17T01:28:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 0x7f3524105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 0x7f3524105f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:37458/0 (socket says 172.21.15.45:37458) 2023-10-17T01:28:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 -- 172.21.15.45:0/3471067138 learned_addr learned my addr 172.21.15.45:0/3471067138 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35240ffc40 0x7f35241059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f352979c700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35241017d0 0x7f3524102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 -- 172.21.15.45:0/3471067138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35241017d0 msgr2=0x7f3524102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35241017d0 0x7f3524102550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 -- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35240ffc40 msgr2=0x7f35241059c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35240ffc40 0x7f35241059c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.427+0000 7f3523fff700 1 -- 172.21.15.45:0/3471067138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0095f0 con 0x7f3524100e30 2023-10-17T01:28:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.428+0000 7f3523fff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 0x7f3524105f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f350c00d950 tx=0x7f350c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.428+0000 7f3521ffb700 1 -- 172.21.15.45:0/3471067138 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c009940 con 0x7f3524100e30 2023-10-17T01:28:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.428+0000 7f3521ffb700 1 -- 172.21.15.45:0/3471067138 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f350c010460 con 0x7f3524100e30 2023-10-17T01:28:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.428+0000 7f3521ffb700 1 -- 172.21.15.45:0/3471067138 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00f620 con 0x7f3524100e30 2023-10-17T01:28:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.428+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3514009c40 con 0x7f3524100e30 2023-10-17T01:28:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.429+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3524198d90 con 0x7f3524100e30 2023-10-17T01:28:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.433+0000 7f3521ffb700 1 -- 172.21.15.45:0/3471067138 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f350c0108d0 con 0x7f3524100e30 2023-10-17T01:28:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 Processor -- start 2023-10-17T01:28:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 -- start start 2023-10-17T01:28:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f17d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f18730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3e6ea10 con 0x55c0a3f17500 2023-10-17T01:28:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3e6c3d0 con 0x55c0a3f182b0 2023-10-17T01:28:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.435+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3e67cc0 con 0x55c0a3f16ba0 2023-10-17T01:28:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.436+0000 7f3523fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f18730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.436+0000 7f3523fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f18730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59462/0 (socket says 172.21.15.45:59462) 2023-10-17T01:28:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.436+0000 7f3523fff700 1 -- 172.21.15.45:0/1937622767 learned_addr learned my addr 172.21.15.45:0/1937622767 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.436+0000 7f352979c700 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.436+0000 7f3528f9b700 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f17d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.437+0000 7f352979c700 1 -- 172.21.15.45:0/1937622767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 msgr2=0x55c0a3f18730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.437+0000 7f352979c700 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f18730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.437+0000 7f352979c700 1 -- 172.21.15.45:0/1937622767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 msgr2=0x55c0a3f17d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.437+0000 7f352979c700 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f17d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.437+0000 7f352979c700 1 -- 172.21.15.45:0/1937622767 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a3c95de0 con 0x55c0a3f16ba0 2023-10-17T01:28:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.438+0000 7f3528f9b700 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f17d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.438+0000 7f352979c700 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f16fc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3514007930 tx=0x7f35140092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d1ea75b1aa99a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.439+0000 7f351affd700 1 -- 172.21.15.45:0/1937622767 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400e040 con 0x55c0a3f16ba0 2023-10-17T01:28:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.439+0000 7f351affd700 1 -- 172.21.15.45:0/1937622767 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3514007420 con 0x55c0a3f16ba0 2023-10-17T01:28:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.439+0000 7f351affd700 1 -- 172.21.15.45:0/1937622767 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400bc50 con 0x55c0a3f16ba0 2023-10-17T01:28:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.439+0000 7f3521ffb700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35100a0ba0 0x7f35100a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f3521ffb700 1 -- 172.21.15.45:0/3471067138 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f350c0cbfb0 con 0x7f3524100e30 2023-10-17T01:28:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 -- 172.21.15.45:0/1937622767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 msgr2=0x55c0a3f16fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f16fc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3514007930 tx=0x7f35140092e0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35100a0ba0 0x7f35100a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 -- 172.21.15.45:0/1937622767 shutdown_connections 2023-10-17T01:28:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f18730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f16fc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/1937622767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f17d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.440+0000 7f353ebf6b80 1 -- 172.21.15.45:0/1937622767 >> 172.21.15.45:0/1937622767 conn(0x55c0a3e760d0 msgr2=0x55c0a3f11950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.441+0000 7f353ebf6b80 1 -- 172.21.15.45:0/1937622767 shutdown_connections 2023-10-17T01:28:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.441+0000 7f353ebf6b80 1 -- 172.21.15.45:0/1937622767 wait complete. 2023-10-17T01:28:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.441+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35100a0ba0 0x7f35100a3060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55c0a3f0c320 tx=0x7f351c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.441+0000 7f353ebf6b80 1 -- start start 2023-10-17T01:28:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.442+0000 7f353ebf6b80 1 Processor -- start 2023-10-17T01:28:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.442+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.442+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f17500 0x55c0a3f31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.442+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f182b0 0x55c0a3f38e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.442+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3e04510 con 0x55c0a3f182b0 2023-10-17T01:28:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.442+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3e041b0 con 0x55c0a3f16ba0 2023-10-17T01:28:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3ceb920 con 0x55c0a3f17500 2023-10-17T01:28:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f353ebf6b80 10 client.0 _ll_register_callbacks cb 0x55c0a3d51a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f3528f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f352979c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f182b0 0x55c0a3f38e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f3528f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f318d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59466/0 (socket says 172.21.15.45:59466) 2023-10-17T01:28:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f3528f9b700 1 -- 172.21.15.45:0/2997589694 learned_addr learned my addr 172.21.15.45:0/2997589694 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f3523fff700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f17500 0x55c0a3f31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f352979c700 1 -- 172.21.15.45:0/2997589694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f17500 msgr2=0x55c0a3f31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f352979c700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f17500 0x55c0a3f31e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f352979c700 1 -- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 msgr2=0x55c0a3f318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f352979c700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f318d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f352979c700 1 -- 172.21.15.45:0/2997589694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c009b50 con 0x55c0a3f182b0 2023-10-17T01:28:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.443+0000 7f3528f9b700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f318d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f352979c700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f182b0 0x55c0a3f38e50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3514009790 tx=0x7f3514007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0f01bfb43f4527 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f3503fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:28:39.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f3503fff700 1 -- 172.21.15.45:0/2997589694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400e040 con 0x55c0a3f182b0 2023-10-17T01:28:39.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f3503fff700 1 -- 172.21.15.45:0/2997589694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f351400bc00 con 0x55c0a3f182b0 2023-10-17T01:28:39.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0a3e92d10 con 0x55c0a3f182b0 2023-10-17T01:28:39.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f3503fff700 1 -- 172.21.15.45:0/2997589694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3514004ac0 con 0x55c0a3f182b0 2023-10-17T01:28:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f353ebf6b80 10 client.16548 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.444+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f3524067eb0 con 0x55c0a3f182b0 2023-10-17T01:28:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f3503fff700 1 -- 172.21.15.45:0/2997589694 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0a3e92d10 con 0x55c0a3f182b0 2023-10-17T01:28:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f3503fff700 1 -- 172.21.15.45:0/2997589694 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3514019040 con 0x55c0a3f182b0 2023-10-17T01:28:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f353ebf6b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:39.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f353ebf6b80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:39.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f353ebf6b80 20 client.16548 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:39.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f3d310 0x55c0a3f3f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f353ebf6b80 4 client.16548 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:28:39.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.445+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0a3ee5d30 con 0x55c0a3f3d310 2023-10-17T01:28:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.446+0000 7f3528f9b700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f3d310 0x55c0a3f3f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.446+0000 7f3528f9b700 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f3d310 0x55c0a3f3f730 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82711c2f9620c593 server_cookie=2ce129f4318eaddd in_seq=0 out_seq=0 2023-10-17T01:28:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.446+0000 7f3503fff700 10 client.16548 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.447+0000 7f3503fff700 1 -- 172.21.15.45:0/2997589694 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55c0a3ee5d30 con 0x55c0a3f3d310 2023-10-17T01:28:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.447+0000 7f3503fff700 10 client.16548 handle_command_reply: tid=0 2023-10-17T01:28:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.447+0000 7f353ebf6b80 1 client.16548 shutdown 2023-10-17T01:28:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.448+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f182b0 msgr2=0x55c0a3f38e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.448+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f182b0 0x55c0a3f38e50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3514009790 tx=0x7f3514007db0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 shutdown_connections 2023-10-17T01:28:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f182b0 0x55c0a3f38e50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f3d310 0x55c0a3f3f730 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f318d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/2997589694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f17500 0x55c0a3f31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 >> 172.21.15.45:0/2997589694 conn(0x55c0a3e760d0 msgr2=0x55c0a3f11020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f3503fff700 0 client.16548 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.449+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 shutdown_connections 2023-10-17T01:28:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.450+0000 7f353ebf6b80 1 -- 172.21.15.45:0/2997589694 wait complete. 2023-10-17T01:28:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.451+0000 7f353ebf6b80 20 client.16548 trim_cache size 0 max 16384 2023-10-17T01:28:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.478+0000 7f353ebf6b80 1 Processor -- start 2023-10-17T01:28:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.478+0000 7f353ebf6b80 1 -- start start 2023-10-17T01:28:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f12650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 0x55c0a3f1e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f182b0 0x55c0a3f1b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3f0e460 con 0x55c0a3f16ba0 2023-10-17T01:28:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3f39270 con 0x55c0a3f17500 2023-10-17T01:28:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3f096a0 con 0x55c0a3f182b0 2023-10-17T01:28:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f3523fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 0x55c0a3f1e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f3523fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 0x55c0a3f1e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59482/0 (socket says 172.21.15.45:59482) 2023-10-17T01:28:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f3523fff700 1 -- 172.21.15.45:0/545769673 learned_addr learned my addr 172.21.15.45:0/545769673 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.479+0000 7f352979c700 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f182b0 0x55c0a3f1b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3528f9b700 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f12650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3523fff700 1 -- 172.21.15.45:0/545769673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f182b0 msgr2=0x55c0a3f1b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3523fff700 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f182b0 0x55c0a3f1b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3523fff700 1 -- 172.21.15.45:0/545769673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f16ba0 msgr2=0x55c0a3f12650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3523fff700 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f12650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3523fff700 1 -- 172.21.15.45:0/545769673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a3c7ec60 con 0x55c0a3f17500 2023-10-17T01:28:39.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f3523fff700 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 0x55c0a3f1e6d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f350c0cc210 tx=0x7f350c010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e714804849ae8be server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:39.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f351b7fe700 1 -- 172.21.15.45:0/545769673 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c099cd0 con 0x55c0a3f17500 2023-10-17T01:28:39.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f351b7fe700 1 -- 172.21.15.45:0/545769673 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f350c0995d0 con 0x55c0a3f17500 2023-10-17T01:28:39.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.480+0000 7f351b7fe700 1 -- 172.21.15.45:0/545769673 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c097a00 con 0x55c0a3f17500 2023-10-17T01:28:39.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 -- 172.21.15.45:0/545769673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 msgr2=0x55c0a3f1e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 0x55c0a3f1e6d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f350c0cc210 tx=0x7f350c010cd0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 -- 172.21.15.45:0/545769673 shutdown_connections 2023-10-17T01:28:39.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f182b0 0x55c0a3f1b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/545769673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f17500 0x55c0a3f1e6d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 -- 172.21.15.45:0/545769673 >> 172.21.15.45:0/545769673 conn(0x55c0a3e760d0 msgr2=0x55c0a3f305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 -- 172.21.15.45:0/545769673 shutdown_connections 2023-10-17T01:28:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.481+0000 7f353ebf6b80 1 -- 172.21.15.45:0/545769673 wait complete. 2023-10-17T01:28:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.482+0000 7f353ebf6b80 1 -- start start 2023-10-17T01:28:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.482+0000 7f353ebf6b80 1 Processor -- start 2023-10-17T01:28:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.483+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f36890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.483+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f34ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.483+0000 7f353ebf6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f3523fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f34ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f353ebf6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3f370b0 con 0x55c0a3f17500 2023-10-17T01:28:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f352979c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f352979c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:59484/0 (socket says 172.21.15.45:59484) 2023-10-17T01:28:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f352979c700 1 -- 172.21.15.45:0/3889082216 learned_addr learned my addr 172.21.15.45:0/3889082216 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3f35920 con 0x55c0a3f182b0 2023-10-17T01:28:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a3f35ae0 con 0x55c0a3f16ba0 2023-10-17T01:28:39.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f36890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.484+0000 7f353ebf6b80 10 client.0 _ll_register_callbacks cb 0x7f351c0091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:39.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.485+0000 7f352979c700 1 -- 172.21.15.45:0/3889082216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 msgr2=0x55c0a3f36890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.485+0000 7f352979c700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f36890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.485+0000 7f352979c700 1 -- 172.21.15.45:0/3889082216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 msgr2=0x55c0a3f34ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:39.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.485+0000 7f352979c700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f34ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.485+0000 7f352979c700 1 -- 172.21.15.45:0/3889082216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3524198d90 con 0x55c0a3f182b0 2023-10-17T01:28:39.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.485+0000 7f3523fff700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f34ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:28:39.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.486+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f36890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:28:39.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.486+0000 7f352979c700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f353e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3514009c10 tx=0x7f351401bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0cd9cd0b2b0b7e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.486+0000 7f3503fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:28:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.486+0000 7f3503fff700 1 -- 172.21.15.45:0/3889082216 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400e070 con 0x55c0a3f182b0 2023-10-17T01:28:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.487+0000 7f3503fff700 1 -- 172.21.15.45:0/3889082216 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3514007e20 con 0x55c0a3f182b0 2023-10-17T01:28:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.487+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3514007420 con 0x55c0a3f182b0 2023-10-17T01:28:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.487+0000 7f3503fff700 1 -- 172.21.15.45:0/3889082216 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3514009400 con 0x55c0a3f182b0 2023-10-17T01:28:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.487+0000 7f353ebf6b80 10 client.26909 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.487+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c0a3cbc250 con 0x55c0a3f182b0 2023-10-17T01:28:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.487+0000 7f3503fff700 1 -- 172.21.15.45:0/3889082216 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3514007420 con 0x55c0a3f182b0 2023-10-17T01:28:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f3503fff700 1 -- 172.21.15.45:0/3889082216 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3514019070 con 0x55c0a3f182b0 2023-10-17T01:28:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f353ebf6b80 10 client.26909 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f353ebf6b80 10 client.26909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f353ebf6b80 20 client.26909 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f8ff60 0x55c0a3f92380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f353ebf6b80 4 client.26909 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:28:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c0a3e60d40 con 0x55c0a3f8ff60 2023-10-17T01:28:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.488+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f8ff60 0x55c0a3f92380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.489+0000 7f3528f9b700 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f8ff60 0x55c0a3f92380 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd82e3bec26d207 server_cookie=d31972a30b07f524 in_seq=0 out_seq=0 2023-10-17T01:28:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.489+0000 7f3503fff700 10 client.26909 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.489+0000 7f3503fff700 1 -- 172.21.15.45:0/3889082216 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c0a3e60d40 con 0x55c0a3f8ff60 2023-10-17T01:28:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.489+0000 7f3503fff700 10 client.26909 handle_command_reply: tid=0 2023-10-17T01:28:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.489+0000 7f353ebf6b80 1 client.26909 shutdown 2023-10-17T01:28:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 msgr2=0x55c0a3f353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f353e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3514009c10 tx=0x7f351401bd70 comp rx=0 tx=0).stop 2023-10-17T01:28:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 shutdown_connections 2023-10-17T01:28:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c0a3f8ff60 0x55c0a3f92380 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c0a3f182b0 0x55c0a3f353e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a3f16ba0 0x55c0a3f36890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 --2- 172.21.15.45:0/3889082216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0a3f17500 0x55c0a3f34ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 >> 172.21.15.45:0/3889082216 conn(0x55c0a3e760d0 msgr2=0x55c0a3f1e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.490+0000 7f3503fff700 0 client.26909 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.491+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 shutdown_connections 2023-10-17T01:28:39.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.491+0000 7f353ebf6b80 1 -- 172.21.15.45:0/3889082216 wait complete. 2023-10-17T01:28:39.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.491+0000 7f353ebf6b80 20 client.26909 trim_cache size 0 max 16384 2023-10-17T01:28:39.504 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:28:39.505 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:28:39.505 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:28:39.505 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:28:39.505 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:28:39.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:28:39.506 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:28:39.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:28:39.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:28:39.507 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:28:39.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35100a0ba0 msgr2=0x7f35100a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35100a0ba0 0x7f35100a3060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55c0a3f0c320 tx=0x7f351c009310 comp rx=0 tx=0).stop 2023-10-17T01:28:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 msgr2=0x7f3524105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 0x7f3524105f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f350c00d950 tx=0x7f350c00dc60 comp rx=0 tx=0).stop 2023-10-17T01:28:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 shutdown_connections 2023-10-17T01:28:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f35100a0ba0 0x7f35100a3060 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35241017d0 0x7f3524102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35240ffc40 0x7f35241059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 --2- 172.21.15.45:0/3471067138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3524100e30 0x7f3524105f00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 >> 172.21.15.45:0/3471067138 conn(0x7f35240fb1b0 msgr2=0x7f35240fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 shutdown_connections 2023-10-17T01:28:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:28:39.492+0000 7f352b1ff700 1 -- 172.21.15.45:0/3471067138 wait complete. 2023-10-17T01:28:39.527 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:28:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 810.0 s, 1254.0 tps, lat 80.109 ms stddev 91.594 2023-10-17T01:28:41.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:40 smithi045 ceph-mon[98474]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 862 op/s 2023-10-17T01:28:41.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:40 smithi060 ceph-mon[134136]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 862 op/s 2023-10-17T01:28:41.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:40 smithi181 ceph-mon[134520]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 862 op/s 2023-10-17T01:28:43.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:42 smithi045 ceph-mon[98474]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-17T01:28:43.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:42 smithi060 ceph-mon[134136]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-17T01:28:43.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:42 smithi181 ceph-mon[134520]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-17T01:28:44.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:43 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:28:44.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:43 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:28:44.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:43 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:28:45.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:28:45.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:28:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:44 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:28:45.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:44 smithi045 ceph-mon[98474]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 859 op/s 2023-10-17T01:28:45.330 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:28:45.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:28:45.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:44 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:28:45.331 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:44 smithi060 ceph-mon[134136]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 859 op/s 2023-10-17T01:28:45.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:28:45.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:28:45.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:44 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:28:45.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:44 smithi181 ceph-mon[134520]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 859 op/s 2023-10-17T01:28:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 815.0 s, 1243.2 tps, lat 80.811 ms stddev 90.593 2023-10-17T01:28:47.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:46 smithi181 ceph-mon[134520]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 MiB/s wr, 575 op/s 2023-10-17T01:28:47.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:46 smithi045 ceph-mon[98474]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 MiB/s wr, 575 op/s 2023-10-17T01:28:47.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:46 smithi060 ceph-mon[134136]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 MiB/s wr, 575 op/s 2023-10-17T01:28:50.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:49 smithi181 ceph-mon[134520]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 618 op/s 2023-10-17T01:28:50.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:49 smithi045 ceph-mon[98474]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 618 op/s 2023-10-17T01:28:50.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:49 smithi060 ceph-mon[134136]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 618 op/s 2023-10-17T01:28:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 820.0 s, 1303.0 tps, lat 76.152 ms stddev 87.938 2023-10-17T01:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 -- 172.21.15.45:0/4240306273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 msgr2=0x7f26dc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 --2- 172.21.15.45:0/4240306273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc101180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009990 tx=0x7f26cc009ca0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 -- 172.21.15.45:0/4240306273 shutdown_connections 2023-10-17T01:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 --2- 172.21.15.45:0/4240306273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc1028f0 0x7f26dc1091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 --2- 172.21.15.45:0/4240306273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc101180 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 --2- 172.21.15.45:0/4240306273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc101f50 0x7f26dc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 -- 172.21.15.45:0/4240306273 >> 172.21.15.45:0/4240306273 conn(0x7f26dc0fc290 msgr2=0x7f26dc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 -- 172.21.15.45:0/4240306273 shutdown_connections 2023-10-17T01:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.581+0000 7f26e11ff700 1 -- 172.21.15.45:0/4240306273 wait complete. 2023-10-17T01:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.583+0000 7f26e11ff700 1 Processor -- start 2023-10-17T01:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.583+0000 7f26e11ff700 1 -- start start 2023-10-17T01:28:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26e11ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26e11ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc101f50 0x7f26dc1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26e11ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc1028f0 0x7f26dc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26e11ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1985c0 con 0x7f26dc101f50 2023-10-17T01:28:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26e11ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc198730 con 0x7f26dc1028f0 2023-10-17T01:28:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26e11ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1988f0 con 0x7f26dc100d60 2023-10-17T01:28:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35948/0 (socket says 172.21.15.45:35948) 2023-10-17T01:28:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26dad9d700 1 -- 172.21.15.45:0/701257078 learned_addr learned my addr 172.21.15.45:0/701257078 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26db59e700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc1028f0 0x7f26dc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.584+0000 7f26da59c700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc101f50 0x7f26dc1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26dad9d700 1 -- 172.21.15.45:0/701257078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc1028f0 msgr2=0x7f26dc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26dad9d700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc1028f0 0x7f26dc197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26dad9d700 1 -- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc101f50 msgr2=0x7f26dc1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26dad9d700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc101f50 0x7f26dc1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26dad9d700 1 -- 172.21.15.45:0/701257078 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc0095f0 con 0x7f26dc100d60 2023-10-17T01:28:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26da59c700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc101f50 0x7f26dc1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:28:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26dad9d700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc196ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005720 tx=0x7f26cc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26c3fff700 1 -- 172.21.15.45:0/701257078 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc01d070 con 0x7f26dc100d60 2023-10-17T01:28:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26c3fff700 1 -- 172.21.15.45:0/701257078 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc021410 con 0x7f26dc100d60 2023-10-17T01:28:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.585+0000 7f26c3fff700 1 -- 172.21.15.45:0/701257078 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc017650 con 0x7f26dc100d60 2023-10-17T01:28:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.586+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc198b70 con 0x7f26dc100d60 2023-10-17T01:28:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.586+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc19f390 con 0x7f26dc100d60 2023-10-17T01:28:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.591+0000 7f26c3fff700 1 -- 172.21.15.45:0/701257078 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f26cc021580 con 0x7f26dc100d60 2023-10-17T01:28:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.592+0000 7f26f4b88b80 1 Processor -- start 2023-10-17T01:28:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.592+0000 7f26f4b88b80 1 -- start start 2023-10-17T01:28:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.592+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.592+0000 7f26c3fff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26c40a0ba0 0x7f26c40a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab3565a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3566470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab34b99b0 con 0x560ab3567710 2023-10-17T01:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab34b7370 con 0x560ab3565fe0 2023-10-17T01:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab34b2c60 con 0x560ab3566db0 2023-10-17T01:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26c3fff700 1 -- 172.21.15.45:0/701257078 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26cc0cd400 con 0x7f26dc100d60 2023-10-17T01:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35952/0 (socket says 172.21.15.45:35952) 2023-10-17T01:28:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26dad9d700 1 -- 172.21.15.45:0/2761995649 learned_addr learned my addr 172.21.15.45:0/2761995649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26da59c700 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab3565a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26db59e700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26c40a0ba0 0x7f26c40a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.593+0000 7f26db59e700 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3566470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.594+0000 7f26dad9d700 1 -- 172.21.15.45:0/2761995649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 msgr2=0x560ab3566470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.594+0000 7f26dad9d700 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3566470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.594+0000 7f26dad9d700 1 -- 172.21.15.45:0/2761995649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 msgr2=0x560ab3565a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.594+0000 7f26dad9d700 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab3565a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.594+0000 7f26dad9d700 1 -- 172.21.15.45:0/2761995649 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ab32e0d80 con 0x560ab3566db0 2023-10-17T01:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.595+0000 7f26db59e700 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3566470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.596+0000 7f26dad9d700 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35671d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26cc09aa40 tx=0x7f26cc021c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c122ec11b2e88828 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.597+0000 7f26c17fa700 1 -- 172.21.15.45:0/2761995649 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc09aab0 con 0x560ab3566db0 2023-10-17T01:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.597+0000 7f26c17fa700 1 -- 172.21.15.45:0/2761995649 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc09e8e0 con 0x560ab3566db0 2023-10-17T01:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.597+0000 7f26c17fa700 1 -- 172.21.15.45:0/2761995649 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc09af70 con 0x560ab3566db0 2023-10-17T01:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 -- 172.21.15.45:0/2761995649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 msgr2=0x560ab35671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26db59e700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26c40a0ba0 0x7f26c40a3060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009ec0 tx=0x7f26d4009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35671d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26cc09aa40 tx=0x7f26cc021c00 comp rx=0 tx=0).stop 2023-10-17T01:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 -- 172.21.15.45:0/2761995649 shutdown_connections 2023-10-17T01:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3566470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35671d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/2761995649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab3565a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 -- 172.21.15.45:0/2761995649 >> 172.21.15.45:0/2761995649 conn(0x560ab34c1070 msgr2=0x560ab355c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 -- 172.21.15.45:0/2761995649 shutdown_connections 2023-10-17T01:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.598+0000 7f26f4b88b80 1 -- 172.21.15.45:0/2761995649 wait complete. 2023-10-17T01:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 -- start start 2023-10-17T01:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 Processor -- start 2023-10-17T01:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 0x560ab356f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 0x560ab356faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab344f4b0 con 0x560ab3567710 2023-10-17T01:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab344f150 con 0x560ab3566db0 2023-10-17T01:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab33368c0 con 0x560ab3565fe0 2023-10-17T01:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.599+0000 7f26f4b88b80 10 client.0 _ll_register_callbacks cb 0x560ab339ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 0x560ab356f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26da59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 0x560ab356faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26da59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 0x560ab356faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:35016/0 (socket says 172.21.15.45:35016) 2023-10-17T01:28:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 0x560ab356f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35956/0 (socket says 172.21.15.45:35956) 2023-10-17T01:28:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26da59c700 1 -- 172.21.15.45:0/3667626514 learned_addr learned my addr 172.21.15.45:0/3667626514 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26db59e700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 -- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 msgr2=0x560ab356faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 0x560ab356faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 -- 172.21.15.45:0/3667626514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 msgr2=0x560ab35768d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35768d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26dad9d700 1 -- 172.21.15.45:0/3667626514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc19f390 con 0x560ab3565fe0 2023-10-17T01:28:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.600+0000 7f26da59c700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 0x560ab356faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.601+0000 7f26dad9d700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 0x560ab356f560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26cc096fb0 tx=0x7f26cc0ba0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892495f4dd52100a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.601+0000 7f26beff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:28:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.601+0000 7f26beff5700 1 -- 172.21.15.45:0/3667626514 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc09af20 con 0x560ab3565fe0 2023-10-17T01:28:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.601+0000 7f26beff5700 1 -- 172.21.15.45:0/3667626514 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26cc0a2600 con 0x560ab3565fe0 2023-10-17T01:28:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.601+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ab34ddcb0 con 0x560ab3565fe0 2023-10-17T01:28:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26beff5700 1 -- 172.21.15.45:0/3667626514 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc0a27c0 con 0x560ab3565fe0 2023-10-17T01:28:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 10 client.26875 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560ab32c9c00 con 0x560ab3565fe0 2023-10-17T01:28:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26beff5700 1 -- 172.21.15.45:0/3667626514 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ab34ddcb0 con 0x560ab3565fe0 2023-10-17T01:28:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26beff5700 1 -- 172.21.15.45:0/3667626514 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26cc0b6070 con 0x560ab3565fe0 2023-10-17T01:28:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 10 client.26875 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 10 client.26875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 20 client.26875 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab3560cf0 0x560ab3561100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 4 client.26875 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:28:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.602+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ab3530cd0 con 0x560ab3560cf0 2023-10-17T01:28:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.603+0000 7f26da59c700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab3560cf0 0x560ab3561100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.603+0000 7f26da59c700 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab3560cf0 0x560ab3561100 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6c321b12ec0757 server_cookie=df3511d22d7953cb in_seq=0 out_seq=0 2023-10-17T01:28:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.603+0000 7f26beff5700 10 client.26875 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.604+0000 7f26beff5700 1 -- 172.21.15.45:0/3667626514 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x560ab3530cd0 con 0x560ab3560cf0 2023-10-17T01:28:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.604+0000 7f26beff5700 10 client.26875 handle_command_reply: tid=0 2023-10-17T01:28:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.604+0000 7f26f4b88b80 1 client.26875 shutdown 2023-10-17T01:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.604+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 msgr2=0x560ab356f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.604+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 0x560ab356f560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26cc096fb0 tx=0x7f26cc0ba0d0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 shutdown_connections 2023-10-17T01:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35768d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3565fe0 0x560ab356f560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab3560cf0 0x560ab3561100 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3667626514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3566db0 0x560ab356faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26beff5700 0 client.26875 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 >> 172.21.15.45:0/3667626514 conn(0x560ab34c1070 msgr2=0x560ab355ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 shutdown_connections 2023-10-17T01:28:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3667626514 wait complete. 2023-10-17T01:28:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.605+0000 7f26f4b88b80 20 client.26875 trim_cache size 0 max 16384 2023-10-17T01:28:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.625+0000 7f26f4b88b80 1 Processor -- start 2023-10-17T01:28:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.625+0000 7f26f4b88b80 1 -- start start 2023-10-17T01:28:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.625+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3565a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab356ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab356e800 con 0x560ab3567710 2023-10-17T01:28:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab3577b20 con 0x560ab3565fe0 2023-10-17T01:28:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab355d5c0 con 0x560ab3566db0 2023-10-17T01:28:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab356ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab356ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35962/0 (socket says 172.21.15.45:35962) 2023-10-17T01:28:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 -- 172.21.15.45:0/3773502367 learned_addr learned my addr 172.21.15.45:0/3773502367 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3565a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26db59e700 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 -- 172.21.15.45:0/3773502367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 msgr2=0x560ab3565a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3565a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 -- 172.21.15.45:0/3773502367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 msgr2=0x560ab35663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.626+0000 7f26da59c700 1 -- 172.21.15.45:0/3773502367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ab33071f0 con 0x560ab3566db0 2023-10-17T01:28:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.627+0000 7f26da59c700 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab356ea00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x560ab3561970 tx=0x7f26d00045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddee5d2d20b20f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.627+0000 7f26c1ffb700 1 -- 172.21.15.45:0/3773502367 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0018040 con 0x560ab3566db0 2023-10-17T01:28:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.627+0000 7f26c1ffb700 1 -- 172.21.15.45:0/3773502367 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d0021070 con 0x560ab3566db0 2023-10-17T01:28:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.627+0000 7f26c1ffb700 1 -- 172.21.15.45:0/3773502367 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d001dbf0 con 0x560ab3566db0 2023-10-17T01:28:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3773502367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 msgr2=0x560ab356ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab356ea00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x560ab3561970 tx=0x7f26d00045f0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3773502367 shutdown_connections 2023-10-17T01:28:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3567710 0x560ab35663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3565fe0 0x560ab3565a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3773502367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab356ea00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3773502367 >> 172.21.15.45:0/3773502367 conn(0x560ab34c1070 msgr2=0x560ab355c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.628+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3773502367 shutdown_connections 2023-10-17T01:28:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.629+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3773502367 wait complete. 2023-10-17T01:28:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.629+0000 7f26f4b88b80 1 -- start start 2023-10-17T01:28:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.629+0000 7f26f4b88b80 1 Processor -- start 2023-10-17T01:28:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3565fe0 0x560ab35644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3567710 0x560ab3564f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab35cfd80 con 0x560ab3565fe0 2023-10-17T01:28:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab35cfef0 con 0x560ab3567710 2023-10-17T01:28:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ab35d00b0 con 0x560ab3566db0 2023-10-17T01:28:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26f4b88b80 10 client.0 _ll_register_callbacks cb 0x7f26cc09a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:28:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26da59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26dad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3565fe0 0x560ab35644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26db59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3567710 0x560ab3564f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26da59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35978/0 (socket says 172.21.15.45:35978) 2023-10-17T01:28:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.630+0000 7f26da59c700 1 -- 172.21.15.45:0/3381066755 learned_addr learned my addr 172.21.15.45:0/3381066755 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:28:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26dad9d700 1 -- 172.21.15.45:0/3381066755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 msgr2=0x560ab35649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26dad9d700 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26dad9d700 1 -- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3567710 msgr2=0x560ab3564f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:28:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26dad9d700 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3567710 0x560ab3564f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26dad9d700 1 -- 172.21.15.45:0/3381066755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d0004040 con 0x560ab3565fe0 2023-10-17T01:28:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26da59c700 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:28:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26dad9d700 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3565fe0 0x560ab35644b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0ba420 tx=0x7f26cc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f26826095db6fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:28:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26beff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:28:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26beff5700 1 -- 172.21.15.45:0/3381066755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0a2800 con 0x560ab3565fe0 2023-10-17T01:28:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26cc09e8e0 con 0x560ab3565fe0 2023-10-17T01:28:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26beff5700 1 -- 172.21.15.45:0/3381066755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26cc09af20 con 0x560ab3565fe0 2023-10-17T01:28:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26beff5700 1 -- 172.21.15.45:0/3381066755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc00fdd0 con 0x560ab3565fe0 2023-10-17T01:28:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26f4b88b80 10 client.16569 fetch_fsmap learned FSMap version 42 2023-10-17T01:28:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560ab32df250 con 0x560ab3565fe0 2023-10-17T01:28:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.631+0000 7f26beff5700 1 -- 172.21.15.45:0/3381066755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc09e8e0 con 0x560ab3565fe0 2023-10-17T01:28:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26beff5700 1 -- 172.21.15.45:0/3381066755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f26cc0b6070 con 0x560ab3565fe0 2023-10-17T01:28:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26f4b88b80 10 client.16569 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:28:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26f4b88b80 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:28:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26f4b88b80 20 client.16569 populate_metadata read hostname 'smithi045' 2023-10-17T01:28:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab35d14d0 0x560ab35d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:28:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26f4b88b80 4 client.16569 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:28:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ab34abce0 con 0x560ab35d14d0 2023-10-17T01:28:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.632+0000 7f26da59c700 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab35d14d0 0x560ab35d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:28:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.633+0000 7f26da59c700 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab35d14d0 0x560ab35d38f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ce6716626d16a8 server_cookie=d5ad82510dda8a15 in_seq=0 out_seq=0 2023-10-17T01:28:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.633+0000 7f26beff5700 10 client.16569 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.645+0000 7f26beff5700 1 -- 172.21.15.45:0/3381066755 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ab34abce0 con 0x560ab35d14d0 2023-10-17T01:28:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.645+0000 7f26beff5700 10 client.16569 handle_command_reply: tid=0 2023-10-17T01:28:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.645+0000 7f26f4b88b80 1 client.16569 shutdown 2023-10-17T01:28:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.645+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3565fe0 msgr2=0x560ab35644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.645+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3565fe0 0x560ab35644b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0ba420 tx=0x7f26cc005bc0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.646+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 shutdown_connections 2023-10-17T01:28:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.646+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x560ab35d14d0 0x560ab35d38f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.646+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560ab3567710 0x560ab3564f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.646+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ab3565fe0 0x560ab35644b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.646+0000 7f26f4b88b80 1 --2- 172.21.15.45:0/3381066755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ab3566db0 0x560ab35649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.646+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 >> 172.21.15.45:0/3381066755 conn(0x560ab34c1070 msgr2=0x560ab3569fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.647+0000 7f26beff5700 0 client.16569 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:28:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.647+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 shutdown_connections 2023-10-17T01:28:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.647+0000 7f26f4b88b80 1 -- 172.21.15.45:0/3381066755 wait complete. 2023-10-17T01:28:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.648+0000 7f26f4b88b80 20 client.16569 trim_cache size 0 max 16384 2023-10-17T01:28:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:28:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:28:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:28:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:28:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.648+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26c40a0ba0 msgr2=0x7f26c40a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.648+0000 7f26e11ff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26c40a0ba0 0x7f26c40a3060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009ec0 tx=0x7f26d4009920 comp rx=0 tx=0).stop 2023-10-17T01:28:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.648+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 msgr2=0x7f26dc196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:28:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.648+0000 7f26e11ff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc196ba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005720 tx=0x7f26cc009580 comp rx=0 tx=0).stop 2023-10-17T01:28:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 shutdown_connections 2023-10-17T01:28:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f26c40a0ba0 0x7f26c40a3060 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc1028f0 0x7f26dc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26dc100d60 0x7f26dc196ba0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 --2- 172.21.15.45:0/701257078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc101f50 0x7f26dc1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:28:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 >> 172.21.15.45:0/701257078 conn(0x7f26dc0fc290 msgr2=0x7f26dc1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:28:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 shutdown_connections 2023-10-17T01:28:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:28:51.649+0000 7f26e11ff700 1 -- 172.21.15.45:0/701257078 wait complete. 2023-10-17T01:28:52.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:51 smithi045 ceph-mon[98474]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-10-17T01:28:52.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:51 smithi060 ceph-mon[134136]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-10-17T01:28:52.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:51 smithi181 ceph-mon[134520]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 677 op/s 2023-10-17T01:28:54.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:53 smithi045 ceph-mon[98474]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 581 op/s 2023-10-17T01:28:54.329 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:53 smithi060 ceph-mon[134136]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 581 op/s 2023-10-17T01:28:54.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:53 smithi181 ceph-mon[134520]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 581 op/s 2023-10-17T01:28:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 825.0 s, 1027.2 tps, lat 97.493 ms stddev 125.333 2023-10-17T01:28:56.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:55 smithi045 ceph-mon[98474]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 772 op/s 2023-10-17T01:28:56.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:55 smithi060 ceph-mon[134136]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 772 op/s 2023-10-17T01:28:56.425 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:55 smithi181 ceph-mon[134520]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 772 op/s 2023-10-17T01:28:58.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:28:58 smithi045 ceph-mon[98474]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 586 op/s 2023-10-17T01:28:58.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:28:58 smithi060 ceph-mon[134136]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 586 op/s 2023-10-17T01:28:58.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:28:58 smithi181 ceph-mon[134520]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 586 op/s 2023-10-17T01:29:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:00 smithi045 ceph-mon[98474]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 966 op/s 2023-10-17T01:29:00.341 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:00 smithi060 ceph-mon[134136]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 966 op/s 2023-10-17T01:29:00.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:00 smithi181 ceph-mon[134520]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 966 op/s 2023-10-17T01:29:00.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 830.0 s, 1100.4 tps, lat 91.411 ms stddev 159.390 2023-10-17T01:29:02.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:02 smithi045 ceph-mon[98474]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 1.74k op/s 2023-10-17T01:29:02.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:02 smithi060 ceph-mon[134136]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 1.74k op/s 2023-10-17T01:29:02.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:02 smithi181 ceph-mon[134520]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 1.74k op/s 2023-10-17T01:29:04.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:04 smithi060 ceph-mon[134136]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2023-10-17T01:29:04.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:04 smithi181 ceph-mon[134520]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2023-10-17T01:29:04.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:04 smithi045 ceph-mon[98474]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2023-10-17T01:29:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 -- 172.21.15.45:0/612634977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40ffbd0 msgr2=0x7ff1e40ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/612634977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e40ffff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00099c0 tx=0x7ff1e0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 -- 172.21.15.45:0/612634977 shutdown_connections 2023-10-17T01:29:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/612634977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1e4101760 0x7ff1e4108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/612634977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e40ffff0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/612634977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e4100dc0 0x7ff1e4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.900+0000 7ff1ebbff700 1 -- 172.21.15.45:0/612634977 >> 172.21.15.45:0/612634977 conn(0x7ff1e40fb140 msgr2=0x7ff1e40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.901+0000 7ff1ebbff700 1 -- 172.21.15.45:0/612634977 shutdown_connections 2023-10-17T01:29:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.901+0000 7ff1ebbff700 1 -- 172.21.15.45:0/612634977 wait complete. 2023-10-17T01:29:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 Processor -- start 2023-10-17T01:29:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 -- start start 2023-10-17T01:29:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e4196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4100dc0 0x7ff1e4196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1e4101760 0x7ff1e419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e580 con 0x7ff1e4101760 2023-10-17T01:29:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e6c0 con 0x7ff1e4100dc0 2023-10-17T01:29:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.903+0000 7ff1ebbff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e850 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.904+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4100dc0 0x7ff1e4196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.904+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4100dc0 0x7ff1e4196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51112/0 (socket says 172.21.15.45:51112) 2023-10-17T01:29:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.904+0000 7ff1e919a700 1 -- 172.21.15.45:0/3821697349 learned_addr learned my addr 172.21.15.45:0/3821697349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.904+0000 7ff1e999b700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e4196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.904+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1e4101760 0x7ff1e419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.905+0000 7ff1e999b700 1 -- 172.21.15.45:0/3821697349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4100dc0 msgr2=0x7ff1e4196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.905+0000 7ff1e999b700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4100dc0 0x7ff1e4196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.905+0000 7ff1e999b700 1 -- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1e4101760 msgr2=0x7ff1e419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.905+0000 7ff1e999b700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1e4101760 0x7ff1e419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.905+0000 7ff1e999b700 1 -- 172.21.15.45:0/3821697349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00095f0 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.905+0000 7ff1e999b700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e4196a00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00058e0 tx=0x7ff1e000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.906+0000 7ff1daffd700 1 -- 172.21.15.45:0/3821697349 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e001d070 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.906+0000 7ff1daffd700 1 -- 172.21.15.45:0/3821697349 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1e0005020 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.906+0000 7ff1daffd700 1 -- 172.21.15.45:0/3821697349 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004a50 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.906+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e410afa0 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.906+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e410b430 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.914+0000 7ff1daffd700 1 -- 172.21.15.45:0/3821697349 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff1e0005400 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.917+0000 7ff1daffd700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1d00a0b20 0x7ff1d00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.918+0000 7ff1daffd700 1 -- 172.21.15.45:0/3821697349 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1e00ccc60 con 0x7ff1e40ffbd0 2023-10-17T01:29:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.918+0000 7ff1e919a700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1d00a0b20 0x7ff1d00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.918+0000 7ff1ff588b80 1 Processor -- start 2023-10-17T01:29:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.918+0000 7ff1ff588b80 1 -- start start 2023-10-17T01:29:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a01830 0x562e34a01c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 0x562e34a02790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a02d90 0x562e34a03210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e349599b0 con 0x562e34a02d90 2023-10-17T01:29:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34957370 con 0x562e34a02190 2023-10-17T01:29:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34952c60 con 0x562e34a01830 2023-10-17T01:29:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 0x562e34a02790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 0x562e34a02790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51124/0 (socket says 172.21.15.45:51124) 2023-10-17T01:29:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 -- 172.21.15.45:0/1156191818 learned_addr learned my addr 172.21.15.45:0/1156191818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e999b700 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a01830 0x562e34a01c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1d00a0b20 0x7ff1d00a2fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009d20 tx=0x7ff1d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a02d90 0x562e34a03210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 -- 172.21.15.45:0/1156191818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a01830 msgr2=0x562e34a01c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a01830 0x562e34a01c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 -- 172.21.15.45:0/1156191818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a02d90 msgr2=0x562e34a03210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.919+0000 7ff1e919a700 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a02d90 0x562e34a03210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.920+0000 7ff1e919a700 1 -- 172.21.15.45:0/1156191818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e34780d80 con 0x562e34a02190 2023-10-17T01:29:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.920+0000 7ff1e999b700 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a01830 0x562e34a01c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.920+0000 7ff1e919a700 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 0x562e34a02790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff1d401c560 tx=0x7ff1d401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb1de98705312ba server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.921+0000 7ff1cffff700 1 -- 172.21.15.45:0/1156191818 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4004a30 con 0x562e34a02190 2023-10-17T01:29:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.921+0000 7ff1cffff700 1 -- 172.21.15.45:0/1156191818 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d40054e0 con 0x562e34a02190 2023-10-17T01:29:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.921+0000 7ff1cffff700 1 -- 172.21.15.45:0/1156191818 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400e990 con 0x562e34a02190 2023-10-17T01:29:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.922+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1156191818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 msgr2=0x562e34a02790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.922+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 0x562e34a02790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff1d401c560 tx=0x7ff1d401b000 comp rx=0 tx=0).stop 2023-10-17T01:29:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1156191818 shutdown_connections 2023-10-17T01:29:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a02d90 0x562e34a03210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a01830 0x562e34a01c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1156191818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02190 0x562e34a02790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1156191818 >> 172.21.15.45:0/1156191818 conn(0x562e34961070 msgr2=0x562e349fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1156191818 shutdown_connections 2023-10-17T01:29:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1156191818 wait complete. 2023-10-17T01:29:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.924+0000 7ff1ff588b80 1 -- start start 2023-10-17T01:29:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 Processor -- start 2023-10-17T01:29:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 0x562e34a0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02d90 0x562e34a12940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e348ef4b0 con 0x562e34a01830 2023-10-17T01:29:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e348ef150 con 0x562e34a02d90 2023-10-17T01:29:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e347d68c0 con 0x562e34a02190 2023-10-17T01:29:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.925+0000 7ff1ff588b80 10 client.0 _ll_register_callbacks cb 0x562e3483ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e999b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 0x562e34a0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 0x562e34a0b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:38792/0 (socket says 172.21.15.45:38792) 2023-10-17T01:29:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 -- 172.21.15.45:0/1916275369 learned_addr learned my addr 172.21.15.45:0/1916275369 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02d90 0x562e34a12940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 -- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02d90 msgr2=0x562e34a12940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02d90 0x562e34a12940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 -- 172.21.15.45:0/1916275369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 msgr2=0x562e34a0b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 -- 172.21.15.45:0/1916275369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e410b430 con 0x562e34a02190 2023-10-17T01:29:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1e919a700 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 0x562e34a0b900 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff1d401cb50 tx=0x7ff1d4005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de6ec556fb1ca31 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.926+0000 7ff1cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:29:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.927+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/1916275369 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4004960 con 0x562e34a02190 2023-10-17T01:29:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.927+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/1916275369 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1d4004ac0 con 0x562e34a02190 2023-10-17T01:29:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.927+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e3497dcb0 con 0x562e34a02190 2023-10-17T01:29:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.928+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/1916275369 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d4004c80 con 0x562e34a02190 2023-10-17T01:29:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.928+0000 7ff1ff588b80 10 client.26893 fetch_fsmap learned FSMap version 42 2023-10-17T01:29:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.928+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7ff1d4017070 con 0x562e34a02190 2023-10-17T01:29:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.928+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/1916275369 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e3497dcb0 con 0x562e34a02190 2023-10-17T01:29:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/1916275369 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1d402c760 con 0x562e34a02190 2023-10-17T01:29:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1ff588b80 10 client.26893 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:29:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1ff588b80 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1ff588b80 20 client.26893 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a170e0 0x562e34a19500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1ff588b80 4 client.26893 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.929+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e349d0cd0 con 0x562e34a170e0 2023-10-17T01:29:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.930+0000 7ff1e999b700 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a170e0 0x562e34a19500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.930+0000 7ff1e999b700 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a170e0 0x562e34a19500 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec04f728c3ecef70 server_cookie=c094033d68747ca8 in_seq=0 out_seq=0 2023-10-17T01:29:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.930+0000 7ff1cd7fa700 10 client.26893 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.931+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/1916275369 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x562e349d0cd0 con 0x562e34a170e0 2023-10-17T01:29:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.931+0000 7ff1cd7fa700 10 client.26893 handle_command_reply: tid=0 2023-10-17T01:29:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.931+0000 7ff1ff588b80 1 client.26893 shutdown 2023-10-17T01:29:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.931+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 msgr2=0x562e34a0b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.931+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 0x562e34a0b900 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff1d401cb50 tx=0x7ff1d4005230 comp rx=0 tx=0).stop 2023-10-17T01:29:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 shutdown_connections 2023-10-17T01:29:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a170e0 0x562e34a19500 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a02d90 0x562e34a12940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1cd7fa700 0 client.26893 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/1916275369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02190 0x562e34a0b900 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 >> 172.21.15.45:0/1916275369 conn(0x562e34961070 msgr2=0x562e349fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 shutdown_connections 2023-10-17T01:29:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.932+0000 7ff1ff588b80 1 -- 172.21.15.45:0/1916275369 wait complete. 2023-10-17T01:29:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.933+0000 7ff1ff588b80 20 client.26893 trim_cache size 0 max 16384 2023-10-17T01:29:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 Processor -- start 2023-10-17T01:29:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 -- start start 2023-10-17T01:29:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a09220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3497dcb0 con 0x562e34a01830 2023-10-17T01:29:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34a0c5b0 con 0x562e34a09760 2023-10-17T01:29:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.956+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e349f69e0 con 0x562e34a02d90 2023-10-17T01:29:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.957+0000 7ff1ea19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.957+0000 7ff1ea19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a05d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51148/0 (socket says 172.21.15.45:51148) 2023-10-17T01:29:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.957+0000 7ff1ea19c700 1 -- 172.21.15.45:0/110805788 learned_addr learned my addr 172.21.15.45:0/110805788 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.957+0000 7ff1e919a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a09220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.957+0000 7ff1ea19c700 1 -- 172.21.15.45:0/110805788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 msgr2=0x562e34a09220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.957+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a09220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.958+0000 7ff1ea19c700 1 -- 172.21.15.45:0/110805788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 msgr2=0x562e34a194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.958+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.958+0000 7ff1ea19c700 1 -- 172.21.15.45:0/110805788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e34769c00 con 0x562e34a09760 2023-10-17T01:29:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.958+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a05d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009fa0 tx=0x7ff1dc009130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3ef11ff39887b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.958+0000 7ff1d8bff700 1 -- 172.21.15.45:0/110805788 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004bb0 con 0x562e34a09760 2023-10-17T01:29:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.959+0000 7ff1d8bff700 1 -- 172.21.15.45:0/110805788 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1dc00f040 con 0x562e34a09760 2023-10-17T01:29:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.959+0000 7ff1d8bff700 1 -- 172.21.15.45:0/110805788 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004780 con 0x562e34a09760 2023-10-17T01:29:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.959+0000 7ff1ff588b80 1 -- 172.21.15.45:0/110805788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 msgr2=0x562e34a05d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.959+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a05d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009fa0 tx=0x7ff1dc009130 comp rx=0 tx=0).stop 2023-10-17T01:29:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 -- 172.21.15.45:0/110805788 shutdown_connections 2023-10-17T01:29:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a05d50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/110805788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a09220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 -- 172.21.15.45:0/110805788 >> 172.21.15.45:0/110805788 conn(0x562e34961070 msgr2=0x562e34a0a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 -- 172.21.15.45:0/110805788 shutdown_connections 2023-10-17T01:29:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.960+0000 7ff1ff588b80 1 -- 172.21.15.45:0/110805788 wait complete. 2023-10-17T01:29:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.961+0000 7ff1ff588b80 1 -- start start 2023-10-17T01:29:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.961+0000 7ff1ff588b80 1 Processor -- start 2023-10-17T01:29:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a0d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34a10290 con 0x562e34a01830 2023-10-17T01:29:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34a103d0 con 0x562e34a09760 2023-10-17T01:29:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34a10590 con 0x562e34a02d90 2023-10-17T01:29:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ff588b80 10 client.0 _ll_register_callbacks cb 0x7ff1e0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ea19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1e999b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ea19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a0fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:51160/0 (socket says 172.21.15.45:51160) 2023-10-17T01:29:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1ea19c700 1 -- 172.21.15.45:0/831525491 learned_addr learned my addr 172.21.15.45:0/831525491 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.962+0000 7ff1e919a700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a0d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1ea19c700 1 -- 172.21.15.45:0/831525491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 msgr2=0x562e34a0d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a0d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1ea19c700 1 -- 172.21.15.45:0/831525491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 msgr2=0x562e34a0cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1ea19c700 1 -- 172.21.15.45:0/831525491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc009c50 con 0x562e34a09760 2023-10-17T01:29:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1e919a700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a0d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:29:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1ea19c700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a0fd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc004350 tx=0x7ff1dc003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e687ad7812bfc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.963+0000 7ff1cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.964+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/831525491 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc024070 con 0x562e34a09760 2023-10-17T01:29:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.964+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/831525491 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff1dc017650 con 0x562e34a09760 2023-10-17T01:29:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.964+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1d40054e0 con 0x562e34a09760 2023-10-17T01:29:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.965+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/831525491 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1dc004780 con 0x562e34a09760 2023-10-17T01:29:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.965+0000 7ff1ff588b80 10 client.26945 fetch_fsmap learned FSMap version 42 2023-10-17T01:29:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.965+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562e347a71f0 con 0x562e34a09760 2023-10-17T01:29:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.965+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/831525491 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d40054e0 con 0x562e34a09760 2023-10-17T01:29:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/831525491 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1dc014050 con 0x562e34a09760 2023-10-17T01:29:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1ff588b80 10 client.26945 fetch_fsmap finished waiting for FSMap version 42 2023-10-17T01:29:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1ff588b80 10 client.26945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1ff588b80 20 client.26945 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a723a0 0x562e34a747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1ff588b80 4 client.26945 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:29:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e3494bce0 con 0x562e34a723a0 2023-10-17T01:29:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.966+0000 7ff1e999b700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a723a0 0x562e34a747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.967+0000 7ff1cd7fa700 10 client.26945 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.967+0000 7ff1e999b700 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a723a0 0x562e34a747c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52001a01c191569 server_cookie=1558750def785798 in_seq=0 out_seq=0 2023-10-17T01:29:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.976+0000 7ff1cd7fa700 1 -- 172.21.15.45:0/831525491 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e3494bce0 con 0x562e34a723a0 2023-10-17T01:29:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.976+0000 7ff1cd7fa700 10 client.26945 handle_command_reply: tid=0 2023-10-17T01:29:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.976+0000 7ff1ff588b80 1 client.26945 shutdown 2023-10-17T01:29:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 msgr2=0x562e34a0fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a0fd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc004350 tx=0x7ff1dc003c70 comp rx=0 tx=0).stop 2023-10-17T01:29:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 shutdown_connections 2023-10-17T01:29:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x562e34a723a0 0x562e34a747c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562e34a09760 0x562e34a0fd50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e34a01830 0x562e34a0cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 --2- 172.21.15.45:0/831525491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e34a02d90 0x562e34a0d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 >> 172.21.15.45:0/831525491 conn(0x562e34961070 msgr2=0x562e34a05d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.977+0000 7ff1cd7fa700 0 client.26945 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.978+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 shutdown_connections 2023-10-17T01:29:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.978+0000 7ff1ff588b80 1 -- 172.21.15.45:0/831525491 wait complete. 2023-10-17T01:29:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.978+0000 7ff1ff588b80 20 client.26945 trim_cache size 0 max 16384 2023-10-17T01:29:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:29:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:29:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:29:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:29:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1d00a0b20 msgr2=0x7ff1d00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1d00a0b20 0x7ff1d00a2fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009d20 tx=0x7ff1d4009260 comp rx=0 tx=0).stop 2023-10-17T01:29:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e40ffbd0 msgr2=0x7ff1e4196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e4196a00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00058e0 tx=0x7ff1e000f690 comp rx=0 tx=0).stop 2023-10-17T01:29:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 shutdown_connections 2023-10-17T01:29:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff1d00a0b20 0x7ff1d00a2fe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1e4101760 0x7ff1e419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e40ffbd0 0x7ff1e4196a00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 --2- 172.21.15.45:0/3821697349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e4100dc0 0x7ff1e4196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 >> 172.21.15.45:0/3821697349 conn(0x7ff1e40fb140 msgr2=0x7ff1e40fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 shutdown_connections 2023-10-17T01:29:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:04.979+0000 7ff1ebbff700 1 -- 172.21.15.45:0/3821697349 wait complete. 2023-10-17T01:29:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 835.0 s, 1402.4 tps, lat 71.623 ms stddev 96.419 2023-10-17T01:29:06.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:06 smithi181 ceph-mon[134520]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 30 MiB/s wr, 1.89k op/s 2023-10-17T01:29:06.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:06 smithi045 ceph-mon[98474]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 30 MiB/s wr, 1.89k op/s 2023-10-17T01:29:06.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:06 smithi060 ceph-mon[134136]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 30 MiB/s wr, 1.89k op/s 2023-10-17T01:29:07.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:07 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:29:07.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:07 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:29:07.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:07 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:29:08.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:08 smithi045 ceph-mon[98474]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 26 MiB/s wr, 1.68k op/s 2023-10-17T01:29:08.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:08 smithi060 ceph-mon[134136]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 26 MiB/s wr, 1.68k op/s 2023-10-17T01:29:08.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:08 smithi181 ceph-mon[134520]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 26 MiB/s wr, 1.68k op/s 2023-10-17T01:29:09.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:29:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 -- 172.21.15.45:0/1132638759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd00ffc40 msgr2=0x7f0dd0100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/1132638759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd00ffc40 0x7f0dd0100060 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00dcb0 tx=0x7f0dcc00f050 comp rx=0 tx=0).stop 2023-10-17T01:29:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 -- 172.21.15.45:0/1132638759 shutdown_connections 2023-10-17T01:29:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/1132638759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd01017d0 0x7f0dd0108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/1132638759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd00ffc40 0x7f0dd0100060 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/1132638759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd0100e30 0x7f0dd0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 -- 172.21.15.45:0/1132638759 >> 172.21.15.45:0/1132638759 conn(0x7f0dd00fb1b0 msgr2=0x7f0dd00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:09.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 -- 172.21.15.45:0/1132638759 shutdown_connections 2023-10-17T01:29:09.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.859+0000 7f0dd7bff700 1 -- 172.21.15.45:0/1132638759 wait complete. 2023-10-17T01:29:09.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.860+0000 7f0dd7bff700 1 Processor -- start 2023-10-17T01:29:09.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.860+0000 7f0dd7bff700 1 -- start start 2023-10-17T01:29:09.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd7bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00ffc40 0x7f0dd01948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd7bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0100e30 0x7f0dd0194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd7bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 0x7f0dd019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd7bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd019c430 con 0x7f0dd01017d0 2023-10-17T01:29:09.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd7bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd019c570 con 0x7f0dd0100e30 2023-10-17T01:29:09.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd7bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd019c700 con 0x7f0dd00ffc40 2023-10-17T01:29:09.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd619c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 0x7f0dd019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00ffc40 0x7f0dd01948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd619c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 0x7f0dd019be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56904/0 (socket says 172.21.15.45:56904) 2023-10-17T01:29:09.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd619c700 1 -- 172.21.15.45:0/3506234374 learned_addr learned my addr 172.21.15.45:0/3506234374 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:09.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.861+0000 7f0dd519a700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0100e30 0x7f0dd0194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd519a700 1 -- 172.21.15.45:0/3506234374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00ffc40 msgr2=0x7f0dd01948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd519a700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00ffc40 0x7f0dd01948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd519a700 1 -- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 msgr2=0x7f0dd019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd519a700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 0x7f0dd019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd519a700 1 -- 172.21.15.45:0/3506234374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc00d930 con 0x7f0dd0100e30 2023-10-17T01:29:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd619c700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 0x7f0dd019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd519a700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0100e30 0x7f0dd0194df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000a8d0 tx=0x7f0dc000abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dc6ffd700 1 -- 172.21.15.45:0/3506234374 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0010070 con 0x7f0dd0100e30 2023-10-17T01:29:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.862+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd019c9e0 con 0x7f0dd0100e30 2023-10-17T01:29:09.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.863+0000 7f0dc6ffd700 1 -- 172.21.15.45:0/3506234374 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dc0007bc0 con 0x7f0dd0100e30 2023-10-17T01:29:09.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.863+0000 7f0dc6ffd700 1 -- 172.21.15.45:0/3506234374 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0014d00 con 0x7f0dd0100e30 2023-10-17T01:29:09.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.863+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd019cf40 con 0x7f0dd0100e30 2023-10-17T01:29:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.867+0000 7f0dc6ffd700 1 -- 172.21.15.45:0/3506234374 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0dc0007d30 con 0x7f0dd0100e30 2023-10-17T01:29:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.868+0000 7f0deb469b80 1 Processor -- start 2023-10-17T01:29:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.868+0000 7f0deb469b80 1 -- start start 2023-10-17T01:29:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.868+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b33fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.868+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.868+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b352b0 0x55b7c8b35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.868+0000 7f0deb469b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8a8ba10 con 0x55b7c8b352b0 2023-10-17T01:29:09.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.869+0000 7f0deb469b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8a893d0 con 0x55b7c8b33ba0 2023-10-17T01:29:09.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.869+0000 7f0deb469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8a84cc0 con 0x55b7c8b34500 2023-10-17T01:29:09.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.869+0000 7f0dd619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b33fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.869+0000 7f0dd619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b33fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48592/0 (socket says 172.21.15.45:48592) 2023-10-17T01:29:09.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.869+0000 7f0dd619c700 1 -- 172.21.15.45:0/372603668 learned_addr learned my addr 172.21.15.45:0/372603668 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:09.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.869+0000 7f0dd519a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b352b0 0x55b7c8b35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.870+0000 7f0dd619c700 1 -- 172.21.15.45:0/372603668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 msgr2=0x55b7c8b34d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.870+0000 7f0dd619c700 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.870+0000 7f0dd619c700 1 -- 172.21.15.45:0/372603668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b352b0 msgr2=0x55b7c8b35730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.870+0000 7f0dd619c700 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b352b0 0x55b7c8b35730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.870+0000 7f0dd619c700 1 -- 172.21.15.45:0/372603668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7c88b2de0 con 0x55b7c8b33ba0 2023-10-17T01:29:09.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.871+0000 7f0dd519a700 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b352b0 0x55b7c8b35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:09.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.872+0000 7f0dd619c700 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b33fc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8007960 tx=0x7f0dc80091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce77727d4e20b50d server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:09.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.872+0000 7f0dbbfff700 1 -- 172.21.15.45:0/372603668 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8015850 con 0x55b7c8b33ba0 2023-10-17T01:29:09.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.872+0000 7f0dbbfff700 1 -- 172.21.15.45:0/372603668 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dc800f040 con 0x55b7c8b33ba0 2023-10-17T01:29:09.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.873+0000 7f0dbbfff700 1 -- 172.21.15.45:0/372603668 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8014e10 con 0x55b7c8b33ba0 2023-10-17T01:29:09.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.873+0000 7f0dc6ffd700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0dbc0a0b20 0x7f0dbc0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.873+0000 7f0dc6ffd700 1 -- 172.21.15.45:0/3506234374 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0dc00cb3c0 con 0x7f0dd0100e30 2023-10-17T01:29:09.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.874+0000 7f0deb469b80 1 -- 172.21.15.45:0/372603668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 msgr2=0x55b7c8b33fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.874+0000 7f0deb469b80 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b33fc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8007960 tx=0x7f0dc80091d0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.874+0000 7f0dd599b700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0dbc0a0b20 0x7f0dbc0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 -- 172.21.15.45:0/372603668 shutdown_connections 2023-10-17T01:29:09.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b352b0 0x55b7c8b35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b33fc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 --2- 172.21.15.45:0/372603668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 -- 172.21.15.45:0/372603668 >> 172.21.15.45:0/372603668 conn(0x55b7c8a930d0 msgr2=0x55b7c8b2e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:09.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 -- 172.21.15.45:0/372603668 shutdown_connections 2023-10-17T01:29:09.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0deb469b80 1 -- 172.21.15.45:0/372603668 wait complete. 2023-10-17T01:29:09.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.875+0000 7f0dd599b700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0dbc0a0b20 0x7f0dbc0a2fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00dc80 tx=0x7f0dcc005de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:09.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 -- start start 2023-10-17T01:29:09.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 Processor -- start 2023-10-17T01:29:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 0x55b7c8b3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b44e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8a21510 con 0x55b7c8b33ba0 2023-10-17T01:29:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8a211b0 con 0x55b7c8b34500 2023-10-17T01:29:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.876+0000 7f0deb469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8908920 con 0x55b7c8b352b0 2023-10-17T01:29:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0deb469b80 10 client.0 _ll_register_callbacks cb 0x55b7c896ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd519a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 0x55b7c8b3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd519a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 0x55b7c8b3ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48598/0 (socket says 172.21.15.45:48598) 2023-10-17T01:29:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd519a700 1 -- 172.21.15.45:0/1835209190 learned_addr learned my addr 172.21.15.45:0/1835209190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd619c700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b44e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd519a700 1 -- 172.21.15.45:0/1835209190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 msgr2=0x55b7c8b44e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b44e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.877+0000 7f0dd519a700 1 -- 172.21.15.45:0/1835209190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 msgr2=0x55b7c8b3d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0dd519a700 1 -- 172.21.15.45:0/1835209190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd019cf40 con 0x55b7c8b34500 2023-10-17T01:29:09.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:29:09.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 0x55b7c8b3ddf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00c8e90 tx=0x7f0dc0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbfcbc18ab52fed7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:09.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0db97fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:09.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0db97fa700 1 -- 172.21.15.45:0/1835209190 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0094d20 con 0x55b7c8b34500 2023-10-17T01:29:09.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0db97fa700 1 -- 172.21.15.45:0/1835209190 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0dc00990e0 con 0x55b7c8b34500 2023-10-17T01:29:09.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7c8aafd10 con 0x55b7c8b34500 2023-10-17T01:29:09.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0db97fa700 1 -- 172.21.15.45:0/1835209190 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dc00987b0 con 0x55b7c8b34500 2023-10-17T01:29:09.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0deb469b80 10 client.26957 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:09.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.878+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f0dc80075c0 con 0x55b7c8b34500 2023-10-17T01:29:09.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0db97fa700 1 -- 172.21.15.45:0/1835209190 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7c8aafd10 con 0x55b7c8b34500 2023-10-17T01:29:09.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0db97fa700 1 -- 172.21.15.45:0/1835209190 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f0dc00b8070 con 0x55b7c8b34500 2023-10-17T01:29:09.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0deb469b80 10 client.26957 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:09.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0deb469b80 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:09.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0deb469b80 20 client.26957 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:09.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b49690 0x55b7c8b4bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0deb469b80 4 client.26957 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:09.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7c8b02d30 con 0x55b7c8b49690 2023-10-17T01:29:09.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.879+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b49690 0x55b7c8b4bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.880+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b49690 0x55b7c8b4bab0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2fdcd743d9925d server_cookie=f41f7058a7978fa0 in_seq=0 out_seq=0 2023-10-17T01:29:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.880+0000 7f0db97fa700 10 client.26957 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0db97fa700 1 -- 172.21.15.45:0/1835209190 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55b7c8b02d30 con 0x55b7c8b49690 2023-10-17T01:29:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0db97fa700 10 client.26957 handle_command_reply: tid=0 2023-10-17T01:29:09.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 client.26957 shutdown 2023-10-17T01:29:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 msgr2=0x55b7c8b3ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 0x55b7c8b3ddf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00c8e90 tx=0x7f0dc0000bc0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 shutdown_connections 2023-10-17T01:29:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b44e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b49690 0x55b7c8b4bab0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1835209190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b34500 0x55b7c8b3ddf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 >> 172.21.15.45:0/1835209190 conn(0x55b7c8a930d0 msgr2=0x55b7c8b2e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0db97fa700 0 client.26957 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:09.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 shutdown_connections 2023-10-17T01:29:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 1 -- 172.21.15.45:0/1835209190 wait complete. 2023-10-17T01:29:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.881+0000 7f0deb469b80 20 client.26957 trim_cache size 0 max 16384 2023-10-17T01:29:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.903+0000 7f0deb469b80 1 Processor -- start 2023-10-17T01:29:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.903+0000 7f0deb469b80 1 -- start start 2023-10-17T01:29:09.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.904+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.904+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b38200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.904+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 0x55b7c8b38740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.904+0000 7f0deb469b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8b3cd90 con 0x55b7c8b33ba0 2023-10-17T01:29:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.904+0000 7f0deb469b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8b3d450 con 0x55b7c8b352b0 2023-10-17T01:29:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.904+0000 7f0deb469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8b2b470 con 0x55b7c8b34500 2023-10-17T01:29:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 0x55b7c8b38740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 0x55b7c8b38740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48608/0 (socket says 172.21.15.45:48608) 2023-10-17T01:29:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd619c700 1 -- 172.21.15.45:0/1608857674 learned_addr learned my addr 172.21.15.45:0/1608857674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd619c700 1 -- 172.21.15.45:0/1608857674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 msgr2=0x55b7c8b38200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b38200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.905+0000 7f0dd619c700 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b38200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dd619c700 1 -- 172.21.15.45:0/1608857674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 msgr2=0x55b7c8b3b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dd619c700 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dd619c700 1 -- 172.21.15.45:0/1608857674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7c889bc60 con 0x55b7c8b352b0 2023-10-17T01:29:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dd619c700 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 0x55b7c8b38740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8007960 tx=0x7f0dc8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5957e433cebbe7 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dc4bff700 1 -- 172.21.15.45:0/1608857674 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8014c50 con 0x55b7c8b352b0 2023-10-17T01:29:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dc4bff700 1 -- 172.21.15.45:0/1608857674 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dc8014860 con 0x55b7c8b352b0 2023-10-17T01:29:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.906+0000 7f0dc4bff700 1 -- 172.21.15.45:0/1608857674 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8015d60 con 0x55b7c8b352b0 2023-10-17T01:29:09.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 -- 172.21.15.45:0/1608857674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 msgr2=0x55b7c8b38740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 0x55b7c8b38740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8007960 tx=0x7f0dc8004b40 comp rx=0 tx=0).stop 2023-10-17T01:29:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 -- 172.21.15.45:0/1608857674 shutdown_connections 2023-10-17T01:29:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b352b0 0x55b7c8b38740 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b3b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1608857674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b34500 0x55b7c8b38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 -- 172.21.15.45:0/1608857674 >> 172.21.15.45:0/1608857674 conn(0x55b7c8a930d0 msgr2=0x55b7c8b4b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 -- 172.21.15.45:0/1608857674 shutdown_connections 2023-10-17T01:29:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.907+0000 7f0deb469b80 1 -- 172.21.15.45:0/1608857674 wait complete. 2023-10-17T01:29:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.908+0000 7f0deb469b80 1 -- start start 2023-10-17T01:29:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.908+0000 7f0deb469b80 1 Processor -- start 2023-10-17T01:29:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.908+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b42c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b34500 0x55b7c8b41270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0deb469b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0deb469b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8b41ea0 con 0x55b7c8b34500 2023-10-17T01:29:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0deb469b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8b42010 con 0x55b7c8b33ba0 2023-10-17T01:29:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0deb469b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8b9b020 con 0x55b7c8b352b0 2023-10-17T01:29:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b42c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0deb469b80 10 client.0 _ll_register_callbacks cb 0x55b7c896ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b42c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:48620/0 (socket says 172.21.15.45:48620) 2023-10-17T01:29:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 -- 172.21.15.45:0/1706480498 learned_addr learned my addr 172.21.15.45:0/1706480498 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd619c700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b34500 0x55b7c8b41270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 -- 172.21.15.45:0/1706480498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 msgr2=0x55b7c8b417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 -- 172.21.15.45:0/1706480498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b34500 msgr2=0x55b7c8b41270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b34500 0x55b7c8b41270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 -- 172.21.15.45:0/1706480498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7c88b2de0 con 0x55b7c8b33ba0 2023-10-17T01:29:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0dd599b700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b42c60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00d750 tx=0x7f0dcc00f050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4203c17101ce35 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.909+0000 7f0db97fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0db97fa700 1 -- 172.21.15.45:0/1706480498 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc037040 con 0x55b7c8b33ba0 2023-10-17T01:29:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0db97fa700 1 -- 172.21.15.45:0/1706480498 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0dcc0392b0 con 0x55b7c8b33ba0 2023-10-17T01:29:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0dc800f040 con 0x55b7c8b33ba0 2023-10-17T01:29:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0db97fa700 1 -- 172.21.15.45:0/1706480498 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dcc028070 con 0x55b7c8b33ba0 2023-10-17T01:29:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 10 client.26963 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b7c88d9250 con 0x55b7c8b33ba0 2023-10-17T01:29:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0db97fa700 1 -- 172.21.15.45:0/1706480498 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dc800f040 con 0x55b7c8b33ba0 2023-10-17T01:29:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0db97fa700 1 -- 172.21.15.45:0/1706480498 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f0dcc004cb0 con 0x55b7c8b33ba0 2023-10-17T01:29:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 10 client.26963 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 20 client.26963 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.910+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b9c950 0x55b7c8b9ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0deb469b80 4 client.26963 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:29:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7c8a7dd40 con 0x55b7c8b9c950 2023-10-17T01:29:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b9c950 0x55b7c8b9ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0dd519a700 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b9c950 0x55b7c8b9ed70 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4988ddd8514f0 server_cookie=99cd4a8c027f99c6 in_seq=0 out_seq=0 2023-10-17T01:29:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0db97fa700 10 client.26963 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0db97fa700 1 -- 172.21.15.45:0/1706480498 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b7c8a7dd40 con 0x55b7c8b9c950 2023-10-17T01:29:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0db97fa700 10 client.26963 handle_command_reply: tid=0 2023-10-17T01:29:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0deb469b80 1 client.26963 shutdown 2023-10-17T01:29:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.911+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 msgr2=0x55b7c8b42c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b42c60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00d750 tx=0x7f0dcc00f050 comp rx=0 tx=0).stop 2023-10-17T01:29:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 shutdown_connections 2023-10-17T01:29:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b7c8b352b0 0x55b7c8b417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b7c8b9c950 0x55b7c8b9ed70 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b7c8b33ba0 0x55b7c8b42c60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 --2- 172.21.15.45:0/1706480498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7c8b34500 0x55b7c8b41270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0db97fa700 0 client.26963 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:09.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 >> 172.21.15.45:0/1706480498 conn(0x55b7c8a930d0 msgr2=0x55b7c8b3b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:09.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 shutdown_connections 2023-10-17T01:29:09.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 1 -- 172.21.15.45:0/1706480498 wait complete. 2023-10-17T01:29:09.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.912+0000 7f0deb469b80 20 client.26963 trim_cache size 0 max 16384 2023-10-17T01:29:09.926 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:29:09.926 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:29:09.926 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:29:09.927 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:29:09.927 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:29:09.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:29:09.927 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:29:09.927 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:29:09.927 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:29:09.928 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:29:09.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0dbc0a0b20 msgr2=0x7f0dbc0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0dbc0a0b20 0x7f0dbc0a2fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00dc80 tx=0x7f0dcc005de0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0100e30 msgr2=0x7f0dd0194df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:09.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0100e30 0x7f0dd0194df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000a8d0 tx=0x7f0dc000abe0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 shutdown_connections 2023-10-17T01:29:09.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0dbc0a0b20 0x7f0dbc0a2fe0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd01017d0 0x7f0dd019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00ffc40 0x7f0dd01948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 --2- 172.21.15.45:0/3506234374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0100e30 0x7f0dd0194df0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:09.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 >> 172.21.15.45:0/3506234374 conn(0x7f0dd00fb1b0 msgr2=0x7f0dd00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:09.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 shutdown_connections 2023-10-17T01:29:09.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:09.913+0000 7f0dd7bff700 1 -- 172.21.15.45:0/3506234374 wait complete. 2023-10-17T01:29:09.938 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:29:10.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:10 smithi045 ceph-mon[98474]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2023-10-17T01:29:10.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:10 smithi060 ceph-mon[134136]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2023-10-17T01:29:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 840.0 s, 1371.0 tps, lat 72.654 ms stddev 98.228 2023-10-17T01:29:10.665 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:10 smithi181 ceph-mon[134520]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2023-10-17T01:29:12.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:12 smithi045 ceph-mon[98474]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-10-17T01:29:12.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:12 smithi060 ceph-mon[134136]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-10-17T01:29:12.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:12 smithi181 ceph-mon[134520]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-10-17T01:29:14.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:14 smithi045 ceph-mon[98474]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 714 op/s 2023-10-17T01:29:14.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:14 smithi060 ceph-mon[134136]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 714 op/s 2023-10-17T01:29:14.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:14 smithi181 ceph-mon[134520]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 714 op/s 2023-10-17T01:29:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 845.0 s, 1333.2 tps, lat 75.301 ms stddev 82.720 2023-10-17T01:29:16.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:16 smithi045 ceph-mon[98474]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 679 op/s 2023-10-17T01:29:16.583 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:16 smithi060 ceph-mon[134136]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 679 op/s 2023-10-17T01:29:16.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:16 smithi181 ceph-mon[134520]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 679 op/s 2023-10-17T01:29:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 -- 172.21.15.45:0/1067039822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44101f70 msgr2=0x7fce441023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 --2- 172.21.15.45:0/1067039822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44101f70 0x7fce441023d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fce38009990 tx=0x7fce38009ca0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 -- 172.21.15.45:0/1067039822 shutdown_connections 2023-10-17T01:29:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 --2- 172.21.15.45:0/1067039822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44102910 0x7fce441091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 --2- 172.21.15.45:0/1067039822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44100d80 0x7fce441011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 --2- 172.21.15.45:0/1067039822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44101f70 0x7fce441023d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 -- 172.21.15.45:0/1067039822 >> 172.21.15.45:0/1067039822 conn(0x7fce440fc2d0 msgr2=0x7fce440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 -- 172.21.15.45:0/1067039822 shutdown_connections 2023-10-17T01:29:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.201+0000 7fce48dff700 1 -- 172.21.15.45:0/1067039822 wait complete. 2023-10-17T01:29:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.204+0000 7fce48dff700 1 Processor -- start 2023-10-17T01:29:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 -- start start 2023-10-17T01:29:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44100d80 0x7fce44198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 0x7fce441992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44102910 0x7fce4419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4419a790 con 0x7fce44100d80 2023-10-17T01:29:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4419a900 con 0x7fce44101f70 2023-10-17T01:29:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.205+0000 7fce48dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4419aac0 con 0x7fce44102910 2023-10-17T01:29:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.206+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 0x7fce441992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.206+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 0x7fce441992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33926/0 (socket says 172.21.15.45:33926) 2023-10-17T01:29:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.206+0000 7fce41d9b700 1 -- 172.21.15.45:0/494833787 learned_addr learned my addr 172.21.15.45:0/494833787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.206+0000 7fce41d9b700 1 -- 172.21.15.45:0/494833787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44102910 msgr2=0x7fce4419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.207+0000 7fce41d9b700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44102910 0x7fce4419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.207+0000 7fce4259c700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44100d80 0x7fce44198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.207+0000 7fce41d9b700 1 -- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44100d80 msgr2=0x7fce44198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.207+0000 7fce41d9b700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44100d80 0x7fce44198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.207+0000 7fce41d9b700 1 -- 172.21.15.45:0/494833787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009520 con 0x7fce44101f70 2023-10-17T01:29:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.207+0000 7fce41d9b700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 0x7fce441992e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fce38000c00 tx=0x7fce380057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.208+0000 7fce337fe700 1 -- 172.21.15.45:0/494833787 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3801d070 con 0x7fce44101f70 2023-10-17T01:29:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.208+0000 7fce337fe700 1 -- 172.21.15.45:0/494833787 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3800fc40 con 0x7fce44101f70 2023-10-17T01:29:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.208+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce380095f0 con 0x7fce44101f70 2023-10-17T01:29:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.208+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce441a1300 con 0x7fce44101f70 2023-10-17T01:29:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.208+0000 7fce337fe700 1 -- 172.21.15.45:0/494833787 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38017650 con 0x7fce44101f70 2023-10-17T01:29:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.211+0000 7fce337fe700 1 -- 172.21.15.45:0/494833787 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fce38021410 con 0x7fce44101f70 2023-10-17T01:29:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.213+0000 7fce337fe700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce2c0a0c70 0x7fce2c0a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.213+0000 7fce337fe700 1 -- 172.21.15.45:0/494833787 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce38021e40 con 0x7fce44101f70 2023-10-17T01:29:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.213+0000 7fce4259c700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce2c0a0c70 0x7fce2c0a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.213+0000 7fce5c7c9b80 1 Processor -- start 2023-10-17T01:29:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.213+0000 7fce5c7c9b80 1 -- start start 2023-10-17T01:29:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b19c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1a190 0x55b6e9b1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b1b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9a719b0 con 0x55b6e9b19830 2023-10-17T01:29:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9a6f370 con 0x55b6e9b1ad90 2023-10-17T01:29:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1a190 0x55b6e9b1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1a190 0x55b6e9b1a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59894/0 (socket says 172.21.15.45:59894) 2023-10-17T01:29:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce41d9b700 1 -- 172.21.15.45:0/120643896 learned_addr learned my addr 172.21.15.45:0/120643896 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce4259c700 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b19c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/120643896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9a6ac60 con 0x55b6e9b1a190 2023-10-17T01:29:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.214+0000 7fce4259c700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce2c0a0c70 0x7fce2c0a3130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fce44199b40 tx=0x7fce34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.215+0000 7fce42d9d700 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b1b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.215+0000 7fce42d9d700 1 -- 172.21.15.45:0/120643896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1a190 msgr2=0x55b6e9b1a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.215+0000 7fce42d9d700 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1a190 0x55b6e9b1a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.215+0000 7fce42d9d700 1 -- 172.21.15.45:0/120643896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 msgr2=0x55b6e9b19c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.216+0000 7fce42d9d700 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b19c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.216+0000 7fce42d9d700 1 -- 172.21.15.45:0/120643896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e9898d80 con 0x55b6e9b1ad90 2023-10-17T01:29:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.216+0000 7fce4259c700 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b19c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.217+0000 7fce42d9d700 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b1b210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e490 tx=0x7fce3c00e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b26f34147bce4fa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.217+0000 7fce30ff9700 1 -- 172.21.15.45:0/120643896 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c004db0 con 0x55b6e9b1ad90 2023-10-17T01:29:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.217+0000 7fce30ff9700 1 -- 172.21.15.45:0/120643896 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3c00f040 con 0x55b6e9b1ad90 2023-10-17T01:29:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.218+0000 7fce30ff9700 1 -- 172.21.15.45:0/120643896 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c004480 con 0x55b6e9b1ad90 2023-10-17T01:29:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.218+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/120643896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 msgr2=0x55b6e9b1b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.218+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b1b210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e490 tx=0x7fce3c00e7a0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/120643896 shutdown_connections 2023-10-17T01:29:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b1b210 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b19c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/120643896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1a190 0x55b6e9b1a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/120643896 >> 172.21.15.45:0/120643896 conn(0x55b6e9a79070 msgr2=0x55b6e9b142c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/120643896 shutdown_connections 2023-10-17T01:29:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.219+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/120643896 wait complete. 2023-10-17T01:29:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 -- start start 2023-10-17T01:29:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 Processor -- start 2023-10-17T01:29:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b2a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9a074b0 con 0x55b6e9b19830 2023-10-17T01:29:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9a07150 con 0x55b6e9b1a190 2023-10-17T01:29:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.220+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e98ee8c0 con 0x55b6e9b1ad90 2023-10-17T01:29:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce5c7c9b80 10 client.0 _ll_register_callbacks cb 0x55b6e9954a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:18.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b238f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:33942/0 (socket says 172.21.15.45:33942) 2023-10-17T01:29:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 -- 172.21.15.45:0/803829217 learned_addr learned my addr 172.21.15.45:0/803829217 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce42d9d700 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b2a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 -- 172.21.15.45:0/803829217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1ad90 msgr2=0x55b6e9b2a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b2a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 -- 172.21.15.45:0/803829217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 msgr2=0x55b6e9b233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.221+0000 7fce41d9b700 1 -- 172.21.15.45:0/803829217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce34009e30 con 0x55b6e9b1a190 2023-10-17T01:29:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.222+0000 7fce41d9b700 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b238f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fce38005f50 tx=0x7fce3809aac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56674da156977469 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.222+0000 7fce29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.232+0000 7fce29ffb700 1 -- 172.21.15.45:0/803829217 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3809e9f0 con 0x55b6e9b1a190 2023-10-17T01:29:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.232+0000 7fce29ffb700 1 -- 172.21.15.45:0/803829217 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce380a2ba0 con 0x55b6e9b1a190 2023-10-17T01:29:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.236+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6e9a95cb0 con 0x55b6e9b1a190 2023-10-17T01:29:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.243+0000 7fce29ffb700 1 -- 172.21.15.45:0/803829217 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce380cd0e0 con 0x55b6e9b1a190 2023-10-17T01:29:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.245+0000 7fce5c7c9b80 10 client.26975 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.246+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fce3c00e140 con 0x55b6e9b1a190 2023-10-17T01:29:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce29ffb700 1 -- 172.21.15.45:0/803829217 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6e9a95cb0 con 0x55b6e9b1a190 2023-10-17T01:29:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce29ffb700 1 -- 172.21.15.45:0/803829217 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fce380a2710 con 0x55b6e9b1a190 2023-10-17T01:29:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce5c7c9b80 10 client.26975 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce5c7c9b80 10 client.26975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce5c7c9b80 20 client.26975 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b2ef90 0x55b6e9b313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce5c7c9b80 4 client.26975 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.251+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6e9ae8cd0 con 0x55b6e9b2ef90 2023-10-17T01:29:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.256+0000 7fce4259c700 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b2ef90 0x55b6e9b313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.266+0000 7fce29ffb700 10 client.26975 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.266+0000 7fce4259c700 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b2ef90 0x55b6e9b313b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323b88a9fc2535f7 server_cookie=dde7c8ba40bae985 in_seq=0 out_seq=0 2023-10-17T01:29:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.277+0000 7fce29ffb700 1 -- 172.21.15.45:0/803829217 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55b6e9ae8cd0 con 0x55b6e9b2ef90 2023-10-17T01:29:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.277+0000 7fce29ffb700 10 client.26975 handle_command_reply: tid=0 2023-10-17T01:29:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 client.26975 shutdown 2023-10-17T01:29:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 msgr2=0x55b6e9b238f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b238f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fce38005f50 tx=0x7fce3809aac0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 shutdown_connections 2023-10-17T01:29:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b2a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b19830 0x55b6e9b233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b2ef90 0x55b6e9b313b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/803829217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b238f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 >> 172.21.15.45:0/803829217 conn(0x55b6e9a79070 msgr2=0x55b6e9b13a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.278+0000 7fce29ffb700 0 client.26975 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.282+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 shutdown_connections 2023-10-17T01:29:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.283+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/803829217 wait complete. 2023-10-17T01:29:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.284+0000 7fce5c7c9b80 20 client.26975 trim_cache size 0 max 16384 2023-10-17T01:29:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.319+0000 7fce5c7c9b80 1 Processor -- start 2023-10-17T01:29:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.319+0000 7fce5c7c9b80 1 -- start start 2023-10-17T01:29:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.319+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.319+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1a190 0x55b6e9b22100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b13a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9b15750 con 0x55b6e9b1a190 2023-10-17T01:29:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9b21ed0 con 0x55b6e9b1ad90 2023-10-17T01:29:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9b15180 con 0x55b6e9b19830 2023-10-17T01:29:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce42d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b13a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b22a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:59902/0 (socket says 172.21.15.45:59902) 2023-10-17T01:29:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce4259c700 1 -- 172.21.15.45:0/2125698793 learned_addr learned my addr 172.21.15.45:0/2125698793 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce41d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1a190 0x55b6e9b22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce42d9d700 1 -- 172.21.15.45:0/2125698793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 msgr2=0x55b6e9b22a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce42d9d700 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b22a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce42d9d700 1 -- 172.21.15.45:0/2125698793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1a190 msgr2=0x55b6e9b22100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce42d9d700 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1a190 0x55b6e9b22100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce42d9d700 1 -- 172.21.15.45:0/2125698793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e9881c00 con 0x55b6e9b1ad90 2023-10-17T01:29:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.320+0000 7fce41d9b700 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1a190 0x55b6e9b22100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.321+0000 7fce42d9d700 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b13a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00eaf0 tx=0x7fce3c009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81370cad05688118 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.321+0000 7fce317fa700 1 -- 172.21.15.45:0/2125698793 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c004db0 con 0x55b6e9b1ad90 2023-10-17T01:29:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.321+0000 7fce317fa700 1 -- 172.21.15.45:0/2125698793 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3c0090d0 con 0x55b6e9b1ad90 2023-10-17T01:29:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.321+0000 7fce317fa700 1 -- 172.21.15.45:0/2125698793 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c014480 con 0x55b6e9b1ad90 2023-10-17T01:29:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/2125698793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 msgr2=0x55b6e9b13a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b13a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00eaf0 tx=0x7fce3c009700 comp rx=0 tx=0).stop 2023-10-17T01:29:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/2125698793 shutdown_connections 2023-10-17T01:29:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b13a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/2125698793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1a190 0x55b6e9b22100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/2125698793 >> 172.21.15.45:0/2125698793 conn(0x55b6e9a79070 msgr2=0x55b6e9b14290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/2125698793 shutdown_connections 2023-10-17T01:29:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.322+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/2125698793 wait complete. 2023-10-17T01:29:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.323+0000 7fce5c7c9b80 1 -- start start 2023-10-17T01:29:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.323+0000 7fce5c7c9b80 1 Processor -- start 2023-10-17T01:29:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b83100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b85650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b87ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9b28b60 con 0x55b6e9b1ad90 2023-10-17T01:29:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9b880e0 con 0x55b6e9b1a190 2023-10-17T01:29:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e9b88250 con 0x55b6e9b19830 2023-10-17T01:29:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce5c7c9b80 10 client.0 _ll_register_callbacks cb 0x55b6e9954a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce42d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b87ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b83100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce42d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b87ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59708/0 (socket says 172.21.15.45:59708) 2023-10-17T01:29:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce42d9d700 1 -- 172.21.15.45:0/1853595365 learned_addr learned my addr 172.21.15.45:0/1853595365 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.324+0000 7fce41d9b700 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b85650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce4259c700 1 -- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 msgr2=0x55b6e9b85650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce4259c700 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b85650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce4259c700 1 -- 172.21.15.45:0/1853595365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1ad90 msgr2=0x55b6e9b87ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce4259c700 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b87ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce4259c700 1 -- 172.21.15.45:0/1853595365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c00e140 con 0x55b6e9b19830 2023-10-17T01:29:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce4259c700 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b83100 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fce34005d90 tx=0x7fce340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607973a2094ac9a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.325+0000 7fce29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:29:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.327+0000 7fce29ffb700 1 -- 172.21.15.45:0/1853595365 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34012ab0 con 0x55b6e9b19830 2023-10-17T01:29:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.327+0000 7fce29ffb700 1 -- 172.21.15.45:0/1853595365 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fce34025070 con 0x55b6e9b19830 2023-10-17T01:29:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.328+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce3c00f040 con 0x55b6e9b19830 2023-10-17T01:29:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.330+0000 7fce29ffb700 1 -- 172.21.15.45:0/1853595365 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce34012c10 con 0x55b6e9b19830 2023-10-17T01:29:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.330+0000 7fce5c7c9b80 10 client.26923 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.330+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b6e9898d80 con 0x55b6e9b19830 2023-10-17T01:29:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce29ffb700 1 -- 172.21.15.45:0/1853595365 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3c00f040 con 0x55b6e9b19830 2023-10-17T01:29:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce29ffb700 1 -- 172.21.15.45:0/1853595365 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fce3403b6b0 con 0x55b6e9b19830 2023-10-17T01:29:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce5c7c9b80 10 client.26923 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce5c7c9b80 10 client.26923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce5c7c9b80 20 client.26923 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b89ca0 0x55b6e9b8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce5c7c9b80 4 client.26923 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:29:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6e9a63ce0 con 0x55b6e9b89ca0 2023-10-17T01:29:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.331+0000 7fce41d9b700 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b89ca0 0x55b6e9b8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.332+0000 7fce41d9b700 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b89ca0 0x55b6e9b8c0c0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1ef987baa88e10 server_cookie=4cea97eac78e3ec7 in_seq=0 out_seq=0 2023-10-17T01:29:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.332+0000 7fce29ffb700 10 client.26923 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce29ffb700 1 -- 172.21.15.45:0/1853595365 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6e9a63ce0 con 0x55b6e9b89ca0 2023-10-17T01:29:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce29ffb700 10 client.26923 handle_command_reply: tid=0 2023-10-17T01:29:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 client.26923 shutdown 2023-10-17T01:29:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 msgr2=0x55b6e9b83100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b83100 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fce34005d90 tx=0x7fce340047c0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 shutdown_connections 2023-10-17T01:29:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6e9b1ad90 0x55b6e9b87ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b6e9b19830 0x55b6e9b83100 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55b6e9b89ca0 0x55b6e9b8c0c0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 --2- 172.21.15.45:0/1853595365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b6e9b1a190 0x55b6e9b85650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce29ffb700 0 client.26923 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 >> 172.21.15.45:0/1853595365 conn(0x55b6e9a79070 msgr2=0x55b6e9b211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 shutdown_connections 2023-10-17T01:29:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 1 -- 172.21.15.45:0/1853595365 wait complete. 2023-10-17T01:29:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.350+0000 7fce5c7c9b80 20 client.26923 trim_cache size 0 max 16384 2023-10-17T01:29:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:29:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:29:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:29:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:29:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.351+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce2c0a0c70 msgr2=0x7fce2c0a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.351+0000 7fce48dff700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce2c0a0c70 0x7fce2c0a3130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fce44199b40 tx=0x7fce34009260 comp rx=0 tx=0).stop 2023-10-17T01:29:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.351+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 msgr2=0x7fce441992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 0x7fce441992e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fce38000c00 tx=0x7fce380057d0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 shutdown_connections 2023-10-17T01:29:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fce2c0a0c70 0x7fce2c0a3130 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce44102910 0x7fce4419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce44100d80 0x7fce44198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 --2- 172.21.15.45:0/494833787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce44101f70 0x7fce441992e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.352+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 >> 172.21.15.45:0/494833787 conn(0x7fce440fc2d0 msgr2=0x7fce440fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.353+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 shutdown_connections 2023-10-17T01:29:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:18.353+0000 7fce48dff700 1 -- 172.21.15.45:0/494833787 wait complete. 2023-10-17T01:29:18.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:18 smithi045 ceph-mon[98474]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 553 op/s 2023-10-17T01:29:18.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:18 smithi060 ceph-mon[134136]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 553 op/s 2023-10-17T01:29:18.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:18 smithi181 ceph-mon[134520]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.8 MiB/s wr, 553 op/s 2023-10-17T01:29:20.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:20 smithi045 ceph-mon[98474]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 585 op/s 2023-10-17T01:29:20.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:20 smithi060 ceph-mon[134136]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 585 op/s 2023-10-17T01:29:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 850.0 s, 1590.4 tps, lat 62.671 ms stddev 83.329 2023-10-17T01:29:20.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:20 smithi181 ceph-mon[134520]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 585 op/s 2023-10-17T01:29:22.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:22 smithi045 ceph-mon[98474]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 645 op/s 2023-10-17T01:29:22.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:22 smithi060 ceph-mon[134136]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 645 op/s 2023-10-17T01:29:22.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:22 smithi181 ceph-mon[134520]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 MiB/s wr, 645 op/s 2023-10-17T01:29:24.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:24 smithi045 ceph-mon[98474]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 559 op/s 2023-10-17T01:29:24.581 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:24 smithi060 ceph-mon[134136]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 559 op/s 2023-10-17T01:29:24.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:24 smithi181 ceph-mon[134520]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 559 op/s 2023-10-17T01:29:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 855.0 s, 1229.8 tps, lat 80.542 ms stddev 92.245 2023-10-17T01:29:26.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:26 smithi045 ceph-mon[98474]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 707 op/s 2023-10-17T01:29:26.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:26 smithi060 ceph-mon[134136]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 707 op/s 2023-10-17T01:29:26.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:26 smithi181 ceph-mon[134520]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 707 op/s 2023-10-17T01:29:28.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:28 smithi060 ceph-mon[134136]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 MiB/s wr, 579 op/s 2023-10-17T01:29:28.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:28 smithi181 ceph-mon[134520]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 MiB/s wr, 579 op/s 2023-10-17T01:29:28.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:28 smithi045 ceph-mon[98474]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 MiB/s wr, 579 op/s 2023-10-17T01:29:30.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 860.0 s, 1295.6 tps, lat 77.814 ms stddev 92.079 2023-10-17T01:29:30.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:30 smithi181 ceph-mon[134520]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.4 MiB/s wr, 630 op/s 2023-10-17T01:29:30.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:30 smithi045 ceph-mon[98474]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.4 MiB/s wr, 630 op/s 2023-10-17T01:29:30.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:30 smithi060 ceph-mon[134136]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.4 MiB/s wr, 630 op/s 2023-10-17T01:29:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.593+0000 7fbff1bff700 1 -- 172.21.15.45:0/1946203162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 msgr2=0x7fbfec108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.593+0000 7fbff1bff700 1 --2- 172.21.15.45:0/1946203162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec108170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0098f0 tx=0x7fbfdc009c00 comp rx=0 tx=0).stop 2023-10-17T01:29:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 -- 172.21.15.45:0/1946203162 shutdown_connections 2023-10-17T01:29:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 --2- 172.21.15.45:0/1946203162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 0x7fbfec1119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 --2- 172.21.15.45:0/1946203162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec108170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 --2- 172.21.15.45:0/1946203162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 0x7fbfec10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 -- 172.21.15.45:0/1946203162 >> 172.21.15.45:0/1946203162 conn(0x7fbfec06df20 msgr2=0x7fbfec070380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 -- 172.21.15.45:0/1946203162 shutdown_connections 2023-10-17T01:29:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.594+0000 7fbff1bff700 1 -- 172.21.15.45:0/1946203162 wait complete. 2023-10-17T01:29:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.596+0000 7fbff1bff700 1 Processor -- start 2023-10-17T01:29:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.599+0000 7fbff1bff700 1 -- start start 2023-10-17T01:29:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.601+0000 7fbff1bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec1a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.601+0000 7fbff1bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 0x7fbfec1a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbff1bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 0x7fbfec1a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbff1bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1a7490 con 0x7fbfec107d50 2023-10-17T01:29:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbff1bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1a7600 con 0x7fbfec1086b0 2023-10-17T01:29:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbff1bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1a77c0 con 0x7fbfec10f5d0 2023-10-17T01:29:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 0x7fbfec1a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 0x7fbfec1a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44172/0 (socket says 172.21.15.45:44172) 2023-10-17T01:29:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.602+0000 7fbfebfff700 1 -- 172.21.15.45:0/3654003422 learned_addr learned my addr 172.21.15.45:0/3654003422 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.603+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 0x7fbfec1a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.604+0000 7fbfeaffd700 1 -- 172.21.15.45:0/3654003422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 msgr2=0x7fbfec1a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.604+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 0x7fbfec1a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.604+0000 7fbfeaffd700 1 -- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 msgr2=0x7fbfec1a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.605+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec1a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.605+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec1a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.605+0000 7fbfeaffd700 1 -- 172.21.15.45:0/3654003422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc009520 con 0x7fbfec1086b0 2023-10-17T01:29:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.605+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec1a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.605+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 0x7fbfec1a5fb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000b450 tx=0x7fbfe000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.606+0000 7fbfe8ff9700 1 -- 172.21.15.45:0/3654003422 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0011830 con 0x7fbfec1086b0 2023-10-17T01:29:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.606+0000 7fbfe8ff9700 1 -- 172.21.15.45:0/3654003422 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfe0011e50 con 0x7fbfec1086b0 2023-10-17T01:29:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.606+0000 7fbfe8ff9700 1 -- 172.21.15.45:0/3654003422 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000f2e0 con 0x7fbfec1086b0 2023-10-17T01:29:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.606+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec1a7aa0 con 0x7fbfec1086b0 2023-10-17T01:29:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.607+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec1ae260 con 0x7fbfec1086b0 2023-10-17T01:29:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.615+0000 7fc0055ecb80 1 Processor -- start 2023-10-17T01:29:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.615+0000 7fbfe8ff9700 1 -- 172.21.15.45:0/3654003422 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fbfe00105a0 con 0x7fbfec1086b0 2023-10-17T01:29:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.616+0000 7fc0055ecb80 1 -- start start 2023-10-17T01:29:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfedd830 0x5654bfeddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfede790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfeded90 0x5654bfedf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfe359b0 con 0x5654bfeded90 2023-10-17T01:29:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfe33370 con 0x5654bfedd830 2023-10-17T01:29:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfe2ec60 con 0x5654bfede190 2023-10-17T01:29:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfedd830 0x5654bfeddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfedd830 0x5654bfeddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58626/0 (socket says 172.21.15.45:58626) 2023-10-17T01:29:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fbfebfff700 1 -- 172.21.15.45:0/856534492 learned_addr learned my addr 172.21.15.45:0/856534492 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.617+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfede790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfe8ff9700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbfd40a0cf0 0x7fbfd40a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfeb7fe700 1 -- 172.21.15.45:0/856534492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfedd830 msgr2=0x5654bfeddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfedd830 0x5654bfeddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfeb7fe700 1 -- 172.21.15.45:0/856534492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfeded90 msgr2=0x5654bfedf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfeded90 0x5654bfedf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfeb7fe700 1 -- 172.21.15.45:0/856534492 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654bfc5cd80 con 0x5654bfede190 2023-10-17T01:29:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfe8ff9700 1 -- 172.21.15.45:0/3654003422 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbfe00cbbe0 con 0x7fbfec1086b0 2023-10-17T01:29:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfede790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009810 tx=0x7fbfdc009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f15e3d3efea1b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.618+0000 7fbfebfff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbfd40a0cf0 0x7fbfd40a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.619+0000 7fbfd1ffb700 1 -- 172.21.15.45:0/856534492 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc01d070 con 0x5654bfede190 2023-10-17T01:29:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.619+0000 7fbfd1ffb700 1 -- 172.21.15.45:0/856534492 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfdc00f920 con 0x5654bfede190 2023-10-17T01:29:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.619+0000 7fbfd1ffb700 1 -- 172.21.15.45:0/856534492 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0050a0 con 0x5654bfede190 2023-10-17T01:29:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.620+0000 7fc0055ecb80 1 -- 172.21.15.45:0/856534492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 msgr2=0x5654bfede790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.620+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfede790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009810 tx=0x7fbfdc009480 comp rx=0 tx=0).stop 2023-10-17T01:29:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.620+0000 7fbfebfff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbfd40a0cf0 0x7fbfd40a31b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009f50 tx=0x7fbfe4004b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 -- 172.21.15.45:0/856534492 shutdown_connections 2023-10-17T01:29:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfeded90 0x5654bfedf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfedd830 0x5654bfeddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/856534492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfede790 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 -- 172.21.15.45:0/856534492 >> 172.21.15.45:0/856534492 conn(0x5654bfe3d070 msgr2=0x5654bfed82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 -- 172.21.15.45:0/856534492 shutdown_connections 2023-10-17T01:29:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.621+0000 7fc0055ecb80 1 -- 172.21.15.45:0/856534492 wait complete. 2023-10-17T01:29:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.622+0000 7fc0055ecb80 1 -- start start 2023-10-17T01:29:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 Processor -- start 2023-10-17T01:29:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfee7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfee78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 0x5654bfeee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfdcb4b0 con 0x5654bfedd830 2023-10-17T01:29:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfdcb150 con 0x5654bfeded90 2023-10-17T01:29:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.623+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfcb28c0 con 0x5654bfede190 2023-10-17T01:29:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.624+0000 7fc0055ecb80 10 client.0 _ll_register_callbacks cb 0x5654bfd18a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.624+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 0x5654bfeee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.624+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 0x5654bfeee8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58634/0 (socket says 172.21.15.45:58634) 2023-10-17T01:29:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.624+0000 7fbfebfff700 1 -- 172.21.15.45:0/2240284553 learned_addr learned my addr 172.21.15.45:0/2240284553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.624+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfee7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.624+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfee78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfebfff700 1 -- 172.21.15.45:0/2240284553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 msgr2=0x5654bfee78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfebfff700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfee78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfebfff700 1 -- 172.21.15.45:0/2240284553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 msgr2=0x5654bfee7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfebfff700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfee7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfebfff700 1 -- 172.21.15.45:0/2240284553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec1122f0 con 0x5654bfeded90 2023-10-17T01:29:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfee78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfebfff700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 0x5654bfeee8f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4017c60 tx=0x7fbfe4009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfba268bcd8c2e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.625+0000 7fbfcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.626+0000 7fbfcaffd700 1 -- 172.21.15.45:0/2240284553 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4016e00 con 0x5654bfeded90 2023-10-17T01:29:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.626+0000 7fbfcaffd700 1 -- 172.21.15.45:0/2240284553 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbfe4022400 con 0x5654bfeded90 2023-10-17T01:29:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.626+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654bfe59cb0 con 0x5654bfeded90 2023-10-17T01:29:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.627+0000 7fbfcaffd700 1 -- 172.21.15.45:0/2240284553 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfe40225c0 con 0x5654bfeded90 2023-10-17T01:29:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.627+0000 7fc0055ecb80 10 client.26990 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.627+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5654bfc45c00 con 0x5654bfeded90 2023-10-17T01:29:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.627+0000 7fbfcaffd700 1 -- 172.21.15.45:0/2240284553 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654bfe59cb0 con 0x5654bfeded90 2023-10-17T01:29:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fbfcaffd700 1 -- 172.21.15.45:0/2240284553 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fbfe402a070 con 0x5654bfeded90 2023-10-17T01:29:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fc0055ecb80 10 client.26990 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fc0055ecb80 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fc0055ecb80 20 client.26990 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bfef3200 0x5654bfef5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fc0055ecb80 4 client.26990 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654bfeaccd0 con 0x5654bfef3200 2023-10-17T01:29:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.628+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bfef3200 0x5654bfef5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.629+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bfef3200 0x5654bfef5620 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8f2ce43b5e52a9 server_cookie=951b7ea82bc53ab3 in_seq=0 out_seq=0 2023-10-17T01:29:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.629+0000 7fbfcaffd700 10 client.26990 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.629+0000 7fbfcaffd700 1 -- 172.21.15.45:0/2240284553 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5654bfeaccd0 con 0x5654bfef3200 2023-10-17T01:29:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.629+0000 7fbfcaffd700 10 client.26990 handle_command_reply: tid=0 2023-10-17T01:29:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.629+0000 7fc0055ecb80 1 client.26990 shutdown 2023-10-17T01:29:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 msgr2=0x5654bfeee8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 0x5654bfeee8f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4017c60 tx=0x7fbfe4009040 comp rx=0 tx=0).stop 2023-10-17T01:29:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 shutdown_connections 2023-10-17T01:29:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bfef3200 0x5654bfef5620 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfeded90 0x5654bfeee8f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfee7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/2240284553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfede190 0x5654bfee78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fbfcaffd700 0 client.26990 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 >> 172.21.15.45:0/2240284553 conn(0x5654bfe3d070 msgr2=0x5654bfed7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 shutdown_connections 2023-10-17T01:29:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 1 -- 172.21.15.45:0/2240284553 wait complete. 2023-10-17T01:29:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.630+0000 7fc0055ecb80 20 client.26990 trim_cache size 0 max 16384 2023-10-17T01:29:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.652+0000 7fc0055ecb80 1 Processor -- start 2023-10-17T01:29:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.652+0000 7fc0055ecb80 1 -- start start 2023-10-17T01:29:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfeddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bfee51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bfee1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfed2990 con 0x5654bfedd830 2023-10-17T01:29:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfee4fa0 con 0x5654bfede190 2023-10-17T01:29:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bfed9000 con 0x5654bfeded90 2023-10-17T01:29:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfeaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bfee51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bfee1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfeb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfeddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfeaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bfee51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:58636/0 (socket says 172.21.15.45:58636) 2023-10-17T01:29:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfeaffd700 1 -- 172.21.15.45:0/1547615632 learned_addr learned my addr 172.21.15.45:0/1547615632 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bfee1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44212/0 (socket says 172.21.15.45:44212) 2023-10-17T01:29:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.653+0000 7fbfeaffd700 1 -- 172.21.15.45:0/1547615632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 msgr2=0x5654bfee1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bfee1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfeaffd700 1 -- 172.21.15.45:0/1547615632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 msgr2=0x5654bfeddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfeddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfeaffd700 1 -- 172.21.15.45:0/1547615632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654bfc831f0 con 0x5654bfede190 2023-10-17T01:29:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfeddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfebfff700 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bfee1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bfee51f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0095c70 tx=0x7fbfe0098dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5010617986d3445 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfd27fc700 1 -- 172.21.15.45:0/1547615632 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0099900 con 0x5654bfede190 2023-10-17T01:29:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfd27fc700 1 -- 172.21.15.45:0/1547615632 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfe0099200 con 0x5654bfede190 2023-10-17T01:29:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.654+0000 7fbfd27fc700 1 -- 172.21.15.45:0/1547615632 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00996c0 con 0x5654bfede190 2023-10-17T01:29:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 -- 172.21.15.45:0/1547615632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 msgr2=0x5654bfee51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bfee51f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0095c70 tx=0x7fbfe0098dc0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 -- 172.21.15.45:0/1547615632 shutdown_connections 2023-10-17T01:29:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bfee1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bfeddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/1547615632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bfee51f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 -- 172.21.15.45:0/1547615632 >> 172.21.15.45:0/1547615632 conn(0x5654bfe3d070 msgr2=0x5654bfee6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 -- 172.21.15.45:0/1547615632 shutdown_connections 2023-10-17T01:29:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 -- 172.21.15.45:0/1547615632 wait complete. 2023-10-17T01:29:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.655+0000 7fc0055ecb80 1 -- start start 2023-10-17T01:29:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 Processor -- start 2023-10-17T01:29:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bff3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bff3d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bff43bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bff44140 con 0x5654bfedd830 2023-10-17T01:29:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bff442b0 con 0x5654bfede190 2023-10-17T01:29:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654bff44470 con 0x5654bfeded90 2023-10-17T01:29:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fc0055ecb80 10 client.0 _ll_register_callbacks cb 0x7fbfe4017820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bff43bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fbfeb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bff3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fbfeaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bff3d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fbfebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bff43bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:44216/0 (socket says 172.21.15.45:44216) 2023-10-17T01:29:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.656+0000 7fbfebfff700 1 -- 172.21.15.45:0/3737092562 learned_addr learned my addr 172.21.15.45:0/3737092562 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfebfff700 1 -- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 msgr2=0x5654bff3d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfebfff700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bff3d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfebfff700 1 -- 172.21.15.45:0/3737092562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 msgr2=0x5654bff3cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfebfff700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bff3cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfebfff700 1 -- 172.21.15.45:0/3737092562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc026040 con 0x5654bfeded90 2023-10-17T01:29:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bff3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfeaffd700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bff3d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfebfff700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bff43bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009010 tx=0x7fbfe402bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e98b70ee5d1c7223 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:29:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.657+0000 7fbfcaffd700 1 -- 172.21.15.45:0/3737092562 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4016c50 con 0x5654bfeded90 2023-10-17T01:29:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.658+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfdc00f920 con 0x5654bfeded90 2023-10-17T01:29:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.658+0000 7fbfcaffd700 1 -- 172.21.15.45:0/3737092562 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbfe402baa0 con 0x5654bfeded90 2023-10-17T01:29:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.658+0000 7fbfcaffd700 1 -- 172.21.15.45:0/3737092562 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfe402b5b0 con 0x5654bfeded90 2023-10-17T01:29:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.658+0000 7fc0055ecb80 10 client.26938 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.659+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5654bfc45c00 con 0x5654bfeded90 2023-10-17T01:29:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.661+0000 7fbfcaffd700 1 -- 172.21.15.45:0/3737092562 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfdc00f920 con 0x5654bfeded90 2023-10-17T01:29:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.661+0000 7fbfcaffd700 1 -- 172.21.15.45:0/3737092562 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fbfe402a050 con 0x5654bfeded90 2023-10-17T01:29:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.661+0000 7fc0055ecb80 10 client.26938 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.661+0000 7fc0055ecb80 10 client.26938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.661+0000 7fc0055ecb80 20 client.26938 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.661+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bff45e40 0x5654bff48260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.662+0000 7fc0055ecb80 4 client.26938 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:29:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.662+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5654bfe27ce0 con 0x5654bff45e40 2023-10-17T01:29:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.662+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bff45e40 0x5654bff48260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.662+0000 7fbfeb7fe700 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bff45e40 0x5654bff48260 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99b161e8328cbe4 server_cookie=5b088573acc25382 in_seq=0 out_seq=0 2023-10-17T01:29:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.662+0000 7fbfcaffd700 10 client.26938 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.675+0000 7fbfcaffd700 1 -- 172.21.15.45:0/3737092562 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5654bfe27ce0 con 0x5654bff45e40 2023-10-17T01:29:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.675+0000 7fbfcaffd700 10 client.26938 handle_command_reply: tid=0 2023-10-17T01:29:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.675+0000 7fc0055ecb80 1 client.26938 shutdown 2023-10-17T01:29:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 msgr2=0x5654bff43bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bff43bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009010 tx=0x7fbfe402bf70 comp rx=0 tx=0).stop 2023-10-17T01:29:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 shutdown_connections 2023-10-17T01:29:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654bfeded90 0x5654bff43bd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bfedd830 0x5654bff3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5654bff45e40 0x5654bff48260 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 --2- 172.21.15.45:0/3737092562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654bfede190 0x5654bff3d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fbfcaffd700 0 client.26938 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 >> 172.21.15.45:0/3737092562 conn(0x5654bfe3d070 msgr2=0x5654bfee6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 shutdown_connections 2023-10-17T01:29:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.676+0000 7fc0055ecb80 1 -- 172.21.15.45:0/3737092562 wait complete. 2023-10-17T01:29:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.677+0000 7fc0055ecb80 20 client.26938 trim_cache size 0 max 16384 2023-10-17T01:29:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:29:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:29:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:29:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:29:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.677+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbfd40a0cf0 msgr2=0x7fbfd40a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.677+0000 7fbff1bff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbfd40a0cf0 0x7fbfd40a31b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009f50 tx=0x7fbfe4004b40 comp rx=0 tx=0).stop 2023-10-17T01:29:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.677+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 msgr2=0x7fbfec1a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.677+0000 7fbff1bff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 0x7fbfec1a5fb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000b450 tx=0x7fbfe000b810 comp rx=0 tx=0).stop 2023-10-17T01:29:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 shutdown_connections 2023-10-17T01:29:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fbfd40a0cf0 0x7fbfd40a31b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfec10f5d0 0x7fbfec1a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfec107d50 0x7fbfec1a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 --2- 172.21.15.45:0/3654003422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfec1086b0 0x7fbfec1a5fb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 >> 172.21.15.45:0/3654003422 conn(0x7fbfec06df20 msgr2=0x7fbfec06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 shutdown_connections 2023-10-17T01:29:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:31.678+0000 7fbff1bff700 1 -- 172.21.15.45:0/3654003422 wait complete. 2023-10-17T01:29:32.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:32 smithi181 ceph-mon[134520]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 MiB/s wr, 697 op/s 2023-10-17T01:29:32.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:32 smithi045 ceph-mon[98474]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 MiB/s wr, 697 op/s 2023-10-17T01:29:32.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:32 smithi060 ceph-mon[134136]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 MiB/s wr, 697 op/s 2023-10-17T01:29:34.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:34 smithi181 ceph-mon[134520]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.5 MiB/s wr, 598 op/s 2023-10-17T01:29:34.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:34 smithi045 ceph-mon[98474]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.5 MiB/s wr, 598 op/s 2023-10-17T01:29:34.835 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:34 smithi060 ceph-mon[134136]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.5 MiB/s wr, 598 op/s 2023-10-17T01:29:35.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 865.0 s, 1329.4 tps, lat 75.173 ms stddev 89.665 2023-10-17T01:29:36.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:36 smithi181 ceph-mon[134520]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-10-17T01:29:36.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:36 smithi045 ceph-mon[98474]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-10-17T01:29:36.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:36 smithi060 ceph-mon[134136]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-10-17T01:29:38.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:38 smithi181 ceph-mon[134520]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 595 op/s 2023-10-17T01:29:38.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:38 smithi045 ceph-mon[98474]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 595 op/s 2023-10-17T01:29:38.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:38 smithi060 ceph-mon[134136]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 595 op/s 2023-10-17T01:29:39.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:29:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 -- 172.21.15.45:0/4135105967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9348107d50 msgr2=0x7f9348108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 --2- 172.21.15.45:0/4135105967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9348107d50 0x7f9348108170 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9344009990 tx=0x7f9344009ca0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 -- 172.21.15.45:0/4135105967 shutdown_connections 2023-10-17T01:29:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 --2- 172.21.15.45:0/4135105967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f934810f5d0 0x7f93481119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 --2- 172.21.15.45:0/4135105967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9348107d50 0x7f9348108170 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 --2- 172.21.15.45:0/4135105967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93481086b0 0x7f934810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 -- 172.21.15.45:0/4135105967 >> 172.21.15.45:0/4135105967 conn(0x7f934806db00 msgr2=0x7f934806ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 -- 172.21.15.45:0/4135105967 shutdown_connections 2023-10-17T01:29:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.267+0000 7f934f9ff700 1 -- 172.21.15.45:0/4135105967 wait complete. 2023-10-17T01:29:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 Processor -- start 2023-10-17T01:29:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 -- start start 2023-10-17T01:29:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 0x7f934810e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93481086b0 0x7f934810ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934810f5d0 0x7f934810b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934810bd10 con 0x7f934810f5d0 2023-10-17T01:29:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934810be80 con 0x7f93481086b0 2023-10-17T01:29:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.269+0000 7f934f9ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934810c040 con 0x7f9348107d50 2023-10-17T01:29:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 0x7f934810e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 0x7f934810e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41352/0 (socket says 172.21.15.45:41352) 2023-10-17T01:29:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 -- 172.21.15.45:0/3966112122 learned_addr learned my addr 172.21.15.45:0/3966112122 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934df9c700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934810f5d0 0x7f934810b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934cf9a700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93481086b0 0x7f934810ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 -- 172.21.15.45:0/3966112122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93481086b0 msgr2=0x7f934810ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93481086b0 0x7f934810ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 -- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934810f5d0 msgr2=0x7f934810b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934810f5d0 0x7f934810b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.270+0000 7f934d79b700 1 -- 172.21.15.45:0/3966112122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440095f0 con 0x7f9348107d50 2023-10-17T01:29:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.271+0000 7f934d79b700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 0x7f934810e9e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f934400ba50 tx=0x7f934400bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.271+0000 7f933e7fc700 1 -- 172.21.15.45:0/3966112122 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934401d070 con 0x7f9348107d50 2023-10-17T01:29:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.272+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93481ad920 con 0x7f9348107d50 2023-10-17T01:29:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.272+0000 7f933e7fc700 1 -- 172.21.15.45:0/3966112122 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9344022470 con 0x7f9348107d50 2023-10-17T01:29:40.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.272+0000 7f933e7fc700 1 -- 172.21.15.45:0/3966112122 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934400f6c0 con 0x7f9348107d50 2023-10-17T01:29:40.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.272+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93481add50 con 0x7f9348107d50 2023-10-17T01:29:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.279+0000 7f9363239b80 1 Processor -- start 2023-10-17T01:29:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.279+0000 7f9363239b80 1 -- start start 2023-10-17T01:29:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.279+0000 7f933e7fc700 1 -- 172.21.15.45:0/3966112122 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f93440225e0 con 0x7f9348107d50 2023-10-17T01:29:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.279+0000 7f9363239b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb878fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f9363239b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb879d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f9363239b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb87a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f934df9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb878fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f934df9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb878fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45046/0 (socket says 172.21.15.45:45046) 2023-10-17T01:29:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f934df9c700 1 -- 172.21.15.45:0/2205270399 learned_addr learned my addr 172.21.15.45:0/2205270399 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f9363239b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb7d0a10 con 0x55c7bb878ba0 2023-10-17T01:29:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb7ce3d0 con 0x55c7bb87a2b0 2023-10-17T01:29:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb7c9cc0 con 0x55c7bb879500 2023-10-17T01:29:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f934cf9a700 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb87a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.280+0000 7f934d79b700 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb879d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934df9c700 1 -- 172.21.15.45:0/2205270399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 msgr2=0x55c7bb879d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934df9c700 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb879d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934df9c700 1 -- 172.21.15.45:0/2205270399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 msgr2=0x55c7bb87a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934df9c700 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb87a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934df9c700 1 -- 172.21.15.45:0/2205270399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7bb5f7de0 con 0x55c7bb878ba0 2023-10-17T01:29:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934d79b700 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb879d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f934df9c700 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb878fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93400093a0 tx=0x7f9340009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48561632f63fd48a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.281+0000 7f93337fe700 1 -- 172.21.15.45:0/2205270399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340003c00 con 0x55c7bb878ba0 2023-10-17T01:29:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.282+0000 7f93337fe700 1 -- 172.21.15.45:0/2205270399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f934000f050 con 0x55c7bb878ba0 2023-10-17T01:29:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.282+0000 7f93337fe700 1 -- 172.21.15.45:0/2205270399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340014670 con 0x55c7bb878ba0 2023-10-17T01:29:40.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.282+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 msgr2=0x55c7bb878fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.282+0000 7f9363239b80 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb878fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93400093a0 tx=0x7f9340009920 comp rx=0 tx=0).stop 2023-10-17T01:29:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 shutdown_connections 2023-10-17T01:29:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb87a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb878fc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 --2- 172.21.15.45:0/2205270399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb879d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 >> 172.21.15.45:0/2205270399 conn(0x55c7bb7d80d0 msgr2=0x55c7bb873950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 shutdown_connections 2023-10-17T01:29:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 -- 172.21.15.45:0/2205270399 wait complete. 2023-10-17T01:29:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f933e7fc700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93340a0a30 0x7f93340a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f9363239b80 1 -- start start 2023-10-17T01:29:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.283+0000 7f933e7fc700 1 -- 172.21.15.45:0/3966112122 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93440cd7a0 con 0x7f9348107d50 2023-10-17T01:29:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.284+0000 7f934df9c700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93340a0a30 0x7f93340a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.284+0000 7f9363239b80 1 Processor -- start 2023-10-17T01:29:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.285+0000 7f9363239b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 0x55c7bb8828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.285+0000 7f9363239b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb879500 0x55c7bb882e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.285+0000 7f934df9c700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93340a0a30 0x7f93340a2ef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9340009550 tx=0x7f9340009ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f9363239b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb889e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.285+0000 7f934d79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 0x55c7bb8828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f934d79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 0x55c7bb8828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50876/0 (socket says 172.21.15.45:50876) 2023-10-17T01:29:40.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f934d79b700 1 -- 172.21.15.45:0/684450080 learned_addr learned my addr 172.21.15.45:0/684450080 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:40.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb766510 con 0x55c7bb879500 2023-10-17T01:29:40.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb7661b0 con 0x55c7bb878ba0 2023-10-17T01:29:40.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb64d920 con 0x55c7bb87a2b0 2023-10-17T01:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f9363239b80 10 client.0 _ll_register_callbacks cb 0x55c7bb6b3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f934df9c700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb889e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.286+0000 7f934cf9a700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb879500 0x55c7bb882e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.287+0000 7f934d79b700 1 -- 172.21.15.45:0/684450080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 msgr2=0x55c7bb889e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.287+0000 7f934d79b700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb889e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.287+0000 7f934d79b700 1 -- 172.21.15.45:0/684450080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb879500 msgr2=0x55c7bb882e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.287+0000 7f934d79b700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb879500 0x55c7bb882e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.287+0000 7f934d79b700 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9340009050 con 0x55c7bb878ba0 2023-10-17T01:29:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.288+0000 7f934d79b700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 0x55c7bb8828f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9344097a40 tx=0x7f9344022ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca358231d547373 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.288+0000 7f9330ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.288+0000 7f9330ff9700 1 -- 172.21.15.45:0/684450080 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440cdde0 con 0x55c7bb878ba0 2023-10-17T01:29:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.288+0000 7f9330ff9700 1 -- 172.21.15.45:0/684450080 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f934409b6d0 con 0x55c7bb878ba0 2023-10-17T01:29:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.288+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7bb7f4d10 con 0x55c7bb878ba0 2023-10-17T01:29:40.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.289+0000 7f9330ff9700 1 -- 172.21.15.45:0/684450080 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f934409afd0 con 0x55c7bb878ba0 2023-10-17T01:29:40.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.289+0000 7f9363239b80 10 client.27008 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:40.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.289+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c7bb5e0c60 con 0x55c7bb878ba0 2023-10-17T01:29:40.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9330ff9700 1 -- 172.21.15.45:0/684450080 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93440b7080 con 0x55c7bb878ba0 2023-10-17T01:29:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9330ff9700 1 -- 172.21.15.45:0/684450080 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f93440b8f80 con 0x55c7bb878ba0 2023-10-17T01:29:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9363239b80 10 client.27008 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9363239b80 10 client.27008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9363239b80 20 client.27008 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9363239b80 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb88e290 0x55c7bb8906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9363239b80 4 client.27008 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7bb847d30 con 0x55c7bb88e290 2023-10-17T01:29:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f934cf9a700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb88e290 0x55c7bb8906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f934cf9a700 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb88e290 0x55c7bb8906b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a20c152f06816c server_cookie=90076e36ef8e602e in_seq=0 out_seq=0 2023-10-17T01:29:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.290+0000 7f9330ff9700 10 client.27008 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9330ff9700 1 -- 172.21.15.45:0/684450080 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x55c7bb847d30 con 0x55c7bb88e290 2023-10-17T01:29:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9330ff9700 10 client.27008 handle_command_reply: tid=0 2023-10-17T01:29:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 client.27008 shutdown 2023-10-17T01:29:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 msgr2=0x55c7bb8828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 0x55c7bb8828f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9344097a40 tx=0x7f9344022ba0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 shutdown_connections 2023-10-17T01:29:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb889e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb88e290 0x55c7bb8906b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb878ba0 0x55c7bb8828f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 --2- 172.21.15.45:0/684450080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb879500 0x55c7bb882e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 >> 172.21.15.45:0/684450080 conn(0x55c7bb7d80d0 msgr2=0x55c7bb873020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9330ff9700 0 client.27008 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 shutdown_connections 2023-10-17T01:29:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.292+0000 7f9363239b80 1 -- 172.21.15.45:0/684450080 wait complete. 2023-10-17T01:29:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.293+0000 7f9363239b80 20 client.27008 trim_cache size 0 max 16384 2023-10-17T01:29:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.322+0000 7f9363239b80 1 Processor -- start 2023-10-17T01:29:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.324+0000 7f9363239b80 1 -- start start 2023-10-17T01:29:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.324+0000 7f9363239b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb8815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.324+0000 7f9363239b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb873020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.324+0000 7f9363239b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.324+0000 7f9363239b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb86b6e0 con 0x55c7bb878ba0 2023-10-17T01:29:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f9363239b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb881df0 con 0x55c7bb87a2b0 2023-10-17T01:29:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f9363239b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb870420 con 0x55c7bb879500 2023-10-17T01:29:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8902a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50890/0 (socket says 172.21.15.45:50890) 2023-10-17T01:29:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 -- 172.21.15.45:0/959473495 learned_addr learned my addr 172.21.15.45:0/959473495 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 -- 172.21.15.45:0/959473495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 msgr2=0x55c7bb873020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb873020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934d79b700 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb8815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 -- 172.21.15.45:0/959473495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 msgr2=0x55c7bb8815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb8815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934df9c700 1 -- 172.21.15.45:0/959473495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7bb61e250 con 0x55c7bb87a2b0 2023-10-17T01:29:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.325+0000 7f934d79b700 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb8815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.326+0000 7f934df9c700 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8902a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9340008fe0 tx=0x7f9340021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00dcfe8356536f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.326+0000 7f9333fff700 1 -- 172.21.15.45:0/959473495 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340005790 con 0x55c7bb87a2b0 2023-10-17T01:29:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.326+0000 7f9333fff700 1 -- 172.21.15.45:0/959473495 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9340023070 con 0x55c7bb87a2b0 2023-10-17T01:29:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.326+0000 7f9333fff700 1 -- 172.21.15.45:0/959473495 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934001f410 con 0x55c7bb87a2b0 2023-10-17T01:29:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 -- 172.21.15.45:0/959473495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 msgr2=0x55c7bb8902a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8902a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9340008fe0 tx=0x7f9340021000 comp rx=0 tx=0).stop 2023-10-17T01:29:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 -- 172.21.15.45:0/959473495 shutdown_connections 2023-10-17T01:29:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8902a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb8815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 --2- 172.21.15.45:0/959473495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb879500 0x55c7bb873020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 -- 172.21.15.45:0/959473495 >> 172.21.15.45:0/959473495 conn(0x55c7bb7d80d0 msgr2=0x55c7bb8738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 -- 172.21.15.45:0/959473495 shutdown_connections 2023-10-17T01:29:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.327+0000 7f9363239b80 1 -- 172.21.15.45:0/959473495 wait complete. 2023-10-17T01:29:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.328+0000 7f9363239b80 1 -- start start 2023-10-17T01:29:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.328+0000 7f9363239b80 1 Processor -- start 2023-10-17T01:29:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb885ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb879500 0x55c7bb8861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8871f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb887730 con 0x55c7bb878ba0 2023-10-17T01:29:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb8878a0 con 0x55c7bb879500 2023-10-17T01:29:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bb887a60 con 0x55c7bb87a2b0 2023-10-17T01:29:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.329+0000 7f9363239b80 10 client.0 _ll_register_callbacks cb 0x55c7bb6b3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934df9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8871f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934df9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8871f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41396/0 (socket says 172.21.15.45:41396) 2023-10-17T01:29:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934df9c700 1 -- 172.21.15.45:0/2953594625 learned_addr learned my addr 172.21.15.45:0/2953594625 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934df9c700 1 -- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb879500 msgr2=0x55c7bb8861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934df9c700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb879500 0x55c7bb8861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934df9c700 1 -- 172.21.15.45:0/2953594625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 msgr2=0x55c7bb885ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.330+0000 7f934d79b700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb885ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f934df9c700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb885ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f934df9c700 1 -- 172.21.15.45:0/2953594625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7bb5f7de0 con 0x55c7bb87a2b0 2023-10-17T01:29:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f934d79b700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb885ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f934df9c700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8871f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f93400093a0 tx=0x7f93400059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944232c93b222e5e server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f9330ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:29:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f9330ff9700 1 -- 172.21.15.45:0/2953594625 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934002c6f0 con 0x55c7bb87a2b0 2023-10-17T01:29:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.331+0000 7f9330ff9700 1 -- 172.21.15.45:0/2953594625 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9340025070 con 0x55c7bb87a2b0 2023-10-17T01:29:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.332+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f934000f050 con 0x55c7bb87a2b0 2023-10-17T01:29:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.332+0000 7f9330ff9700 1 -- 172.21.15.45:0/2953594625 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f934002c850 con 0x55c7bb87a2b0 2023-10-17T01:29:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.332+0000 7f9363239b80 10 client.26950 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.332+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c7bb5f62b0 con 0x55c7bb87a2b0 2023-10-17T01:29:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.332+0000 7f9330ff9700 1 -- 172.21.15.45:0/2953594625 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f934000f050 con 0x55c7bb87a2b0 2023-10-17T01:29:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.332+0000 7f9330ff9700 1 -- 172.21.15.45:0/2953594625 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7f9340046070 con 0x55c7bb87a2b0 2023-10-17T01:29:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f9363239b80 10 client.26950 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f9363239b80 10 client.26950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f9363239b80 20 client.26950 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f9363239b80 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb8e0900 0x55c7bb8e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f9363239b80 4 client.26950 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:29:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7bb7c2d40 con 0x55c7bb8e0900 2023-10-17T01:29:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.333+0000 7f934d79b700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb8e0900 0x55c7bb8e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.335+0000 7f934d79b700 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb8e0900 0x55c7bb8e2d20 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4875f5db4ee9733a server_cookie=bd017d5b6596de03 in_seq=0 out_seq=0 2023-10-17T01:29:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.336+0000 7f9330ff9700 10 client.26950 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.360+0000 7f9330ff9700 1 -- 172.21.15.45:0/2953594625 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c7bb7c2d40 con 0x55c7bb8e0900 2023-10-17T01:29:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.360+0000 7f9330ff9700 10 client.26950 handle_command_reply: tid=0 2023-10-17T01:29:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.361+0000 7f9363239b80 1 client.26950 shutdown 2023-10-17T01:29:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 msgr2=0x55c7bb8871f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8871f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f93400093a0 tx=0x7f93400059b0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 shutdown_connections 2023-10-17T01:29:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7bb87a2b0 0x55c7bb8871f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7bb878ba0 0x55c7bb885ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x55c7bb8e0900 0x55c7bb8e2d20 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 --2- 172.21.15.45:0/2953594625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7bb879500 0x55c7bb8861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 >> 172.21.15.45:0/2953594625 conn(0x55c7bb7d80d0 msgr2=0x55c7bb8902a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.363+0000 7f9330ff9700 0 client.26950 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.365+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 shutdown_connections 2023-10-17T01:29:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.365+0000 7f9363239b80 1 -- 172.21.15.45:0/2953594625 wait complete. 2023-10-17T01:29:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.365+0000 7f9363239b80 20 client.26950 trim_cache size 0 max 16384 2023-10-17T01:29:40.367 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:29:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:29:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:29:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:29:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:29:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:29:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:29:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:29:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:29:40.369 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:29:40.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93340a0a30 msgr2=0x7f93340a2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93340a0a30 0x7f93340a2ef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9340009550 tx=0x7f9340009ce0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 msgr2=0x7f934810e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:40.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 0x7f934810e9e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f934400ba50 tx=0x7f934400bb30 comp rx=0 tx=0).stop 2023-10-17T01:29:40.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 shutdown_connections 2023-10-17T01:29:40.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f93340a0a30 0x7f93340a2ef0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934810f5d0 0x7f934810b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9348107d50 0x7f934810e9e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 --2- 172.21.15.45:0/3966112122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93481086b0 0x7f934810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:40.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.366+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 >> 172.21.15.45:0/3966112122 conn(0x7f934806db00 msgr2=0x7f9348111820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:40.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.367+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 shutdown_connections 2023-10-17T01:29:40.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:29:40.367+0000 7f934f9ff700 1 -- 172.21.15.45:0/3966112122 wait complete. 2023-10-17T01:29:40.391 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:29:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 870.0 s, 1330.6 tps, lat 74.105 ms stddev 94.541 2023-10-17T01:29:40.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:40 smithi181 ceph-mon[134520]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 MiB/s wr, 668 op/s 2023-10-17T01:29:40.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:40 smithi045 ceph-mon[98474]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 MiB/s wr, 668 op/s 2023-10-17T01:29:40.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:40 smithi060 ceph-mon[134136]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 MiB/s wr, 668 op/s 2023-10-17T01:29:42.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:42 smithi045 ceph-mon[98474]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.7 MiB/s wr, 720 op/s 2023-10-17T01:29:42.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:42 smithi060 ceph-mon[134136]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.7 MiB/s wr, 720 op/s 2023-10-17T01:29:42.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:42 smithi181 ceph-mon[134520]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.7 MiB/s wr, 720 op/s 2023-10-17T01:29:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:44 smithi045 ceph-mon[98474]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 623 op/s 2023-10-17T01:29:44.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:44 smithi060 ceph-mon[134136]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 623 op/s 2023-10-17T01:29:44.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:44 smithi181 ceph-mon[134520]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 623 op/s 2023-10-17T01:29:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.944+0000 7fb6bebff700 1 -- 172.21.15.45:0/2822143983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 msgr2=0x7fb6b81043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.944+0000 7fb6bebff700 1 --2- 172.21.15.45:0/2822143983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 0x7fb6b81043c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009990 tx=0x7fb6ac009ca0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.945+0000 7fb6bebff700 1 -- 172.21.15.45:0/2822143983 shutdown_connections 2023-10-17T01:29:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.945+0000 7fb6bebff700 1 --2- 172.21.15.45:0/2822143983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b8104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.945+0000 7fb6bebff700 1 --2- 172.21.15.45:0/2822143983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b80ff5c0 0x7fb6b81019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.945+0000 7fb6bebff700 1 --2- 172.21.15.45:0/2822143983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 0x7fb6b81043c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.945+0000 7fb6bebff700 1 -- 172.21.15.45:0/2822143983 >> 172.21.15.45:0/2822143983 conn(0x7fb6b80fb1a0 msgr2=0x7fb6b80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.946+0000 7fb6bebff700 1 -- 172.21.15.45:0/2822143983 shutdown_connections 2023-10-17T01:29:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.947+0000 7fb6bebff700 1 -- 172.21.15.45:0/2822143983 wait complete. 2023-10-17T01:29:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.948+0000 7fb6bebff700 1 Processor -- start 2023-10-17T01:29:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 -- start start 2023-10-17T01:29:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b80ff5c0 0x7fb6b819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 0x7fb6b819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819ec40 con 0x7fb6b80ff5c0 2023-10-17T01:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819edb0 con 0x7fb6b8104900 2023-10-17T01:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.949+0000 7fb6bebff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819ef70 con 0x7fb6b8101f30 2023-10-17T01:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.950+0000 7fb6bd19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.950+0000 7fb6bd19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50918/0 (socket says 172.21.15.45:50918) 2023-10-17T01:29:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.950+0000 7fb6bd19c700 1 -- 172.21.15.45:0/3147790602 learned_addr learned my addr 172.21.15.45:0/3147790602 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.950+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 0x7fb6b819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.950+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b80ff5c0 0x7fb6b819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.951+0000 7fb6bd19c700 1 -- 172.21.15.45:0/3147790602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 msgr2=0x7fb6b819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.951+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 0x7fb6b819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.951+0000 7fb6bd19c700 1 -- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b80ff5c0 msgr2=0x7fb6b819d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.951+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b80ff5c0 0x7fb6b819d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.951+0000 7fb6bd19c700 1 -- 172.21.15.45:0/3147790602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0095f0 con 0x7fb6b8104900 2023-10-17T01:29:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.957+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b819e5e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b970 tx=0x7fb6b000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.960+0000 7fb6b5ffb700 1 -- 172.21.15.45:0/3147790602 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000c9b0 con 0x7fb6b8104900 2023-10-17T01:29:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.960+0000 7fb6b5ffb700 1 -- 172.21.15.45:0/3147790602 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b0014440 con 0x7fb6b8104900 2023-10-17T01:29:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.960+0000 7fb6b5ffb700 1 -- 172.21.15.45:0/3147790602 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00075d0 con 0x7fb6b8104900 2023-10-17T01:29:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.960+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b819f250 con 0x7fb6b8104900 2023-10-17T01:29:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.960+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b81a5a10 con 0x7fb6b8104900 2023-10-17T01:29:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.972+0000 7fb6b5ffb700 1 -- 172.21.15.45:0/3147790602 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb6b000cb10 con 0x7fb6b8104900 2023-10-17T01:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.975+0000 7fb6b5ffb700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6a00a0bf0 0x7fb6a00a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.975+0000 7fb6b5ffb700 1 -- 172.21.15.45:0/3147790602 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6b0006e40 con 0x7fb6b8104900 2023-10-17T01:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.975+0000 7fb6d251db80 1 Processor -- start 2023-10-17T01:29:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.977+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6a00a0bf0 0x7fb6a00a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.977+0000 7fb6d251db80 1 -- start start 2023-10-17T01:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x56306808ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808b190 0x56306808b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808bd90 0x56306808c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6d251db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563067fe29b0 con 0x56306808bd90 2023-10-17T01:29:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6d251db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563067fe0370 con 0x56306808b190 2023-10-17T01:29:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6d251db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563067fdbc60 con 0x56306808a830 2023-10-17T01:29:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6a00a0bf0 0x7fb6a00a30b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8006fd0 tx=0x7fb6a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6bc99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x56306808ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6bc99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x56306808ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41430/0 (socket says 172.21.15.45:41430) 2023-10-17T01:29:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.978+0000 7fb6bc99b700 1 -- 172.21.15.45:0/314136761 learned_addr learned my addr 172.21.15.45:0/314136761 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.979+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808bd90 0x56306808c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.979+0000 7fb6bc99b700 1 -- 172.21.15.45:0/314136761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808b190 msgr2=0x56306808b790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-17T01:29:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.980+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808b190 0x56306808b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.980+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808b190 0x56306808b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.980+0000 7fb6bc99b700 1 -- 172.21.15.45:0/314136761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808bd90 msgr2=0x56306808c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.980+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808bd90 0x56306808c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.980+0000 7fb6bc99b700 1 -- 172.21.15.45:0/314136761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563067e09d80 con 0x56306808a830 2023-10-17T01:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.980+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808bd90 0x56306808c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.981+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x56306808ac50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009730 tx=0x7fb6a8010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd2ed1222496125 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.991+0000 7fb69effd700 1 -- 172.21.15.45:0/314136761 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8014640 con 0x56306808a830 2023-10-17T01:29:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.992+0000 7fb69effd700 1 -- 172.21.15.45:0/314136761 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6a8014c60 con 0x56306808a830 2023-10-17T01:29:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.992+0000 7fb69effd700 1 -- 172.21.15.45:0/314136761 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a801fdb0 con 0x56306808a830 2023-10-17T01:29:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.993+0000 7fb6d251db80 1 -- 172.21.15.45:0/314136761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 msgr2=0x56306808ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.993+0000 7fb6d251db80 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x56306808ac50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009730 tx=0x7fb6a8010000 comp rx=0 tx=0).stop 2023-10-17T01:29:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.995+0000 7fb6d251db80 1 -- 172.21.15.45:0/314136761 shutdown_connections 2023-10-17T01:29:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.995+0000 7fb6d251db80 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808bd90 0x56306808c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.995+0000 7fb6d251db80 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x56306808ac50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.995+0000 7fb6d251db80 1 --2- 172.21.15.45:0/314136761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808b190 0x56306808b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.995+0000 7fb6d251db80 1 -- 172.21.15.45:0/314136761 >> 172.21.15.45:0/314136761 conn(0x563067fea070 msgr2=0x5630680852c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.996+0000 7fb6d251db80 1 -- 172.21.15.45:0/314136761 shutdown_connections 2023-10-17T01:29:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.997+0000 7fb6d251db80 1 -- 172.21.15.45:0/314136761 wait complete. 2023-10-17T01:29:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.997+0000 7fb6d251db80 1 -- start start 2023-10-17T01:29:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:44.998+0000 7fb6d251db80 1 Processor -- start 2023-10-17T01:29:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.000+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x563068094450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.000+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068094990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.000+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808bd90 0x56306809b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.000+0000 7fb6bc99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x563068094450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.001+0000 7fb6d251db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563067f784b0 con 0x56306808b190 2023-10-17T01:29:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.001+0000 7fb6bc99b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x563068094450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:41436/0 (socket says 172.21.15.45:41436) 2023-10-17T01:29:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.001+0000 7fb6bc99b700 1 -- 172.21.15.45:0/562731745 learned_addr learned my addr 172.21.15.45:0/562731745 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.001+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808bd90 0x56306809b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.001+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563067f78150 con 0x56306808bd90 2023-10-17T01:29:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.001+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563067e5f8c0 con 0x56306808a830 2023-10-17T01:29:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.002+0000 7fb6d251db80 10 client.0 _ll_register_callbacks cb 0x563067ec5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.002+0000 7fb6bd19c700 1 -- 172.21.15.45:0/562731745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 msgr2=0x563068094450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.002+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x563068094450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.002+0000 7fb6bd19c700 1 -- 172.21.15.45:0/562731745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 msgr2=0x563068094990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.002+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068094990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.002+0000 7fb6bd19c700 1 -- 172.21.15.45:0/562731745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a800e040 con 0x56306808bd90 2023-10-17T01:29:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.003+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x563068094450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:29:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.003+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808bd90 0x56306809b9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x56306808c750 tx=0x7fb6b0099360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5d29461f0fd049 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.003+0000 7fb697fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.004+0000 7fb697fff700 1 -- 172.21.15.45:0/562731745 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b009cc50 con 0x56306808bd90 2023-10-17T01:29:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.004+0000 7fb697fff700 1 -- 172.21.15.45:0/562731745 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6b00a0e90 con 0x56306808bd90 2023-10-17T01:29:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.004+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563068006cb0 con 0x56306808bd90 2023-10-17T01:29:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.004+0000 7fb697fff700 1 -- 172.21.15.45:0/562731745 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b00b1250 con 0x56306808bd90 2023-10-17T01:29:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.004+0000 7fb6d251db80 10 client.27017 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.004+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x563067df2c00 con 0x56306808bd90 2023-10-17T01:29:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.005+0000 7fb697fff700 1 -- 172.21.15.45:0/562731745 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563068006cb0 con 0x56306808bd90 2023-10-17T01:29:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.005+0000 7fb697fff700 1 -- 172.21.15.45:0/562731745 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fb6b0097e10 con 0x56306808bd90 2023-10-17T01:29:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6d251db80 10 client.27017 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6d251db80 10 client.27017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6d251db80 20 client.27017 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6d251db80 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680a0250 0x5630680a2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6d251db80 4 client.27017 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563068059cd0 con 0x5630680a0250 2023-10-17T01:29:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.006+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680a0250 0x5630680a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.007+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680a0250 0x5630680a2670 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4153cf54c58b9e9 server_cookie=b4944552925807c4 in_seq=0 out_seq=0 2023-10-17T01:29:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.007+0000 7fb697fff700 10 client.27017 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.008+0000 7fb697fff700 1 -- 172.21.15.45:0/562731745 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x563068059cd0 con 0x5630680a0250 2023-10-17T01:29:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.008+0000 7fb697fff700 10 client.27017 handle_command_reply: tid=0 2023-10-17T01:29:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.008+0000 7fb6d251db80 1 client.27017 shutdown 2023-10-17T01:29:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808bd90 msgr2=0x56306809b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808bd90 0x56306809b9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x56306808c750 tx=0x7fb6b0099360 comp rx=0 tx=0).stop 2023-10-17T01:29:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 shutdown_connections 2023-10-17T01:29:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680a0250 0x5630680a2670 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808bd90 0x56306809b9d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808a830 0x563068094450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb697fff700 0 client.27017 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 --2- 172.21.15.45:0/562731745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068094990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.009+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 >> 172.21.15.45:0/562731745 conn(0x563067fea070 msgr2=0x563068084a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.010+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 shutdown_connections 2023-10-17T01:29:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.010+0000 7fb6d251db80 1 -- 172.21.15.45:0/562731745 wait complete. 2023-10-17T01:29:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.010+0000 7fb6d251db80 20 client.27017 trim_cache size 0 max 16384 2023-10-17T01:29:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.044+0000 7fb6d251db80 1 Processor -- start 2023-10-17T01:29:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.045+0000 7fb6d251db80 1 -- start start 2023-10-17T01:29:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.045+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x563068086360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.045+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068084a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.045+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x56306809c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6d251db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306809bdf0 con 0x56306808b190 2023-10-17T01:29:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6d251db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306807f9e0 con 0x56306808a830 2023-10-17T01:29:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6d251db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306809d600 con 0x56306808bd90 2023-10-17T01:29:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068084a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068084a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45124/0 (socket says 172.21.15.45:45124) 2023-10-17T01:29:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 -- 172.21.15.45:0/4276017574 learned_addr learned my addr 172.21.15.45:0/4276017574 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x563068086360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 -- 172.21.15.45:0/4276017574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 msgr2=0x56306809c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x56306809c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 -- 172.21.15.45:0/4276017574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 msgr2=0x563068086360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x563068086360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6b7fff700 1 -- 172.21.15.45:0/4276017574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563067e301f0 con 0x56306808b190 2023-10-17T01:29:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.046+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x563068086360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.047+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068084a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009fb0 tx=0x7fb6ac00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac9805e1af0782 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.047+0000 7fb69f7fe700 1 -- 172.21.15.45:0/4276017574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00b9d0 con 0x56306808b190 2023-10-17T01:29:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.047+0000 7fb69f7fe700 1 -- 172.21.15.45:0/4276017574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ac01b070 con 0x56306808b190 2023-10-17T01:29:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.047+0000 7fb69f7fe700 1 -- 172.21.15.45:0/4276017574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac004cf0 con 0x56306808b190 2023-10-17T01:29:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 -- 172.21.15.45:0/4276017574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 msgr2=0x563068084a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068084a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009fb0 tx=0x7fb6ac00be00 comp rx=0 tx=0).stop 2023-10-17T01:29:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 -- 172.21.15.45:0/4276017574 shutdown_connections 2023-10-17T01:29:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x56306809c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x563068086360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 --2- 172.21.15.45:0/4276017574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x563068084a00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 -- 172.21.15.45:0/4276017574 >> 172.21.15.45:0/4276017574 conn(0x563067fea070 msgr2=0x563068085290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 -- 172.21.15.45:0/4276017574 shutdown_connections 2023-10-17T01:29:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.048+0000 7fb6d251db80 1 -- 172.21.15.45:0/4276017574 wait complete. 2023-10-17T01:29:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.049+0000 7fb6d251db80 1 -- start start 2023-10-17T01:29:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.049+0000 7fb6d251db80 1 Processor -- start 2023-10-17T01:29:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x5630680ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x5630680ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x5630680f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630680f0d40 con 0x56306808b190 2023-10-17T01:29:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630680f0eb0 con 0x56306808a830 2023-10-17T01:29:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630680f1070 con 0x56306808bd90 2023-10-17T01:29:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6d251db80 10 client.0 _ll_register_callbacks cb 0x7fb6b0098960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6bc99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x5630680ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6bc99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x5630680ebb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:50968/0 (socket says 172.21.15.45:50968) 2023-10-17T01:29:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.050+0000 7fb6bc99b700 1 -- 172.21.15.45:0/3787576986 learned_addr learned my addr 172.21.15.45:0/3787576986 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x5630680ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x5630680f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bc99b700 1 -- 172.21.15.45:0/3787576986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 msgr2=0x5630680f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x5630680f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bc99b700 1 -- 172.21.15.45:0/3787576986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 msgr2=0x5630680ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x5630680ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bc99b700 1 -- 172.21.15.45:0/3787576986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563067e09d80 con 0x56306808a830 2023-10-17T01:29:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.051+0000 7fb6bd19c700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x5630680f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.052+0000 7fb6bc99b700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x5630680ebb70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800ddc0 tx=0x7fb6a801f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb780c8c1f22539a server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.052+0000 7fb697fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.052+0000 7fb697fff700 1 -- 172.21.15.45:0/3787576986 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a802f040 con 0x56306808a830 2023-10-17T01:29:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.052+0000 7fb697fff700 1 -- 172.21.15.45:0/3787576986 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6a8014850 con 0x56306808a830 2023-10-17T01:29:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.052+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6a8014c60 con 0x56306808a830 2023-10-17T01:29:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.053+0000 7fb697fff700 1 -- 172.21.15.45:0/3787576986 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6a802f040 con 0x56306808a830 2023-10-17T01:29:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.053+0000 7fb6d251db80 10 client.27023 fetch_fsmap learned FSMap version 43 2023-10-17T01:29:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.053+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fb6b81058b0 con 0x56306808a830 2023-10-17T01:29:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.053+0000 7fb697fff700 1 -- 172.21.15.45:0/3787576986 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6a8009600 con 0x56306808a830 2023-10-17T01:29:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.054+0000 7fb697fff700 1 -- 172.21.15.45:0/3787576986 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 43) v1 ==== 7458+0+0 (secure 0 0 0) 0x7fb6a8032030 con 0x56306808a830 2023-10-17T01:29:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6d251db80 10 client.27023 fetch_fsmap finished waiting for FSMap version 43 2023-10-17T01:29:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6d251db80 10 client.27023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6d251db80 20 client.27023 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6d251db80 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680f2970 0x5630680f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6d251db80 4 client.27023 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:29:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fb6a8014c60 con 0x5630680f2970 2023-10-17T01:29:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.055+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680f2970 0x5630680f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:45.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.056+0000 7fb697fff700 10 client.27023 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.056+0000 7fb6b7fff700 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680f2970 0x5630680f4d90 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13dd4499f8460dd server_cookie=798cb82524ea3044 in_seq=0 out_seq=0 2023-10-17T01:29:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb697fff700 1 -- 172.21.15.45:0/3787576986 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fb6a8014c60 con 0x5630680f2970 2023-10-17T01:29:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb697fff700 10 client.27023 handle_command_reply: tid=0 2023-10-17T01:29:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 client.27023 shutdown 2023-10-17T01:29:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 msgr2=0x5630680ebb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x5630680ebb70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800ddc0 tx=0x7fb6a801f4e0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 shutdown_connections 2023-10-17T01:29:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56306808bd90 0x5630680f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5630680f2970 0x5630680f4d90 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56306808a830 0x5630680ebb70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 --2- 172.21.15.45:0/3787576986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56306808b190 0x5630680ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.065+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 >> 172.21.15.45:0/3787576986 conn(0x563067fea070 msgr2=0x56306809c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.066+0000 7fb697fff700 0 client.27023 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:29:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:29:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:29:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:29:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.066+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 shutdown_connections 2023-10-17T01:29:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.066+0000 7fb6d251db80 1 -- 172.21.15.45:0/3787576986 wait complete. 2023-10-17T01:29:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.066+0000 7fb6d251db80 20 client.27023 trim_cache size 0 max 16384 2023-10-17T01:29:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6a00a0bf0 msgr2=0x7fb6a00a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6a00a0bf0 0x7fb6a00a30b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8006fd0 tx=0x7fb6a8008040 comp rx=0 tx=0).stop 2023-10-17T01:29:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 msgr2=0x7fb6b819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b819e5e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b970 tx=0x7fb6b000bd30 comp rx=0 tx=0).stop 2023-10-17T01:29:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 shutdown_connections 2023-10-17T01:29:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb6a00a0bf0 0x7fb6a00a30b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6b8104900 0x7fb6b819e5e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b80ff5c0 0x7fb6b819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 --2- 172.21.15.45:0/3147790602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6b8101f30 0x7fb6b819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 >> 172.21.15.45:0/3147790602 conn(0x7fb6b80fb1a0 msgr2=0x7fb6b81002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 shutdown_connections 2023-10-17T01:29:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:45.067+0000 7fb6bebff700 1 -- 172.21.15.45:0/3147790602 wait complete. 2023-10-17T01:29:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 875.0 s, 1427.4 tps, lat 71.091 ms stddev 80.917 2023-10-17T01:29:45.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:45 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:29:45.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:45 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:29:46.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:45 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:29:46.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:46 smithi060 ceph-mon[134136]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.8 MiB/s wr, 760 op/s 2023-10-17T01:29:46.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:29:46.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:29:46.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:46 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:29:46.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:46 smithi181 ceph-mon[134520]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.8 MiB/s wr, 760 op/s 2023-10-17T01:29:46.969 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:29:46.969 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:29:46.970 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:46 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:29:47.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:46 smithi045 ceph-mon[98474]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.8 MiB/s wr, 760 op/s 2023-10-17T01:29:47.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:29:47.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:29:47.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:46 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:29:48.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:48 smithi060 ceph-mon[134136]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 587 op/s 2023-10-17T01:29:48.912 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:48 smithi181 ceph-mon[134520]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 587 op/s 2023-10-17T01:29:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:48 smithi045 ceph-mon[98474]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 587 op/s 2023-10-17T01:29:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 880.0 s, 1040.4 tps, lat 95.488 ms stddev 109.838 2023-10-17T01:29:50.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:50 smithi060 ceph-mon[134136]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 610 op/s 2023-10-17T01:29:50.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:50 smithi181 ceph-mon[134520]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 610 op/s 2023-10-17T01:29:51.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:50 smithi045 ceph-mon[98474]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 610 op/s 2023-10-17T01:29:52.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:51 smithi045 ceph-mon[98474]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 MiB/s wr, 658 op/s 2023-10-17T01:29:52.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:51 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:29:52.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:51 smithi060 ceph-mon[134136]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 MiB/s wr, 658 op/s 2023-10-17T01:29:52.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:51 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:29:52.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:51 smithi181 ceph-mon[134520]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 MiB/s wr, 658 op/s 2023-10-17T01:29:52.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:51 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:29:54.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:54 smithi045 ceph-mon[98474]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 583 op/s 2023-10-17T01:29:54.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:54 smithi060 ceph-mon[134136]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 583 op/s 2023-10-17T01:29:54.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:54 smithi181 ceph-mon[134520]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 583 op/s 2023-10-17T01:29:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 885.0 s, 915.8 tps, lat 109.895 ms stddev 148.349 2023-10-17T01:29:56.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:56 smithi045 ceph-mon[98474]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 792 op/s 2023-10-17T01:29:56.335 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:56 smithi060 ceph-mon[134136]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 792 op/s 2023-10-17T01:29:56.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:56 smithi181 ceph-mon[134520]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 792 op/s 2023-10-17T01:29:58.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:29:58 smithi045 ceph-mon[98474]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 658 op/s 2023-10-17T01:29:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.314+0000 7f2be61ff700 1 -- 172.21.15.45:0/2447312881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 msgr2=0x7f2be01023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.314+0000 7f2be61ff700 1 --2- 172.21.15.45:0/2447312881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 0x7f2be01023b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009990 tx=0x7f2bc8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 -- 172.21.15.45:0/2447312881 shutdown_connections 2023-10-17T01:29:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 --2- 172.21.15.45:0/2447312881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be01028f0 0x7f2be01091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 --2- 172.21.15.45:0/2447312881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be0100d60 0x7f2be0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 --2- 172.21.15.45:0/2447312881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 0x7f2be01023b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 -- 172.21.15.45:0/2447312881 >> 172.21.15.45:0/2447312881 conn(0x7f2be00fc290 msgr2=0x7f2be00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 -- 172.21.15.45:0/2447312881 shutdown_connections 2023-10-17T01:29:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.315+0000 7f2be61ff700 1 -- 172.21.15.45:0/2447312881 wait complete. 2023-10-17T01:29:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.317+0000 7f2be61ff700 1 Processor -- start 2023-10-17T01:29:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.317+0000 7f2be61ff700 1 -- start start 2023-10-17T01:29:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.318+0000 7f2be61ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 0x7f2be019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.318+0000 7f2be61ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 0x7f2be019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2be61ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be01028f0 0x7f2be019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 0x7f2be019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2be61ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be019ebb0 con 0x7f2be01028f0 2023-10-17T01:29:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2be61ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be019ed20 con 0x7f2be0100d60 2023-10-17T01:29:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2be61ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be019eee0 con 0x7f2be0101f50 2023-10-17T01:29:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 0x7f2be019d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:52784/0 (socket says 172.21.15.45:52784) 2023-10-17T01:29:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.319+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1392431557 learned_addr learned my addr 172.21.15.45:0/1392431557 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.320+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 0x7f2be019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.320+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1392431557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 msgr2=0x7f2be019d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.320+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 0x7f2be019d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.320+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be01028f0 msgr2=0x7f2be019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.320+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be01028f0 0x7f2be019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.320+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1392431557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80095f0 con 0x7f2be0100d60 2023-10-17T01:29:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.321+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 0x7f2be019d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007ee0 tx=0x7f2bd000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.321+0000 7f2bdd7fa700 1 -- 172.21.15.45:0/1392431557 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000de90 con 0x7f2be0100d60 2023-10-17T01:29:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.321+0000 7f2bdd7fa700 1 -- 172.21.15.45:0/1392431557 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd000f040 con 0x7f2be0100d60 2023-10-17T01:29:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.321+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be019f1c0 con 0x7f2be0100d60 2023-10-17T01:29:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.321+0000 7f2bdd7fa700 1 -- 172.21.15.45:0/1392431557 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0014720 con 0x7f2be0100d60 2023-10-17T01:29:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.322+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01a5980 con 0x7f2be0100d60 2023-10-17T01:29:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.326+0000 7f2bdd7fa700 1 -- 172.21.15.45:0/1392431557 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2bd0015400 con 0x7f2be0100d60 2023-10-17T01:29:58.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:29:58 smithi060 ceph-mon[134136]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 658 op/s 2023-10-17T01:29:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.328+0000 7f2bdd7fa700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2bcc0a0c60 0x7f2bcc0a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.328+0000 7f2bdd7fa700 1 -- 172.21.15.45:0/1392431557 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2bd00cbdd0 con 0x7f2be0100d60 2023-10-17T01:29:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.330+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2bcc0a0c60 0x7f2bcc0a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.331+0000 7f2bf9a5bb80 1 Processor -- start 2023-10-17T01:29:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.332+0000 7f2bf9a5bb80 1 -- start start 2023-10-17T01:29:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 0x559821a3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a3c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3cd90 0x559821a3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 0x559821a3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598219939b0 con 0x559821a3cd90 2023-10-17T01:29:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 0x559821a3bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:52798/0 (socket says 172.21.15.45:52798) 2023-10-17T01:29:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1740078500 learned_addr learned my addr 172.21.15.45:0/1740078500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821991370 con 0x559821a3b830 2023-10-17T01:29:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982198cc60 con 0x559821a3c190 2023-10-17T01:29:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.333+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2bcc0a0c60 0x7f2bcc0a3120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b5c0 tx=0x7f2bc8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdffff700 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3cd90 0x559821a3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1740078500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 msgr2=0x559821a3c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a3c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1740078500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3cd90 msgr2=0x559821a3d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3cd90 0x559821a3d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1740078500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598217bad80 con 0x559821a3b830 2023-10-17T01:29:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 0x559821a3bc50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00958f0 tx=0x7f2bd00992f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da43ba49bff0cc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.334+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/1740078500 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00993f0 con 0x559821a3b830 2023-10-17T01:29:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.335+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/1740078500 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd009d220 con 0x559821a3b830 2023-10-17T01:29:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.335+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/1740078500 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0099650 con 0x559821a3b830 2023-10-17T01:29:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 msgr2=0x559821a3bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 0x559821a3bc50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00958f0 tx=0x7f2bd00992f0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 shutdown_connections 2023-10-17T01:29:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3cd90 0x559821a3d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3b830 0x559821a3bc50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1740078500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a3c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 >> 172.21.15.45:0/1740078500 conn(0x55982199b070 msgr2=0x559821a362c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 shutdown_connections 2023-10-17T01:29:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1740078500 wait complete. 2023-10-17T01:29:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.336+0000 7f2bf9a5bb80 1 -- start start 2023-10-17T01:29:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 Processor -- start 2023-10-17T01:29:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a4c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598219294b0 con 0x559821a3c190 2023-10-17T01:29:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821929150 con 0x559821a3cd90 2023-10-17T01:29:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598218108c0 con 0x559821a3b830 2023-10-17T01:29:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bf9a5bb80 10 client.0 _ll_register_callbacks cb 0x559821876a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35308/0 (socket says 172.21.15.45:35308) 2023-10-17T01:29:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.337+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1893103055 learned_addr learned my addr 172.21.15.45:0/1893103055 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdffff700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a4c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 msgr2=0x559821a4c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a4c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1893103055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 msgr2=0x559821a458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdf7fe700 1 -- 172.21.15.45:0/1893103055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01a5980 con 0x559821a3b830 2023-10-17T01:29:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:29:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a453a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2bd009d190 tx=0x7f2bd00c9bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89780bb0aab10b12 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:29:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bbeffd700 1 -- 172.21.15.45:0/1893103055 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00b0070 con 0x559821a3b830 2023-10-17T01:29:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bbeffd700 1 -- 172.21.15.45:0/1893103055 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2bd0098af0 con 0x559821a3b830 2023-10-17T01:29:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.338+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598219b7cb0 con 0x559821a3b830 2023-10-17T01:29:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bbeffd700 1 -- 172.21.15.45:0/1893103055 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd0099650 con 0x559821a3b830 2023-10-17T01:29:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 10 client.26971 fetch_fsmap learned FSMap version 44 2023-10-17T01:29:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5598217a3c00 con 0x559821a3b830 2023-10-17T01:29:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bbeffd700 1 -- 172.21.15.45:0/1893103055 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598219b7cb0 con 0x559821a3b830 2023-10-17T01:29:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bbeffd700 1 -- 172.21.15.45:0/1893103055 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f2bd0004cb0 con 0x559821a3b830 2023-10-17T01:29:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 10 client.26971 fetch_fsmap finished waiting for FSMap version 44 2023-10-17T01:29:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 10 client.26971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 20 client.26971 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821a50e10 0x559821a53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 4 client.26971 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:29:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559821a0acd0 con 0x559821a50e10 2023-10-17T01:29:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.339+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821a50e10 0x559821a53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.340+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821a50e10 0x559821a53230 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd350ea0d7d6f6c server_cookie=5472ef5639b3cfba in_seq=0 out_seq=0 2023-10-17T01:29:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.340+0000 7f2bbeffd700 10 client.26971 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bbeffd700 1 -- 172.21.15.45:0/1893103055 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x559821a0acd0 con 0x559821a50e10 2023-10-17T01:29:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bbeffd700 10 client.26971 handle_command_reply: tid=0 2023-10-17T01:29:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 client.26971 shutdown 2023-10-17T01:29:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 msgr2=0x559821a453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a453a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2bd009d190 tx=0x7f2bd00c9bc0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 shutdown_connections 2023-10-17T01:29:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821a50e10 0x559821a53230 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a4c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bbeffd700 0 client.26971 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a453a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/1893103055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 >> 172.21.15.45:0/1893103055 conn(0x55982199b070 msgr2=0x559821a35a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 shutdown_connections 2023-10-17T01:29:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/1893103055 wait complete. 2023-10-17T01:29:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.341+0000 7f2bf9a5bb80 20 client.26971 trim_cache size 0 max 16384 2023-10-17T01:29:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.363+0000 7f2bf9a5bb80 1 Processor -- start 2023-10-17T01:29:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 -- start start 2023-10-17T01:29:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3b830 0x559821a43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a35a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a43240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821a46590 con 0x559821a3b830 2023-10-17T01:29:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821a43dc0 con 0x559821a3cd90 2023-10-17T01:29:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821a309b0 con 0x559821a3c190 2023-10-17T01:29:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bd6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a35a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bd6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a35a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35310/0 (socket says 172.21.15.45:35310) 2023-10-17T01:29:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bd6ffd700 1 -- 172.21.15.45:0/2995049328 learned_addr learned my addr 172.21.15.45:0/2995049328 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bdffff700 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a43240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3b830 0x559821a43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bd6ffd700 1 -- 172.21.15.45:0/2995049328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 msgr2=0x559821a43240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.364+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a43240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd6ffd700 1 -- 172.21.15.45:0/2995049328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3b830 msgr2=0x559821a43fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3b830 0x559821a43fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd6ffd700 1 -- 172.21.15.45:0/2995049328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598217e11f0 con 0x559821a3c190 2023-10-17T01:29:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a35a00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8015000 tx=0x7f2bc801a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be14c3ff924f782b server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd67fc700 1 -- 172.21.15.45:0/2995049328 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80050c0 con 0x559821a3c190 2023-10-17T01:29:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd67fc700 1 -- 172.21.15.45:0/2995049328 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc8029040 con 0x559821a3c190 2023-10-17T01:29:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.365+0000 7f2bd67fc700 1 -- 172.21.15.45:0/2995049328 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8030e70 con 0x559821a3c190 2023-10-17T01:29:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.366+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2995049328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 msgr2=0x559821a35a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.366+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a35a00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8015000 tx=0x7f2bc801a070 comp rx=0 tx=0).stop 2023-10-17T01:29:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.367+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2995049328 shutdown_connections 2023-10-17T01:29:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.367+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821a43240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.367+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3b830 0x559821a43fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.367+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2995049328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3c190 0x559821a35a00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.367+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2995049328 >> 172.21.15.45:0/2995049328 conn(0x55982199b070 msgr2=0x559821a32460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.368+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2995049328 shutdown_connections 2023-10-17T01:29:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.368+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2995049328 wait complete. 2023-10-17T01:29:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.368+0000 7f2bf9a5bb80 1 -- start start 2023-10-17T01:29:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 Processor -- start 2023-10-17T01:29:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a9a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a9aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821aa17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821aa1d20 con 0x559821a3c190 2023-10-17T01:29:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821aa1e60 con 0x559821a3cd90 2023-10-17T01:29:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559821ab3000 con 0x559821a3b830 2023-10-17T01:29:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bf9a5bb80 10 client.0 _ll_register_callbacks cb 0x7f2bd0098f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:29:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bdffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821aa17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bdffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821aa17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:52834/0 (socket says 172.21.15.45:52834) 2023-10-17T01:29:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bdffff700 1 -- 172.21.15.45:0/2543094153 learned_addr learned my addr 172.21.15.45:0/2543094153 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:29:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.369+0000 7f2bdf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a9a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a9aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdffff700 1 -- 172.21.15.45:0/2543094153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 msgr2=0x559821a9a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdffff700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a9a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdffff700 1 -- 172.21.15.45:0/2543094153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 msgr2=0x559821a9aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:29:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdffff700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a9aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdffff700 1 -- 172.21.15.45:0/2543094153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be0109af0 con 0x559821a3cd90 2023-10-17T01:29:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bd6ffd700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a9aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:29:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a9a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:29:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bdffff700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821aa17e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009dc0 tx=0x7f2bd8012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d080a019171e7896 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:29:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:29:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.370+0000 7f2bbeffd700 1 -- 172.21.15.45:0/2543094153 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8017070 con 0x559821a3cd90 2023-10-17T01:29:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd009d220 con 0x559821a3cd90 2023-10-17T01:29:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bbeffd700 1 -- 172.21.15.45:0/2543094153 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2bd8004020 con 0x559821a3cd90 2023-10-17T01:29:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bbeffd700 1 -- 172.21.15.45:0/2543094153 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd80041e0 con 0x559821a3cd90 2023-10-17T01:29:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 10 client.27041 fetch_fsmap learned FSMap version 44 2023-10-17T01:29:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5598217b9250 con 0x559821a3cd90 2023-10-17T01:29:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bbeffd700 1 -- 172.21.15.45:0/2543094153 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd009d220 con 0x559821a3cd90 2023-10-17T01:29:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bbeffd700 1 -- 172.21.15.45:0/2543094153 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f2bd80116e0 con 0x559821a3cd90 2023-10-17T01:29:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 10 client.27041 fetch_fsmap finished waiting for FSMap version 44 2023-10-17T01:29:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 10 client.27041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:29:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 20 client.27041 populate_metadata read hostname 'smithi045' 2023-10-17T01:29:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.371+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821ab4ad0 0x559821ab6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:29:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.372+0000 7f2bf9a5bb80 4 client.27041 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:29:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.372+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559821985ce0 con 0x559821ab4ad0 2023-10-17T01:29:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.372+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821ab4ad0 0x559821ab6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:29:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.373+0000 7f2bdf7fe700 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821ab4ad0 0x559821ab6ef0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c26ab786ced9f74 server_cookie=d64f39dc7d85fb8e in_seq=0 out_seq=0 2023-10-17T01:29:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.373+0000 7f2bbeffd700 10 client.27041 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.381+0000 7f2bbeffd700 1 -- 172.21.15.45:0/2543094153 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559821985ce0 con 0x559821ab4ad0 2023-10-17T01:29:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.381+0000 7f2bbeffd700 10 client.27041 handle_command_reply: tid=0 2023-10-17T01:29:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.381+0000 7f2bf9a5bb80 1 client.27041 shutdown 2023-10-17T01:29:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.381+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 msgr2=0x559821aa17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.381+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821aa17e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009dc0 tx=0x7f2bd8012730 comp rx=0 tx=0).stop 2023-10-17T01:29:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 shutdown_connections 2023-10-17T01:29:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x559821ab4ad0 0x559821ab6ef0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559821a3cd90 0x559821aa17e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559821a3b830 0x559821a9a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bbeffd700 0 client.27041 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:29:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 --2- 172.21.15.45:0/2543094153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559821a3c190 0x559821a9aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 >> 172.21.15.45:0/2543094153 conn(0x55982199b070 msgr2=0x559821a43fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 shutdown_connections 2023-10-17T01:29:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 1 -- 172.21.15.45:0/2543094153 wait complete. 2023-10-17T01:29:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.382+0000 7f2bf9a5bb80 20 client.27041 trim_cache size 0 max 16384 2023-10-17T01:29:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:29:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:29:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:29:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:29:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2bcc0a0c60 msgr2=0x7f2bcc0a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2bcc0a0c60 0x7f2bcc0a3120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b5c0 tx=0x7f2bc8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 msgr2=0x7f2be019d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:29:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 0x7f2be019d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007ee0 tx=0x7f2bd000d3b0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 shutdown_connections 2023-10-17T01:29:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f2bcc0a0c60 0x7f2bcc0a3120 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be01028f0 0x7f2be019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be0100d60 0x7f2be019d190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 --2- 172.21.15.45:0/1392431557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2be0101f50 0x7f2be019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:29:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 >> 172.21.15.45:0/1392431557 conn(0x7f2be00fc290 msgr2=0x7f2be010d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:29:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.384+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 shutdown_connections 2023-10-17T01:29:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:29:58.385+0000 7f2be61ff700 1 -- 172.21.15.45:0/1392431557 wait complete. 2023-10-17T01:29:58.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:29:58 smithi181 ceph-mon[134520]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 658 op/s 2023-10-17T01:30:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:00 smithi045 ceph-mon[98474]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 735 op/s 2023-10-17T01:30:00.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:00 smithi045 ceph-mon[98474]: overall HEALTH_OK 2023-10-17T01:30:00.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:00 smithi060 ceph-mon[134136]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 735 op/s 2023-10-17T01:30:00.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:00 smithi060 ceph-mon[134136]: overall HEALTH_OK 2023-10-17T01:30:00.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:00 smithi181 ceph-mon[134520]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 735 op/s 2023-10-17T01:30:00.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:00 smithi181 ceph-mon[134520]: overall HEALTH_OK 2023-10-17T01:30:00.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 890.0 s, 1075.0 tps, lat 91.118 ms stddev 111.031 2023-10-17T01:30:02.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:02 smithi045 ceph-mon[98474]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 817 op/s 2023-10-17T01:30:02.328 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:02 smithi060 ceph-mon[134136]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 817 op/s 2023-10-17T01:30:02.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:02 smithi181 ceph-mon[134520]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 817 op/s 2023-10-17T01:30:04.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:04 smithi181 ceph-mon[134520]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 725 op/s 2023-10-17T01:30:04.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:04 smithi045 ceph-mon[98474]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 725 op/s 2023-10-17T01:30:04.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:04 smithi060 ceph-mon[134136]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 725 op/s 2023-10-17T01:30:05.648 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 895.0 s, 1339.6 tps, lat 75.957 ms stddev 101.061 2023-10-17T01:30:06.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:06 smithi181 ceph-mon[134520]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 863 op/s 2023-10-17T01:30:06.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:06 smithi045 ceph-mon[98474]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 863 op/s 2023-10-17T01:30:06.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:06 smithi060 ceph-mon[134136]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 863 op/s 2023-10-17T01:30:08.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:08 smithi181 ceph-mon[134520]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 657 op/s 2023-10-17T01:30:08.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:08 smithi045 ceph-mon[98474]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 657 op/s 2023-10-17T01:30:08.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:08 smithi060 ceph-mon[134136]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 657 op/s 2023-10-17T01:30:10.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-17T01:30:10.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:10 smithi181 ceph-mon[134520]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 680 op/s 2023-10-17T01:30:10.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:10 smithi045 ceph-mon[98474]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 680 op/s 2023-10-17T01:30:10.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:10 smithi060 ceph-mon[134136]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 680 op/s 2023-10-17T01:30:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:progress: 900.0 s, 1165.8 tps, lat 86.299 ms stddev 96.826 2023-10-17T01:30:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 -- 172.21.15.45:0/633429337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10d090 msgr2=0x7f801c10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 --2- 172.21.15.45:0/633429337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10d090 0x7f801c10d540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8014009f90 tx=0x7f801400b330 comp rx=0 tx=0).stop 2023-10-17T01:30:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 -- 172.21.15.45:0/633429337 shutdown_connections 2023-10-17T01:30:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 --2- 172.21.15.45:0/633429337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10d090 0x7f801c10d540 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 --2- 172.21.15.45:0/633429337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c107d50 0x7f801c10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 --2- 172.21.15.45:0/633429337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f801c10a6c0 0x7f801c10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.804+0000 7f80227ff700 1 -- 172.21.15.45:0/633429337 >> 172.21.15.45:0/633429337 conn(0x7f801c06e4a0 msgr2=0x7f801c0708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.805+0000 7f80227ff700 1 -- 172.21.15.45:0/633429337 shutdown_connections 2023-10-17T01:30:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.806+0000 7f80227ff700 1 -- 172.21.15.45:0/633429337 wait complete. 2023-10-17T01:30:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 Processor -- start 2023-10-17T01:30:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 -- start start 2023-10-17T01:30:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f801c107d50 0x7f801c1a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10a6c0 0x7f801c1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 0x7f801c1acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1ad370 con 0x7f801c107d50 2023-10-17T01:30:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1ad4b0 con 0x7f801c10d090 2023-10-17T01:30:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.808+0000 7f80227ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1ad640 con 0x7f801c10a6c0 2023-10-17T01:30:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 0x7f801c1acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 0x7f801c1acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45372/0 (socket says 172.21.15.45:45372) 2023-10-17T01:30:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 -- 172.21.15.45:0/2109761020 learned_addr learned my addr 172.21.15.45:0/2109761020 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f801b7fe700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10a6c0 0x7f801c1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 -- 172.21.15.45:0/2109761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10a6c0 msgr2=0x7f801c1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f801bfff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f801c107d50 0x7f801c1a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10a6c0 0x7f801c1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 -- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f801c107d50 msgr2=0x7f801c1a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f801c107d50 0x7f801c1a57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.809+0000 7f8020d9c700 1 -- 172.21.15.45:0/2109761020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100095f0 con 0x7f801c10d090 2023-10-17T01:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.810+0000 7f8020d9c700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 0x7f801c1acda0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8014009af0 tx=0x7f8014004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.810+0000 7f80197fa700 1 -- 172.21.15.45:0/2109761020 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400e040 con 0x7f801c10d090 2023-10-17T01:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.810+0000 7f80197fa700 1 -- 172.21.15.45:0/2109761020 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801400be00 con 0x7f801c10d090 2023-10-17T01:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.810+0000 7f80197fa700 1 -- 172.21.15.45:0/2109761020 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140052a0 con 0x7f801c10d090 2023-10-17T01:30:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.810+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c009520 con 0x7f801c10d090 2023-10-17T01:30:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.811+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8014009c40 con 0x7f801c10d090 2023-10-17T01:30:10.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.820+0000 7f8036110b80 1 Processor -- start 2023-10-17T01:30:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.820+0000 7f80197fa700 1 -- 172.21.15.45:0/2109761020 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f801401d470 con 0x7f801c10d090 2023-10-17T01:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 -- start start 2023-10-17T01:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fdba0 0x5633a24fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 0x5633a24fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a24ff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a2455a10 con 0x5633a24fe500 2023-10-17T01:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a24533d0 con 0x5633a24ff2b0 2023-10-17T01:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.821+0000 7f8036110b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a244ecc0 con 0x5633a24fdba0 2023-10-17T01:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.822+0000 7f801bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 0x5633a24fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.822+0000 7f801bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 0x5633a24fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55060/0 (socket says 172.21.15.45:55060) 2023-10-17T01:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.822+0000 7f801bfff700 1 -- 172.21.15.45:0/710500537 learned_addr learned my addr 172.21.15.45:0/710500537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801b7fe700 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fdba0 0x5633a24fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801b7fe700 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a24ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801bfff700 1 -- 172.21.15.45:0/710500537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fdba0 msgr2=0x5633a24fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801bfff700 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fdba0 0x5633a24fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801bfff700 1 -- 172.21.15.45:0/710500537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 msgr2=0x5633a24ff730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801bfff700 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a24ff730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801bfff700 1 -- 172.21.15.45:0/710500537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633a227cde0 con 0x5633a24fe500 2023-10-17T01:30:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.823+0000 7f801bfff700 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 0x5633a24fed20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8010009e50 tx=0x7f8010010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517f8566d0219ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.824+0000 7f80197fa700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f80040a0ba0 0x7f80040a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.824+0000 7f80027fc700 1 -- 172.21.15.45:0/710500537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010010db0 con 0x5633a24fe500 2023-10-17T01:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.825+0000 7f80027fc700 1 -- 172.21.15.45:0/710500537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8010004cf0 con 0x5633a24fe500 2023-10-17T01:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.825+0000 7f80027fc700 1 -- 172.21.15.45:0/710500537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80100126b0 con 0x5633a24fe500 2023-10-17T01:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.825+0000 7f80197fa700 1 -- 172.21.15.45:0/2109761020 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f80140cd640 con 0x7f801c10d090 2023-10-17T01:30:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.825+0000 7f801b7fe700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f80040a0ba0 0x7f80040a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f801b7fe700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f80040a0ba0 0x7f80040a3060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f800c0094f0 tx=0x7f800c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 -- 172.21.15.45:0/710500537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 msgr2=0x5633a24fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 0x5633a24fed20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8010009e50 tx=0x7f8010010650 comp rx=0 tx=0).stop 2023-10-17T01:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 -- 172.21.15.45:0/710500537 shutdown_connections 2023-10-17T01:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a24ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fdba0 0x5633a24fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 --2- 172.21.15.45:0/710500537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fe500 0x5633a24fed20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 -- 172.21.15.45:0/710500537 >> 172.21.15.45:0/710500537 conn(0x5633a245d0d0 msgr2=0x5633a24f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 -- 172.21.15.45:0/710500537 shutdown_connections 2023-10-17T01:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.826+0000 7f8036110b80 1 -- 172.21.15.45:0/710500537 wait complete. 2023-10-17T01:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.827+0000 7f8036110b80 1 -- start start 2023-10-17T01:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.827+0000 7f8036110b80 1 Processor -- start 2023-10-17T01:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a2507910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 0x5633a2507e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24ff2b0 0x5633a250ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a23eb510 con 0x5633a24fdba0 2023-10-17T01:30:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a23eb1b0 con 0x5633a24fe500 2023-10-17T01:30:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a22d2920 con 0x5633a24ff2b0 2023-10-17T01:30:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f8036110b80 10 client.0 _ll_register_callbacks cb 0x5633a2338a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:30:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 0x5633a2507e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 0x5633a2507e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45386/0 (socket says 172.21.15.45:45386) 2023-10-17T01:30:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.828+0000 7f801b7fe700 1 -- 172.21.15.45:0/766136147 learned_addr learned my addr 172.21.15.45:0/766136147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f8020d9c700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24ff2b0 0x5633a250ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f801b7fe700 1 -- 172.21.15.45:0/766136147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24ff2b0 msgr2=0x5633a250ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f801b7fe700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24ff2b0 0x5633a250ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f801b7fe700 1 -- 172.21.15.45:0/766136147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 msgr2=0x5633a2507910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f801b7fe700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a2507910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f801b7fe700 1 -- 172.21.15.45:0/766136147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8010009ab0 con 0x5633a24fe500 2023-10-17T01:30:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.829+0000 7f8020d9c700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24ff2b0 0x5633a250ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:30:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f801b7fe700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 0x5633a2507e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f800c011b90 tx=0x7f800c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b8ad2b5400bdf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f7ffb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-10-17T01:30:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/766136147 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c01fc90 con 0x5633a24fe500 2023-10-17T01:30:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/766136147 <== mon.2 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f800c01fdf0 con 0x5633a24fe500 2023-10-17T01:30:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633a2479d10 con 0x5633a24fe500 2023-10-17T01:30:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/766136147 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f800c004830 con 0x5633a24fe500 2023-10-17T01:30:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.830+0000 7f8036110b80 10 client.27053 fetch_fsmap learned FSMap version 44 2023-10-17T01:30:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5633a2265c60 con 0x5633a24fe500 2023-10-17T01:30:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/766136147 <== mon.2 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633a2479d10 con 0x5633a24fe500 2023-10-17T01:30:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/766136147 <== mon.2 v2:172.21.15.60:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f800c029700 con 0x5633a24fe500 2023-10-17T01:30:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 10 client.27053 fetch_fsmap finished waiting for FSMap version 44 2023-10-17T01:30:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:30:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 20 client.27053 populate_metadata read hostname 'smithi045' 2023-10-17T01:30:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2515b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 4 client.27053 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:30:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.831+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633a24ccd30 con 0x5633a2513710 2023-10-17T01:30:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.832+0000 7f801bfff700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2515b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.832+0000 7f801bfff700 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2515b30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a434d55482f99d server_cookie=362d0438b22d1e59 in_seq=0 out_seq=0 2023-10-17T01:30:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.832+0000 7f7ffb7fe700 10 client.27053 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.833+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/766136147 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x5633a24ccd30 con 0x5633a2513710 2023-10-17T01:30:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.833+0000 7f7ffb7fe700 10 client.27053 handle_command_reply: tid=0 2023-10-17T01:30:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.834+0000 7f8036110b80 1 client.27053 shutdown 2023-10-17T01:30:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.834+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 msgr2=0x5633a2507e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.834+0000 7f8036110b80 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 0x5633a2507e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f800c011b90 tx=0x7f800c004750 comp rx=0 tx=0).stop 2023-10-17T01:30:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 shutdown_connections 2023-10-17T01:30:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24ff2b0 0x5633a250ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a2507910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2515b30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f7ffb7fe700 0 client.27053 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:10.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 --2- 172.21.15.45:0/766136147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fe500 0x5633a2507e50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 >> 172.21.15.45:0/766136147 conn(0x5633a245d0d0 msgr2=0x5633a24f8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:10.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 shutdown_connections 2023-10-17T01:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.835+0000 7f8036110b80 1 -- 172.21.15.45:0/766136147 wait complete. 2023-10-17T01:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.836+0000 7f8036110b80 20 client.27053 trim_cache size 0 max 16384 2023-10-17T01:30:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.856+0000 7f8036110b80 1 Processor -- start 2023-10-17T01:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 -- start start 2023-10-17T01:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a25056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a2502260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a25027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a25063e0 con 0x5633a24fdba0 2023-10-17T01:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a2508810 con 0x5633a24ff2b0 2023-10-17T01:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.860+0000 7f8036110b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a2506dc0 con 0x5633a24fe500 2023-10-17T01:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.862+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a2502260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.862+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a2502260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40506/0 (socket says 172.21.15.45:40506) 2023-10-17T01:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.862+0000 7f801b7fe700 1 -- 172.21.15.45:0/128961239 learned_addr learned my addr 172.21.15.45:0/128961239 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.862+0000 7f801bfff700 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a25056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.865+0000 7f801bfff700 1 -- 172.21.15.45:0/128961239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 msgr2=0x5633a2502260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.865+0000 7f801bfff700 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a2502260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.865+0000 7f801bfff700 1 -- 172.21.15.45:0/128961239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 msgr2=0x5633a25027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.865+0000 7f801bfff700 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a25027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.865+0000 7f801bfff700 1 -- 172.21.15.45:0/128961239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633a22a3250 con 0x5633a24fdba0 2023-10-17T01:30:10.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.866+0000 7f801bfff700 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a25056b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f8010009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a34c27af2c67b45 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:10.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.867+0000 7f8002ffd700 1 -- 172.21.15.45:0/128961239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010028390 con 0x5633a24fdba0 2023-10-17T01:30:10.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.867+0000 7f8002ffd700 1 -- 172.21.15.45:0/128961239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801001f040 con 0x5633a24fdba0 2023-10-17T01:30:10.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.867+0000 7f8002ffd700 1 -- 172.21.15.45:0/128961239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010028c70 con 0x5633a24fdba0 2023-10-17T01:30:10.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.868+0000 7f8036110b80 1 -- 172.21.15.45:0/128961239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 msgr2=0x5633a25056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.868+0000 7f8036110b80 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a25056b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f8010009370 comp rx=0 tx=0).stop 2023-10-17T01:30:10.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 -- 172.21.15.45:0/128961239 shutdown_connections 2023-10-17T01:30:10.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24ff2b0 0x5633a25027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24fdba0 0x5633a25056b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 --2- 172.21.15.45:0/128961239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a2502260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 -- 172.21.15.45:0/128961239 >> 172.21.15.45:0/128961239 conn(0x5633a245d0d0 msgr2=0x5633a25065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:10.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 -- 172.21.15.45:0/128961239 shutdown_connections 2023-10-17T01:30:10.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.871+0000 7f8036110b80 1 -- 172.21.15.45:0/128961239 wait complete. 2023-10-17T01:30:10.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.872+0000 7f8036110b80 1 -- start start 2023-10-17T01:30:10.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.872+0000 7f8036110b80 1 Processor -- start 2023-10-17T01:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.876+0000 7f8036110b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fdba0 0x5633a255d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.876+0000 7f8036110b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a255d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.877+0000 7f801bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fdba0 0x5633a255d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.877+0000 7f801bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fdba0 0x5633a255d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45414/0 (socket says 172.21.15.45:45414) 2023-10-17T01:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.878+0000 7f801bfff700 1 -- 172.21.15.45:0/24983297 learned_addr learned my addr 172.21.15.45:0/24983297 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.878+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24ff2b0 0x5633a2564820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.878+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a2564d60 con 0x5633a24ff2b0 2023-10-17T01:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.878+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a2564ea0 con 0x5633a24fdba0 2023-10-17T01:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.878+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a2565060 con 0x5633a24fe500 2023-10-17T01:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.878+0000 7f8036110b80 10 client.0 _ll_register_callbacks cb 0x5633a2338a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.879+0000 7f8020d9c700 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24ff2b0 0x5633a2564820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.879+0000 7f8020d9c700 1 -- 172.21.15.45:0/24983297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 msgr2=0x5633a255d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.879+0000 7f8020d9c700 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a255d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.879+0000 7f8020d9c700 1 -- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fdba0 msgr2=0x5633a255d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.880+0000 7f8020d9c700 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fdba0 0x5633a255d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.880+0000 7f8020d9c700 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633a227cde0 con 0x5633a24ff2b0 2023-10-17T01:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.880+0000 7f8020d9c700 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24ff2b0 0x5633a2564820 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f80140cb1c0 tx=0x7f801409a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb720ab02a75189 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.880+0000 7f7ffb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-17T01:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.881+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/24983297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801409a440 con 0x5633a24ff2b0 2023-10-17T01:30:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.883+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/24983297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8014025e30 con 0x5633a24ff2b0 2023-10-17T01:30:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.883+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8010004cf0 con 0x5633a24ff2b0 2023-10-17T01:30:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.883+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/24983297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f801409ad50 con 0x5633a24ff2b0 2023-10-17T01:30:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.884+0000 7f8036110b80 10 client.16662 fetch_fsmap learned FSMap version 44 2023-10-17T01:30:10.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.884+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5633a227b2b0 con 0x5633a24ff2b0 2023-10-17T01:30:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.884+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/24983297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8010004cf0 con 0x5633a24ff2b0 2023-10-17T01:30:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.884+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/24983297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f80140a28a0 con 0x5633a24ff2b0 2023-10-17T01:30:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.885+0000 7f8036110b80 10 client.16662 fetch_fsmap finished waiting for FSMap version 44 2023-10-17T01:30:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.885+0000 7f8036110b80 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-17T01:30:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.885+0000 7f8036110b80 20 client.16662 populate_metadata read hostname 'smithi045' 2023-10-17T01:30:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.885+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2568810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.885+0000 7f8036110b80 4 client.16662 mds_command: new command op to 24587 tid=0[{"prefix": "scrub status"}] 2023-10-17T01:30:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.885+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633a2447d40 con 0x5633a2513710 2023-10-17T01:30:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.904+0000 7f801bfff700 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2568810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:10.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.912+0000 7f7ffb7fe700 10 client.16662 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:10.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.912+0000 7f801bfff700 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2568810 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb47f85c76e7ed88 server_cookie=99650dd58d4ca91a in_seq=0 out_seq=0 2023-10-17T01:30:10.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.923+0000 7f7ffb7fe700 1 -- 172.21.15.45:0/24983297 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5633a2447d40 con 0x5633a2513710 2023-10-17T01:30:10.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.923+0000 7f7ffb7fe700 10 client.16662 handle_command_reply: tid=0 2023-10-17T01:30:10.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.923+0000 7f8036110b80 1 client.16662 shutdown 2023-10-17T01:30:10.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24ff2b0 msgr2=0x5633a2564820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24ff2b0 0x5633a2564820 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f80140cb1c0 tx=0x7f801409a9c0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 shutdown_connections 2023-10-17T01:30:10.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633a24ff2b0 0x5633a2564820 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x5633a2513710 0x5633a2568810 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5633a24fdba0 0x5633a255d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 --2- 172.21.15.45:0/24983297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5633a24fe500 0x5633a255d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f7ffb7fe700 0 client.16662 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:10.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 >> 172.21.15.45:0/24983297 conn(0x5633a245d0d0 msgr2=0x5633a25056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:10.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 shutdown_connections 2023-10-17T01:30:10.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 1 -- 172.21.15.45:0/24983297 wait complete. 2023-10-17T01:30:10.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.924+0000 7f8036110b80 20 client.16662 trim_cache size 0 max 16384 2023-10-17T01:30:10.928 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:30:10.928 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-17T01:30:10.928 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-17T01:30:10.928 INFO:teuthology.orchestra.run.smithi045.stdout: "fea595e1-1012-4c00-8aa3-36a726565169": { 2023-10-17T01:30:10.929 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2023-10-17T01:30:10.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fea595e1-1012-4c00-8aa3-36a726565169", 2023-10-17T01:30:10.929 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2023-10-17T01:30:10.929 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:10.929 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:10.930 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:30:10.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.925+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f80040a0ba0 msgr2=0x7f80040a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.925+0000 7f80227ff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f80040a0ba0 0x7f80040a3060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f800c0094f0 tx=0x7f800c009260 comp rx=0 tx=0).stop 2023-10-17T01:30:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.925+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 msgr2=0x7f801c1acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.925+0000 7f80227ff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 0x7f801c1acda0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8014009af0 tx=0x7f8014004d30 comp rx=0 tx=0).stop 2023-10-17T01:30:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 shutdown_connections 2023-10-17T01:30:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f80040a0ba0 0x7f80040a3060 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c10d090 0x7f801c1acda0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f801c107d50 0x7f801c1a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 --2- 172.21.15.45:0/2109761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c10a6c0 0x7f801c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 >> 172.21.15.45:0/2109761020 conn(0x7f801c06e4a0 msgr2=0x7f801c10bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 shutdown_connections 2023-10-17T01:30:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:10.926+0000 7f80227ff700 1 -- 172.21.15.45:0/2109761020 wait complete. 2023-10-17T01:30:10.958 INFO:tasks.cephfs.filesystem:scrub status for tag:fea595e1-1012-4c00-8aa3-36a726565169 - {'path': '/', 'tag': 'fea595e1-1012-4c00-8aa3-36a726565169', 'options': 'recursive,force'} 2023-10-17T01:30:10.958 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks/cephfs/filesystem.py", line 1681, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-10-17T01:30:10.968 INFO:teuthology.orchestra.run.smithi045.stdout:transaction type: 2023-10-17T01:30:10.969 INFO:teuthology.orchestra.run.smithi045.stdout:scaling factor: 32 2023-10-17T01:30:10.969 INFO:teuthology.orchestra.run.smithi045.stdout:query mode: simple 2023-10-17T01:30:10.969 INFO:teuthology.orchestra.run.smithi045.stdout:number of clients: 100 2023-10-17T01:30:10.969 INFO:teuthology.orchestra.run.smithi045.stdout:number of threads: 4 2023-10-17T01:30:10.969 INFO:teuthology.orchestra.run.smithi045.stdout:duration: 900 s 2023-10-17T01:30:10.970 INFO:teuthology.orchestra.run.smithi045.stdout:number of transactions actually processed: 1624059 2023-10-17T01:30:10.970 INFO:teuthology.orchestra.run.smithi045.stdout:latency average = 55.412 ms 2023-10-17T01:30:10.970 INFO:teuthology.orchestra.run.smithi045.stdout:latency stddev = 76.265 ms 2023-10-17T01:30:10.970 INFO:teuthology.orchestra.run.smithi045.stdout:tps = 1803.866255 (including connections establishing) 2023-10-17T01:30:10.971 INFO:teuthology.orchestra.run.smithi045.stdout:tps = 1803.881596 (excluding connections establishing) 2023-10-17T01:30:10.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-10-17T01:30:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 -- 172.21.15.45:0/3150129886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0100d80 msgr2=0x7fe9d01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/3150129886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0100d80 0x7fe9d01011a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00099c0 tx=0x7fe9c0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 -- 172.21.15.45:0/3150129886 shutdown_connections 2023-10-17T01:30:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/3150129886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 0x7fe9d01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/3150129886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0100d80 0x7fe9d01011a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/3150129886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0101f70 0x7fe9d01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 -- 172.21.15.45:0/3150129886 >> 172.21.15.45:0/3150129886 conn(0x7fe9d00fc2d0 msgr2=0x7fe9d00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 -- 172.21.15.45:0/3150129886 shutdown_connections 2023-10-17T01:30:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.593+0000 7fe9d49ff700 1 -- 172.21.15.45:0/3150129886 wait complete. 2023-10-17T01:30:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.595+0000 7fe9d49ff700 1 Processor -- start 2023-10-17T01:30:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.595+0000 7fe9d49ff700 1 -- start start 2023-10-17T01:30:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.595+0000 7fe9d49ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0100d80 0x7fe9d019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.595+0000 7fe9d49ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0101f70 0x7fe9d019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.595+0000 7fe9d49ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 0x7fe9d019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9d49ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d019ebd0 con 0x7fe9d0102910 2023-10-17T01:30:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9d49ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d019ed40 con 0x7fe9d0101f70 2023-10-17T01:30:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9d49ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d019ef00 con 0x7fe9d0100d80 2023-10-17T01:30:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0100d80 0x7fe9d019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0101f70 0x7fe9d019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0100d80 0x7fe9d019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40526/0 (socket says 172.21.15.45:40526) 2023-10-17T01:30:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9ce59c700 1 -- 172.21.15.45:0/858927805 learned_addr learned my addr 172.21.15.45:0/858927805 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 0x7fe9d019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/858927805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0100d80 msgr2=0x7fe9d019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0100d80 0x7fe9d019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 msgr2=0x7fe9d019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 0x7fe9d019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/858927805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00095f0 con 0x7fe9d0101f70 2023-10-17T01:30:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.596+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 0x7fe9d019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:30:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.597+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0101f70 0x7fe9d019d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400e9f0 tx=0x7fe9c400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.597+0000 7fe9bf7fe700 1 -- 172.21.15.45:0/858927805 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400eee0 con 0x7fe9d0101f70 2023-10-17T01:30:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.597+0000 7fe9bf7fe700 1 -- 172.21.15.45:0/858927805 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c400cbd0 con 0x7fe9d0101f70 2023-10-17T01:30:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.597+0000 7fe9bf7fe700 1 -- 172.21.15.45:0/858927805 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4018680 con 0x7fe9d0101f70 2023-10-17T01:30:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.597+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d019f1e0 con 0x7fe9d0101f70 2023-10-17T01:30:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.597+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d01a59a0 con 0x7fe9d0101f70 2023-10-17T01:30:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.602+0000 7fe9bf7fe700 1 -- 172.21.15.45:0/858927805 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fe9c400cd40 con 0x7fe9d0101f70 2023-10-17T01:30:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.602+0000 7fe9e83a9b80 1 Processor -- start 2023-10-17T01:30:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.602+0000 7fe9e83a9b80 1 -- start start 2023-10-17T01:30:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc20790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cb779b0 con 0x56279cc1f830 2023-10-17T01:30:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cb75370 con 0x56279cc20d90 2023-10-17T01:30:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cb70c60 con 0x56279cc20190 2023-10-17T01:30:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc21210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45454/0 (socket says 172.21.15.45:45454) 2023-10-17T01:30:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/3726025221 learned_addr learned my addr 172.21.15.45:0/3726025221 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.603+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc20790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.604+0000 7fe9ced9d700 1 -- 172.21.15.45:0/3726025221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 msgr2=0x56279cc20790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.604+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc20790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.604+0000 7fe9ced9d700 1 -- 172.21.15.45:0/3726025221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 msgr2=0x56279cc21210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.604+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc21210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.604+0000 7fe9ced9d700 1 -- 172.21.15.45:0/3726025221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56279c99ed80 con 0x56279cc1f830 2023-10-17T01:30:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.604+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc20790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:30:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc1fc50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009d90 tx=0x7fe9c80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd8446937882e66 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9bf7fe700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe9b80a0ba0 0x7fe9b80a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9bf7fe700 1 -- 172.21.15.45:0/858927805 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe9c4014070 con 0x7fe9d0101f70 2023-10-17T01:30:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe9b80a0ba0 0x7fe9b80a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9bcff9700 1 -- 172.21.15.45:0/3726025221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8014610 con 0x56279cc1f830 2023-10-17T01:30:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9bcff9700 1 -- 172.21.15.45:0/3726025221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c800f050 con 0x56279cc1f830 2023-10-17T01:30:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.605+0000 7fe9bcff9700 1 -- 172.21.15.45:0/3726025221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8014610 con 0x56279cc1f830 2023-10-17T01:30:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3726025221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 msgr2=0x56279cc1fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc1fc50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009d90 tx=0x7fe9c80077e0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe9b80a0ba0 0x7fe9b80a3060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x56279cc1f5d0 tx=0x7fe9c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3726025221 shutdown_connections 2023-10-17T01:30:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc21210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc1fc50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3726025221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc20790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3726025221 >> 172.21.15.45:0/3726025221 conn(0x56279cb7f070 msgr2=0x56279cc1a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3726025221 shutdown_connections 2023-10-17T01:30:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.606+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3726025221 wait complete. 2023-10-17T01:30:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.607+0000 7fe9e83a9b80 1 -- start start 2023-10-17T01:30:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.607+0000 7fe9e83a9b80 1 Processor -- start 2023-10-17T01:30:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc30910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cb0d4b0 con 0x56279cc1f830 2023-10-17T01:30:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cb0d150 con 0x56279cc20d90 2023-10-17T01:30:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279c9f48c0 con 0x56279cc20190 2023-10-17T01:30:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9e83a9b80 10 client.0 _ll_register_callbacks cb 0x56279ca5aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:30:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9ced9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc30910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc298d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40534/0 (socket says 172.21.15.45:40534) 2023-10-17T01:30:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9ced9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc30910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:45464/0 (socket says 172.21.15.45:45464) 2023-10-17T01:30:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/2762619728 learned_addr learned my addr 172.21.15.45:0/2762619728 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.608+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 msgr2=0x56279cc30910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc30910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/2762619728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 msgr2=0x56279cc29390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc29390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/2762619728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d0109b10 con 0x56279cc20190 2023-10-17T01:30:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc30910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:30:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc298d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40963e0 tx=0x7fe9c4010710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37218a1c19a126 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:30:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/2762619728 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40994b0 con 0x56279cc20190 2023-10-17T01:30:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/2762619728 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe9c409a070 con 0x56279cc20190 2023-10-17T01:30:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.609+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56279cb9bcb0 con 0x56279cc20190 2023-10-17T01:30:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/2762619728 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c40992d0 con 0x56279cc20190 2023-10-17T01:30:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 10 client.27004 fetch_fsmap learned FSMap version 44 2023-10-17T01:30:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fe9c80099c0 con 0x56279cc20190 2023-10-17T01:30:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/2762619728 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56279cb9bcb0 con 0x56279cc20190 2023-10-17T01:30:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/2762619728 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe9c40b90e0 con 0x56279cc20190 2023-10-17T01:30:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 10 client.27004 fetch_fsmap finished waiting for FSMap version 44 2023-10-17T01:30:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 10 client.27004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:30:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 20 client.27004 populate_metadata read hostname 'smithi045' 2023-10-17T01:30:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc35150 0x56279cc37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 4 client.27004 mds_command: new command op to 24587 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-17T01:30:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.610+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56279cbeecd0 con 0x56279cc35150 2023-10-17T01:30:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.611+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc35150 0x56279cc37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.611+0000 7fe9b5ffb700 10 client.27004 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.611+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc35150 0x56279cc37570 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41dc1976aad944d server_cookie=25fd70f12c0ea2ee in_seq=0 out_seq=0 2023-10-17T01:30:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/2762619728 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17627 (crc 0 0 0) 0x56279cbeecd0 con 0x56279cc35150 2023-10-17T01:30:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9b5ffb700 10 client.27004 handle_command_reply: tid=0 2023-10-17T01:30:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 client.27004 shutdown 2023-10-17T01:30:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 msgr2=0x56279cc298d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc298d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40963e0 tx=0x7fe9c4010710 comp rx=0 tx=0).stop 2023-10-17T01:30:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 shutdown_connections 2023-10-17T01:30:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc35150 0x56279cc37570 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc30910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc29390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/2762619728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc298d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 >> 172.21.15.45:0/2762619728 conn(0x56279cb7f070 msgr2=0x56279cc19a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9b5ffb700 0 client.27004 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 shutdown_connections 2023-10-17T01:30:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.612+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/2762619728 wait complete. 2023-10-17T01:30:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.613+0000 7fe9e83a9b80 20 client.27004 trim_cache size 0 max 16384 2023-10-17T01:30:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.633+0000 7fe9e83a9b80 1 Processor -- start 2023-10-17T01:30:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 -- start start 2023-10-17T01:30:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 0x56279cc271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc20190 0x56279cc31080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cc2a290 con 0x56279cc20190 2023-10-17T01:30:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cc28840 con 0x56279cc20d90 2023-10-17T01:30:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cc28f00 con 0x56279cc1f830 2023-10-17T01:30:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 0x56279cc271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 0x56279cc271f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40536/0 (socket says 172.21.15.45:40536) 2023-10-17T01:30:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9ce59c700 1 -- 172.21.15.45:0/738122833 learned_addr learned my addr 172.21.15.45:0/738122833 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.634+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc20190 0x56279cc31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9ce59c700 1 -- 172.21.15.45:0/738122833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 msgr2=0x56279cc23d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc23d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9ce59c700 1 -- 172.21.15.45:0/738122833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc20190 msgr2=0x56279cc31080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc20190 0x56279cc31080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9ce59c700 1 -- 172.21.15.45:0/738122833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56279c987c00 con 0x56279cc1f830 2023-10-17T01:30:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 0x56279cc271f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009990 tx=0x7fe9c001a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b71cd41caf2621 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9bd7fa700 1 -- 172.21.15.45:0/738122833 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0033050 con 0x56279cc1f830 2023-10-17T01:30:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9bd7fa700 1 -- 172.21.15.45:0/738122833 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c001e040 con 0x56279cc1f830 2023-10-17T01:30:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.635+0000 7fe9bd7fa700 1 -- 172.21.15.45:0/738122833 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0033db0 con 0x56279cc1f830 2023-10-17T01:30:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/738122833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 msgr2=0x56279cc271f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 0x56279cc271f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009990 tx=0x7fe9c001a050 comp rx=0 tx=0).stop 2023-10-17T01:30:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/738122833 shutdown_connections 2023-10-17T01:30:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc23d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc1f830 0x56279cc271f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/738122833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc20190 0x56279cc31080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/738122833 >> 172.21.15.45:0/738122833 conn(0x56279cb7f070 msgr2=0x56279cc28020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.636+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/738122833 shutdown_connections 2023-10-17T01:30:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.637+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/738122833 wait complete. 2023-10-17T01:30:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.637+0000 7fe9e83a9b80 1 -- start start 2023-10-17T01:30:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.637+0000 7fe9e83a9b80 1 Processor -- start 2023-10-17T01:30:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc2da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc84cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cc85210 con 0x56279cc1f830 2023-10-17T01:30:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cc85350 con 0x56279cc20d90 2023-10-17T01:30:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279cc854e0 con 0x56279cc20190 2023-10-17T01:30:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9e83a9b80 10 client.0 _ll_register_callbacks cb 0x56279ca5aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-17T01:30:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc2dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:40550/0 (socket says 172.21.15.45:40550) 2023-10-17T01:30:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/3204952879 learned_addr learned my addr 172.21.15.45:0/3204952879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.638+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc84cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 msgr2=0x56279cc84cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc84cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/3204952879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 msgr2=0x56279cc2da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-17T01:30:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc2da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9cdd9b700 1 -- 172.21.15.45:0/3204952879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00038f0 con 0x56279cc20190 2023-10-17T01:30:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9ced9d700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc84cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-17T01:30:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9cdd9b700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc2dfb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4099900 tx=0x7fe9c4010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8836051d043da8a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-17T01:30:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/3204952879 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4099060 con 0x56279cc20190 2023-10-17T01:30:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/3204952879 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe9c409a070 con 0x56279cc20190 2023-10-17T01:30:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9c800f050 con 0x56279cc20190 2023-10-17T01:30:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/3204952879 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c40cfd80 con 0x56279cc20190 2023-10-17T01:30:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9e83a9b80 10 client.27010 fetch_fsmap learned FSMap version 44 2023-10-17T01:30:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.639+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56279c9c51f0 con 0x56279cc20190 2023-10-17T01:30:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/3204952879 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c800f050 con 0x56279cc20190 2023-10-17T01:30:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/3204952879 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 44) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe9c40b6060 con 0x56279cc20190 2023-10-17T01:30:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9e83a9b80 10 client.27010 fetch_fsmap finished waiting for FSMap version 44 2023-10-17T01:30:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9e83a9b80 10 client.27010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-17T01:30:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9e83a9b80 20 client.27010 populate_metadata read hostname 'smithi045' 2023-10-17T01:30:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc86b50 0x56279cc88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9e83a9b80 4 client.27010 mds_command: new command op to 24587 tid=0[{"prefix": "flush journal"}] 2023-10-17T01:30:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 --> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56279cb69ce0 con 0x56279cc86b50 2023-10-17T01:30:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.640+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc86b50 0x56279cc88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.641+0000 7fe9ce59c700 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc86b50 0x56279cc88f70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c7b0fbc74b2752 server_cookie=b0ffbe5b88589bb2 in_seq=0 out_seq=0 2023-10-17T01:30:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.641+0000 7fe9b5ffb700 10 client.27010 ms_handle_connect on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9b5ffb700 1 -- 172.21.15.45:0/3204952879 <== mds.0 v2:172.21.15.60:6838/165785544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56279cb69ce0 con 0x56279cc86b50 2023-10-17T01:30:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9b5ffb700 10 client.27010 handle_command_reply: tid=0 2023-10-17T01:30:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 client.27010 shutdown 2023-10-17T01:30:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 msgr2=0x56279cc2dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc2dfb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4099900 tx=0x7fe9c4010b60 comp rx=0 tx=0).stop 2023-10-17T01:30:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 shutdown_connections 2023-10-17T01:30:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:6838/165785544,v1:172.21.15.60:6839/165785544] conn(0x56279cc86b50 0x56279cc88f70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56279cc20d90 0x56279cc84cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56279cc1f830 0x56279cc2da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9b5ffb700 0 client.27010 ms_handle_reset on v2:172.21.15.60:6838/165785544 2023-10-17T01:30:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 --2- 172.21.15.45:0/3204952879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56279cc20190 0x56279cc2dfb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.649+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 >> 172.21.15.45:0/3204952879 conn(0x56279cb7f070 msgr2=0x56279cc271f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 shutdown_connections 2023-10-17T01:30:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9e83a9b80 1 -- 172.21.15.45:0/3204952879 wait complete. 2023-10-17T01:30:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9e83a9b80 20 client.27010 trim_cache size 0 max 16384 2023-10-17T01:30:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2023-10-17T01:30:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2023-10-17T01:30:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2023-10-17T01:30:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2023-10-17T01:30:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe9b80a0ba0 msgr2=0x7fe9b80a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe9b80a0ba0 0x7fe9b80a3060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x56279cc1f5d0 tx=0x7fe9c0005fb0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0101f70 msgr2=0x7fe9d019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.650+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0101f70 0x7fe9d019d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400e9f0 tx=0x7fe9c400ed00 comp rx=0 tx=0).stop 2023-10-17T01:30:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 shutdown_connections 2023-10-17T01:30:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe9b80a0ba0 0x7fe9b80a3060 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0102910 0x7fe9d019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d0100d80 0x7fe9d019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 --2- 172.21.15.45:0/858927805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d0101f70 0x7fe9d019d6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 >> 172.21.15.45:0/858927805 conn(0x7fe9d00fc2d0 msgr2=0x7fe9d00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 shutdown_connections 2023-10-17T01:30:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2023-10-17T01:30:11.651+0000 7fe9d49ff700 1 -- 172.21.15.45:0/858927805 wait complete. 2023-10-17T01:30:12.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:12 smithi181 ceph-mon[134520]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 758 op/s 2023-10-17T01:30:12.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:12 smithi045 ceph-mon[98474]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 758 op/s 2023-10-17T01:30:12.584 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:12 smithi060 ceph-mon[134136]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 758 op/s 2023-10-17T01:30:14.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:14 smithi045 ceph-mon[98474]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 794 op/s 2023-10-17T01:30:14.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:14 smithi060 ceph-mon[134136]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 794 op/s 2023-10-17T01:30:14.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:14 smithi181 ceph-mon[134520]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 794 op/s 2023-10-17T01:30:16.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:16 smithi045 ceph-mon[98474]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.10k op/s 2023-10-17T01:30:16.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:16 smithi060 ceph-mon[134136]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.10k op/s 2023-10-17T01:30:16.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:16 smithi181 ceph-mon[134520]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.10k op/s 2023-10-17T01:30:18.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:18 smithi045 ceph-mon[98474]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.60k op/s 2023-10-17T01:30:18.580 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:18 smithi060 ceph-mon[134136]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.60k op/s 2023-10-17T01:30:18.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:18 smithi181 ceph-mon[134520]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.60k op/s 2023-10-17T01:30:20.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:20 smithi045 ceph-mon[98474]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-10-17T01:30:20.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:20 smithi060 ceph-mon[134136]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-10-17T01:30:20.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:20 smithi181 ceph-mon[134520]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 1.83k op/s 2023-10-17T01:30:21.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-17T01:30:21.509 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/: 2023-10-17T01:30:21.509 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.510 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:30:21.510 INFO:teuthology.orchestra.run.smithi045.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 .. 2023-10-17T01:30:21.510 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 data 2023-10-17T01:30:21.510 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.511 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data: 2023-10-17T01:30:21.511 INFO:teuthology.orchestra.run.smithi045.stdout:total 31 2023-10-17T01:30:21.511 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 . 2023-10-17T01:30:21.511 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 .. 2023-10-17T01:30:21.512 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 17 01:15 base 2023-10-17T01:30:21.512 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Oct 17 01:14 current_logfiles 2023-10-17T01:30:21.512 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 17 01:14 global 2023-10-17T01:30:21.512 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 log 2023-10-17T01:30:21.513 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_commit_ts 2023-10-17T01:30:21.513 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_dynshmem 2023-10-17T01:30:21.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 17 01:14 pg_hba.conf 2023-10-17T01:30:21.513 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 17 01:14 pg_ident.conf 2023-10-17T01:30:21.514 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:30 pg_logical 2023-10-17T01:30:21.514 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 pg_multixact 2023-10-17T01:30:21.514 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 pg_notify 2023-10-17T01:30:21.514 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_replslot 2023-10-17T01:30:21.514 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_serial 2023-10-17T01:30:21.515 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_snapshots 2023-10-17T01:30:21.515 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:30 pg_stat 2023-10-17T01:30:21.515 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:30 pg_stat_tmp 2023-10-17T01:30:21.515 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:30 pg_subtrans 2023-10-17T01:30:21.516 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_tblspc 2023-10-17T01:30:21.516 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 pg_twophase 2023-10-17T01:30:21.516 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:30:21.516 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 17 01:30 pg_wal 2023-10-17T01:30:21.517 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:23 pg_xact 2023-10-17T01:30:21.517 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 17 01:14 postgresql.auto.conf 2023-10-17T01:30:21.517 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 17 01:14 postgresql.conf 2023-10-17T01:30:21.517 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Oct 17 01:14 postmaster.opts 2023-10-17T01:30:21.517 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.518 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base: 2023-10-17T01:30:21.518 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.518 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 17 01:15 . 2023-10-17T01:30:21.519 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.519 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 1 2023-10-17T01:30:21.519 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 12346 2023-10-17T01:30:21.519 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 17 01:15 12347 2023-10-17T01:30:21.519 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:15 pgsql_tmp 2023-10-17T01:30:21.519 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.520 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-17T01:30:21.520 INFO:teuthology.orchestra.run.smithi045.stdout:total 7185 2023-10-17T01:30:21.520 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 . 2023-10-17T01:30:21.520 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 17 01:15 .. 2023-10-17T01:30:21.521 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 112 2023-10-17T01:30:21.521 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 113 2023-10-17T01:30:21.521 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 17 01:14 12182 2023-10-17T01:30:21.521 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12182_fsm 2023-10-17T01:30:21.521 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12182_vm 2023-10-17T01:30:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12184 2023-10-17T01:30:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12186 2023-10-17T01:30:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187 2023-10-17T01:30:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12187_fsm 2023-10-17T01:30:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187_vm 2023-10-17T01:30:21.523 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12189 2023-10-17T01:30:21.523 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12191 2023-10-17T01:30:21.523 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192 2023-10-17T01:30:21.523 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12192_fsm 2023-10-17T01:30:21.524 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192_vm 2023-10-17T01:30:21.524 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12194 2023-10-17T01:30:21.524 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12196 2023-10-17T01:30:21.524 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197 2023-10-17T01:30:21.525 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12197_fsm 2023-10-17T01:30:21.525 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197_vm 2023-10-17T01:30:21.525 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12199 2023-10-17T01:30:21.525 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12201 2023-10-17T01:30:21.525 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202 2023-10-17T01:30:21.526 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12202_fsm 2023-10-17T01:30:21.526 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202_vm 2023-10-17T01:30:21.526 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12204 2023-10-17T01:30:21.526 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12206 2023-10-17T01:30:21.527 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207 2023-10-17T01:30:21.527 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12207_fsm 2023-10-17T01:30:21.527 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207_vm 2023-10-17T01:30:21.527 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12209 2023-10-17T01:30:21.527 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12211 2023-10-17T01:30:21.528 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12212 2023-10-17T01:30:21.528 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12214 2023-10-17T01:30:21.528 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12216 2023-10-17T01:30:21.528 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 17 01:14 1247 2023-10-17T01:30:21.528 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1247_fsm 2023-10-17T01:30:21.529 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1247_vm 2023-10-17T01:30:21.529 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 17 01:14 1249 2023-10-17T01:30:21.529 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1249_fsm 2023-10-17T01:30:21.530 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1249_vm 2023-10-17T01:30:21.530 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 17 01:14 1255 2023-10-17T01:30:21.530 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1255_fsm 2023-10-17T01:30:21.530 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1255_vm 2023-10-17T01:30:21.530 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:14 1259 2023-10-17T01:30:21.531 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1259_fsm 2023-10-17T01:30:21.531 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1259_vm 2023-10-17T01:30:21.531 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417 2023-10-17T01:30:21.531 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417_vm 2023-10-17T01:30:21.532 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418 2023-10-17T01:30:21.532 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418_vm 2023-10-17T01:30:21.532 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 174 2023-10-17T01:30:21.532 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 175 2023-10-17T01:30:21.532 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2187 2023-10-17T01:30:21.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224 2023-10-17T01:30:21.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224_vm 2023-10-17T01:30:21.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328 2023-10-17T01:30:21.533 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328_vm 2023-10-17T01:30:21.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336 2023-10-17T01:30:21.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336_vm 2023-10-17T01:30:21.534 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2337 2023-10-17T01:30:21.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2600 2023-10-17T01:30:21.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2600_fsm 2023-10-17T01:30:21.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2600_vm 2023-10-17T01:30:21.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601 2023-10-17T01:30:21.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2601_fsm 2023-10-17T01:30:21.535 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601_vm 2023-10-17T01:30:21.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2602 2023-10-17T01:30:21.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2602_fsm 2023-10-17T01:30:21.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2602_vm 2023-10-17T01:30:21.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2603 2023-10-17T01:30:21.536 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2603_fsm 2023-10-17T01:30:21.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2603_vm 2023-10-17T01:30:21.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604 2023-10-17T01:30:21.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604_vm 2023-10-17T01:30:21.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2605 2023-10-17T01:30:21.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2605_fsm 2023-10-17T01:30:21.537 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2605_vm 2023-10-17T01:30:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606 2023-10-17T01:30:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2606_fsm 2023-10-17T01:30:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606_vm 2023-10-17T01:30:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607 2023-10-17T01:30:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607_fsm 2023-10-17T01:30:21.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2607_vm 2023-10-17T01:30:21.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 17 01:14 2608 2023-10-17T01:30:21.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2608_fsm 2023-10-17T01:30:21.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2608_vm 2023-10-17T01:30:21.539 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 17 01:14 2609 2023-10-17T01:30:21.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2609_fsm 2023-10-17T01:30:21.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2609_vm 2023-10-17T01:30:21.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610 2023-10-17T01:30:21.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610_fsm 2023-10-17T01:30:21.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2610_vm 2023-10-17T01:30:21.540 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611 2023-10-17T01:30:21.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611_vm 2023-10-17T01:30:21.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612 2023-10-17T01:30:21.541 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2612_fsm 2023-10-17T01:30:21.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612_vm 2023-10-17T01:30:21.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613 2023-10-17T01:30:21.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613_vm 2023-10-17T01:30:21.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615 2023-10-17T01:30:21.542 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2615_fsm 2023-10-17T01:30:21.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615_vm 2023-10-17T01:30:21.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616 2023-10-17T01:30:21.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616_fsm 2023-10-17T01:30:21.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2616_vm 2023-10-17T01:30:21.543 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 17 01:14 2617 2023-10-17T01:30:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2617_fsm 2023-10-17T01:30:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2617_vm 2023-10-17T01:30:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 17 01:14 2618 2023-10-17T01:30:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2618_fsm 2023-10-17T01:30:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2618_vm 2023-10-17T01:30:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 17 01:14 2619 2023-10-17T01:30:21.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2619_fsm 2023-10-17T01:30:21.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2619_vm 2023-10-17T01:30:21.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620 2023-10-17T01:30:21.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620_vm 2023-10-17T01:30:21.545 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2650 2023-10-17T01:30:21.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2651 2023-10-17T01:30:21.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2652 2023-10-17T01:30:21.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2653 2023-10-17T01:30:21.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2654 2023-10-17T01:30:21.546 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2655 2023-10-17T01:30:21.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2656 2023-10-17T01:30:21.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2657 2023-10-17T01:30:21.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 17 01:14 2658 2023-10-17T01:30:21.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2659 2023-10-17T01:30:21.547 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2660 2023-10-17T01:30:21.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2661 2023-10-17T01:30:21.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2662 2023-10-17T01:30:21.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2663 2023-10-17T01:30:21.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2664 2023-10-17T01:30:21.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2665 2023-10-17T01:30:21.548 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2666 2023-10-17T01:30:21.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2667 2023-10-17T01:30:21.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2668 2023-10-17T01:30:21.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2669 2023-10-17T01:30:21.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2670 2023-10-17T01:30:21.549 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 17 01:14 2673 2023-10-17T01:30:21.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 17 01:14 2674 2023-10-17T01:30:21.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 17 01:14 2675 2023-10-17T01:30:21.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2678 2023-10-17T01:30:21.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2679 2023-10-17T01:30:21.550 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2680 2023-10-17T01:30:21.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2681 2023-10-17T01:30:21.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2682 2023-10-17T01:30:21.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2683 2023-10-17T01:30:21.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2684 2023-10-17T01:30:21.551 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2685 2023-10-17T01:30:21.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2686 2023-10-17T01:30:21.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2687 2023-10-17T01:30:21.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2688 2023-10-17T01:30:21.552 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2689 2023-10-17T01:30:21.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2690 2023-10-17T01:30:21.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 17 01:14 2691 2023-10-17T01:30:21.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2692 2023-10-17T01:30:21.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2693 2023-10-17T01:30:21.553 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2696 2023-10-17T01:30:21.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2699 2023-10-17T01:30:21.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2701 2023-10-17T01:30:21.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2702 2023-10-17T01:30:21.554 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2703 2023-10-17T01:30:21.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2704 2023-10-17T01:30:21.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2753 2023-10-17T01:30:21.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2753_fsm 2023-10-17T01:30:21.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2753_vm 2023-10-17T01:30:21.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2754 2023-10-17T01:30:21.555 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2755 2023-10-17T01:30:21.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2756 2023-10-17T01:30:21.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2757 2023-10-17T01:30:21.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830 2023-10-17T01:30:21.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830_vm 2023-10-17T01:30:21.556 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2831 2023-10-17T01:30:21.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832 2023-10-17T01:30:21.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832_vm 2023-10-17T01:30:21.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2833 2023-10-17T01:30:21.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834 2023-10-17T01:30:21.557 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834_vm 2023-10-17T01:30:21.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2835 2023-10-17T01:30:21.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836 2023-10-17T01:30:21.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836_vm 2023-10-17T01:30:21.558 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2837 2023-10-17T01:30:21.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 17 01:14 2838 2023-10-17T01:30:21.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2838_fsm 2023-10-17T01:30:21.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2838_vm 2023-10-17T01:30:21.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2839 2023-10-17T01:30:21.559 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840 2023-10-17T01:30:21.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840_fsm 2023-10-17T01:30:21.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2840_vm 2023-10-17T01:30:21.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2841 2023-10-17T01:30:21.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995 2023-10-17T01:30:21.560 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995_vm 2023-10-17T01:30:21.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2996 2023-10-17T01:30:21.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079 2023-10-17T01:30:21.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3079_fsm 2023-10-17T01:30:21.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079_vm 2023-10-17T01:30:21.561 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3080 2023-10-17T01:30:21.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3081 2023-10-17T01:30:21.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3085 2023-10-17T01:30:21.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118 2023-10-17T01:30:21.562 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118_vm 2023-10-17T01:30:21.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3119 2023-10-17T01:30:21.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3164 2023-10-17T01:30:21.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256 2023-10-17T01:30:21.563 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256_vm 2023-10-17T01:30:21.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3257 2023-10-17T01:30:21.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3258 2023-10-17T01:30:21.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350 2023-10-17T01:30:21.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350_vm 2023-10-17T01:30:21.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3351 2023-10-17T01:30:21.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3379 2023-10-17T01:30:21.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3380 2023-10-17T01:30:21.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381 2023-10-17T01:30:21.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381_vm 2023-10-17T01:30:21.565 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3394 2023-10-17T01:30:21.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3394_fsm 2023-10-17T01:30:21.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3394_vm 2023-10-17T01:30:21.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3395 2023-10-17T01:30:21.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439 2023-10-17T01:30:21.566 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439_vm 2023-10-17T01:30:21.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3440 2023-10-17T01:30:21.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 3455 2023-10-17T01:30:21.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3456 2023-10-17T01:30:21.567 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3456_fsm 2023-10-17T01:30:21.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3456_vm 2023-10-17T01:30:21.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466 2023-10-17T01:30:21.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466_vm 2023-10-17T01:30:21.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3467 2023-10-17T01:30:21.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3468 2023-10-17T01:30:21.568 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501 2023-10-17T01:30:21.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501_vm 2023-10-17T01:30:21.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3502 2023-10-17T01:30:21.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3503 2023-10-17T01:30:21.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3534 2023-10-17T01:30:21.569 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541 2023-10-17T01:30:21.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3541_fsm 2023-10-17T01:30:21.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541_vm 2023-10-17T01:30:21.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3542 2023-10-17T01:30:21.570 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3574 2023-10-17T01:30:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3575 2023-10-17T01:30:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576 2023-10-17T01:30:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576_vm 2023-10-17T01:30:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596 2023-10-17T01:30:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596_vm 2023-10-17T01:30:21.573 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3597 2023-10-17T01:30:21.573 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598 2023-10-17T01:30:21.574 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598_vm 2023-10-17T01:30:21.574 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3599 2023-10-17T01:30:21.574 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600 2023-10-17T01:30:21.574 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3600_fsm 2023-10-17T01:30:21.574 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600_vm 2023-10-17T01:30:21.575 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601 2023-10-17T01:30:21.575 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3601_fsm 2023-10-17T01:30:21.575 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601_vm 2023-10-17T01:30:21.575 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602 2023-10-17T01:30:21.575 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3602_fsm 2023-10-17T01:30:21.576 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602_vm 2023-10-17T01:30:21.576 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3603 2023-10-17T01:30:21.576 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3603_fsm 2023-10-17T01:30:21.576 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3603_vm 2023-10-17T01:30:21.580 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3604 2023-10-17T01:30:21.580 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3605 2023-10-17T01:30:21.580 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3606 2023-10-17T01:30:21.580 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3607 2023-10-17T01:30:21.581 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3608 2023-10-17T01:30:21.581 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 3609 2023-10-17T01:30:21.581 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3712 2023-10-17T01:30:21.581 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764 2023-10-17T01:30:21.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3764_fsm 2023-10-17T01:30:21.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764_vm 2023-10-17T01:30:21.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3766 2023-10-17T01:30:21.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3767 2023-10-17T01:30:21.582 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3997 2023-10-17T01:30:21.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 5002 2023-10-17T01:30:21.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 548 2023-10-17T01:30:21.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 549 2023-10-17T01:30:21.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102 2023-10-17T01:30:21.583 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102_vm 2023-10-17T01:30:21.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104 2023-10-17T01:30:21.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104_vm 2023-10-17T01:30:21.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106 2023-10-17T01:30:21.584 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106_vm 2023-10-17T01:30:21.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6110 2023-10-17T01:30:21.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6111 2023-10-17T01:30:21.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6112 2023-10-17T01:30:21.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6113 2023-10-17T01:30:21.585 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6117 2023-10-17T01:30:21.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826 2023-10-17T01:30:21.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826_vm 2023-10-17T01:30:21.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 827 2023-10-17T01:30:21.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 828 2023-10-17T01:30:21.586 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:30:21.587 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:30:21.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.587 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-17T01:30:21.587 INFO:teuthology.orchestra.run.smithi045.stdout:total 7185 2023-10-17T01:30:21.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 17 01:14 . 2023-10-17T01:30:21.588 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 17 01:15 .. 2023-10-17T01:30:21.588 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 112 2023-10-17T01:30:21.589 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 113 2023-10-17T01:30:21.589 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 17 01:14 12182 2023-10-17T01:30:21.589 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12182_fsm 2023-10-17T01:30:21.589 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12182_vm 2023-10-17T01:30:21.590 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12184 2023-10-17T01:30:21.590 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12186 2023-10-17T01:30:21.590 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187 2023-10-17T01:30:21.590 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12187_fsm 2023-10-17T01:30:21.591 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187_vm 2023-10-17T01:30:21.591 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12189 2023-10-17T01:30:21.591 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12191 2023-10-17T01:30:21.592 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192 2023-10-17T01:30:21.592 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12192_fsm 2023-10-17T01:30:21.592 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192_vm 2023-10-17T01:30:21.593 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12194 2023-10-17T01:30:21.593 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12196 2023-10-17T01:30:21.593 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197 2023-10-17T01:30:21.593 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12197_fsm 2023-10-17T01:30:21.594 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197_vm 2023-10-17T01:30:21.594 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12199 2023-10-17T01:30:21.594 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12201 2023-10-17T01:30:21.594 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202 2023-10-17T01:30:21.595 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12202_fsm 2023-10-17T01:30:21.595 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202_vm 2023-10-17T01:30:21.595 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12204 2023-10-17T01:30:21.595 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12206 2023-10-17T01:30:21.595 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207 2023-10-17T01:30:21.596 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12207_fsm 2023-10-17T01:30:21.596 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207_vm 2023-10-17T01:30:21.596 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12209 2023-10-17T01:30:21.596 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12211 2023-10-17T01:30:21.596 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12212 2023-10-17T01:30:21.597 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12214 2023-10-17T01:30:21.597 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12216 2023-10-17T01:30:21.597 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 17 01:14 1247 2023-10-17T01:30:21.597 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1247_fsm 2023-10-17T01:30:21.598 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1247_vm 2023-10-17T01:30:21.598 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 17 01:14 1249 2023-10-17T01:30:21.598 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1249_fsm 2023-10-17T01:30:21.598 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1249_vm 2023-10-17T01:30:21.599 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 17 01:14 1255 2023-10-17T01:30:21.599 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1255_fsm 2023-10-17T01:30:21.599 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1255_vm 2023-10-17T01:30:21.599 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:14 1259 2023-10-17T01:30:21.599 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1259_fsm 2023-10-17T01:30:21.600 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1259_vm 2023-10-17T01:30:21.600 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417 2023-10-17T01:30:21.600 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417_vm 2023-10-17T01:30:21.600 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418 2023-10-17T01:30:21.600 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418_vm 2023-10-17T01:30:21.601 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 174 2023-10-17T01:30:21.601 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 175 2023-10-17T01:30:21.601 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2187 2023-10-17T01:30:21.601 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224 2023-10-17T01:30:21.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224_vm 2023-10-17T01:30:21.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328 2023-10-17T01:30:21.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328_vm 2023-10-17T01:30:21.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336 2023-10-17T01:30:21.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336_vm 2023-10-17T01:30:21.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2337 2023-10-17T01:30:21.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2600 2023-10-17T01:30:21.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2600_fsm 2023-10-17T01:30:21.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2600_vm 2023-10-17T01:30:21.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601 2023-10-17T01:30:21.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2601_fsm 2023-10-17T01:30:21.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601_vm 2023-10-17T01:30:21.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2602 2023-10-17T01:30:21.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2602_fsm 2023-10-17T01:30:21.605 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2602_vm 2023-10-17T01:30:21.605 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2603 2023-10-17T01:30:21.605 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2603_fsm 2023-10-17T01:30:21.606 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2603_vm 2023-10-17T01:30:21.606 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604 2023-10-17T01:30:21.606 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604_vm 2023-10-17T01:30:21.606 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2605 2023-10-17T01:30:21.607 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2605_fsm 2023-10-17T01:30:21.607 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2605_vm 2023-10-17T01:30:21.607 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606 2023-10-17T01:30:21.607 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2606_fsm 2023-10-17T01:30:21.607 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2606_vm 2023-10-17T01:30:21.608 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607 2023-10-17T01:30:21.608 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607_fsm 2023-10-17T01:30:21.608 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2607_vm 2023-10-17T01:30:21.608 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 17 01:14 2608 2023-10-17T01:30:21.609 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2608_fsm 2023-10-17T01:30:21.609 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2608_vm 2023-10-17T01:30:21.609 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 17 01:14 2609 2023-10-17T01:30:21.609 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2609_fsm 2023-10-17T01:30:21.609 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2609_vm 2023-10-17T01:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610 2023-10-17T01:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2610_fsm 2023-10-17T01:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2610_vm 2023-10-17T01:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611 2023-10-17T01:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611_vm 2023-10-17T01:30:21.611 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612 2023-10-17T01:30:21.611 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2612_fsm 2023-10-17T01:30:21.611 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612_vm 2023-10-17T01:30:21.611 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613 2023-10-17T01:30:21.611 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613_vm 2023-10-17T01:30:21.612 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615 2023-10-17T01:30:21.612 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2615_fsm 2023-10-17T01:30:21.612 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615_vm 2023-10-17T01:30:21.612 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616 2023-10-17T01:30:21.612 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616_fsm 2023-10-17T01:30:21.613 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2616_vm 2023-10-17T01:30:21.613 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 17 01:14 2617 2023-10-17T01:30:21.613 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2617_fsm 2023-10-17T01:30:21.613 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2617_vm 2023-10-17T01:30:21.613 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 17 01:14 2618 2023-10-17T01:30:21.614 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2618_fsm 2023-10-17T01:30:21.614 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2618_vm 2023-10-17T01:30:21.614 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 17 01:14 2619 2023-10-17T01:30:21.614 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2619_fsm 2023-10-17T01:30:21.614 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2619_vm 2023-10-17T01:30:21.615 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620 2023-10-17T01:30:21.615 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620_vm 2023-10-17T01:30:21.615 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2650 2023-10-17T01:30:21.615 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2651 2023-10-17T01:30:21.615 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2652 2023-10-17T01:30:21.616 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2653 2023-10-17T01:30:21.616 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2654 2023-10-17T01:30:21.616 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2655 2023-10-17T01:30:21.616 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2656 2023-10-17T01:30:21.617 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2657 2023-10-17T01:30:21.617 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 17 01:14 2658 2023-10-17T01:30:21.617 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2659 2023-10-17T01:30:21.617 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2660 2023-10-17T01:30:21.617 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2661 2023-10-17T01:30:21.618 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2662 2023-10-17T01:30:21.618 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2663 2023-10-17T01:30:21.618 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2664 2023-10-17T01:30:21.618 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2665 2023-10-17T01:30:21.619 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2666 2023-10-17T01:30:21.619 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2667 2023-10-17T01:30:21.619 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2668 2023-10-17T01:30:21.619 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2669 2023-10-17T01:30:21.619 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2670 2023-10-17T01:30:21.620 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 17 01:14 2673 2023-10-17T01:30:21.620 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 17 01:14 2674 2023-10-17T01:30:21.620 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 17 01:14 2675 2023-10-17T01:30:21.620 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2678 2023-10-17T01:30:21.620 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2679 2023-10-17T01:30:21.621 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2680 2023-10-17T01:30:21.621 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2681 2023-10-17T01:30:21.621 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2682 2023-10-17T01:30:21.621 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2683 2023-10-17T01:30:21.621 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2684 2023-10-17T01:30:21.621 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2685 2023-10-17T01:30:21.622 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2686 2023-10-17T01:30:21.622 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2687 2023-10-17T01:30:21.622 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2688 2023-10-17T01:30:21.622 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2689 2023-10-17T01:30:21.623 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2690 2023-10-17T01:30:21.623 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 17 01:14 2691 2023-10-17T01:30:21.623 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2692 2023-10-17T01:30:21.623 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2693 2023-10-17T01:30:21.623 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2696 2023-10-17T01:30:21.624 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2699 2023-10-17T01:30:21.624 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2701 2023-10-17T01:30:21.624 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2702 2023-10-17T01:30:21.624 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2703 2023-10-17T01:30:21.624 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2704 2023-10-17T01:30:21.624 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2753 2023-10-17T01:30:21.625 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2753_fsm 2023-10-17T01:30:21.625 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2753_vm 2023-10-17T01:30:21.625 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2754 2023-10-17T01:30:21.625 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2755 2023-10-17T01:30:21.625 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2756 2023-10-17T01:30:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2757 2023-10-17T01:30:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830 2023-10-17T01:30:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830_vm 2023-10-17T01:30:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2831 2023-10-17T01:30:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832 2023-10-17T01:30:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832_vm 2023-10-17T01:30:21.627 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2833 2023-10-17T01:30:21.627 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834 2023-10-17T01:30:21.627 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834_vm 2023-10-17T01:30:21.627 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2835 2023-10-17T01:30:21.628 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836 2023-10-17T01:30:21.628 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836_vm 2023-10-17T01:30:21.628 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2837 2023-10-17T01:30:21.628 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 17 01:14 2838 2023-10-17T01:30:21.628 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2838_fsm 2023-10-17T01:30:21.629 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2838_vm 2023-10-17T01:30:21.629 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2839 2023-10-17T01:30:21.629 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840 2023-10-17T01:30:21.629 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840_fsm 2023-10-17T01:30:21.629 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2840_vm 2023-10-17T01:30:21.630 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2841 2023-10-17T01:30:21.630 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995 2023-10-17T01:30:21.630 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995_vm 2023-10-17T01:30:21.630 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2996 2023-10-17T01:30:21.630 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079 2023-10-17T01:30:21.631 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3079_fsm 2023-10-17T01:30:21.631 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079_vm 2023-10-17T01:30:21.631 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3080 2023-10-17T01:30:21.631 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3081 2023-10-17T01:30:21.631 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3085 2023-10-17T01:30:21.632 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118 2023-10-17T01:30:21.632 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118_vm 2023-10-17T01:30:21.632 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3119 2023-10-17T01:30:21.632 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3164 2023-10-17T01:30:21.633 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256 2023-10-17T01:30:21.633 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256_vm 2023-10-17T01:30:21.633 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3257 2023-10-17T01:30:21.633 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3258 2023-10-17T01:30:21.633 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350 2023-10-17T01:30:21.634 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350_vm 2023-10-17T01:30:21.634 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3351 2023-10-17T01:30:21.634 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3379 2023-10-17T01:30:21.634 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3380 2023-10-17T01:30:21.634 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381 2023-10-17T01:30:21.635 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381_vm 2023-10-17T01:30:21.635 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3394 2023-10-17T01:30:21.635 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3394_fsm 2023-10-17T01:30:21.635 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3394_vm 2023-10-17T01:30:21.635 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3395 2023-10-17T01:30:21.636 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439 2023-10-17T01:30:21.636 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439_vm 2023-10-17T01:30:21.636 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3440 2023-10-17T01:30:21.636 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 3455 2023-10-17T01:30:21.636 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3456 2023-10-17T01:30:21.636 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3456_fsm 2023-10-17T01:30:21.637 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3456_vm 2023-10-17T01:30:21.637 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466 2023-10-17T01:30:21.637 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466_vm 2023-10-17T01:30:21.637 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3467 2023-10-17T01:30:21.638 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3468 2023-10-17T01:30:21.638 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501 2023-10-17T01:30:21.638 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501_vm 2023-10-17T01:30:21.638 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3502 2023-10-17T01:30:21.638 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3503 2023-10-17T01:30:21.639 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3534 2023-10-17T01:30:21.639 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541 2023-10-17T01:30:21.639 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3541_fsm 2023-10-17T01:30:21.640 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541_vm 2023-10-17T01:30:21.640 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3542 2023-10-17T01:30:21.640 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3574 2023-10-17T01:30:21.640 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3575 2023-10-17T01:30:21.640 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576 2023-10-17T01:30:21.640 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576_vm 2023-10-17T01:30:21.641 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596 2023-10-17T01:30:21.641 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596_vm 2023-10-17T01:30:21.641 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3597 2023-10-17T01:30:21.641 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598 2023-10-17T01:30:21.641 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598_vm 2023-10-17T01:30:21.642 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3599 2023-10-17T01:30:21.642 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600 2023-10-17T01:30:21.642 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3600_fsm 2023-10-17T01:30:21.642 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600_vm 2023-10-17T01:30:21.642 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601 2023-10-17T01:30:21.643 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3601_fsm 2023-10-17T01:30:21.643 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601_vm 2023-10-17T01:30:21.643 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602 2023-10-17T01:30:21.643 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3602_fsm 2023-10-17T01:30:21.644 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602_vm 2023-10-17T01:30:21.644 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3603 2023-10-17T01:30:21.644 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3603_fsm 2023-10-17T01:30:21.644 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3603_vm 2023-10-17T01:30:21.644 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3604 2023-10-17T01:30:21.645 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3605 2023-10-17T01:30:21.645 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3606 2023-10-17T01:30:21.645 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3607 2023-10-17T01:30:21.645 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3608 2023-10-17T01:30:21.645 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 3609 2023-10-17T01:30:21.645 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3712 2023-10-17T01:30:21.646 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764 2023-10-17T01:30:21.646 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3764_fsm 2023-10-17T01:30:21.646 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764_vm 2023-10-17T01:30:21.646 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3766 2023-10-17T01:30:21.646 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3767 2023-10-17T01:30:21.647 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3997 2023-10-17T01:30:21.647 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 5002 2023-10-17T01:30:21.647 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 548 2023-10-17T01:30:21.647 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 549 2023-10-17T01:30:21.647 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102 2023-10-17T01:30:21.648 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102_vm 2023-10-17T01:30:21.648 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104 2023-10-17T01:30:21.648 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104_vm 2023-10-17T01:30:21.648 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106 2023-10-17T01:30:21.649 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106_vm 2023-10-17T01:30:21.649 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6110 2023-10-17T01:30:21.649 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6111 2023-10-17T01:30:21.649 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6112 2023-10-17T01:30:21.649 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6113 2023-10-17T01:30:21.650 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6117 2023-10-17T01:30:21.650 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826 2023-10-17T01:30:21.650 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826_vm 2023-10-17T01:30:21.650 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 827 2023-10-17T01:30:21.650 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 828 2023-10-17T01:30:21.651 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:30:21.651 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:30:21.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.651 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-17T01:30:21.651 INFO:teuthology.orchestra.run.smithi045.stdout:total 624032 2023-10-17T01:30:21.651 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 17 01:15 . 2023-10-17T01:30:21.652 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 17 01:15 .. 2023-10-17T01:30:21.652 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 112 2023-10-17T01:30:21.652 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 113 2023-10-17T01:30:21.652 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 17 01:14 12182 2023-10-17T01:30:21.652 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12182_fsm 2023-10-17T01:30:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12182_vm 2023-10-17T01:30:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12184 2023-10-17T01:30:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12186 2023-10-17T01:30:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187 2023-10-17T01:30:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12187_fsm 2023-10-17T01:30:21.654 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12187_vm 2023-10-17T01:30:21.654 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12189 2023-10-17T01:30:21.654 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12191 2023-10-17T01:30:21.654 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192 2023-10-17T01:30:21.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12192_fsm 2023-10-17T01:30:21.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12192_vm 2023-10-17T01:30:21.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12194 2023-10-17T01:30:21.655 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12196 2023-10-17T01:30:21.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197 2023-10-17T01:30:21.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12197_fsm 2023-10-17T01:30:21.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12197_vm 2023-10-17T01:30:21.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12199 2023-10-17T01:30:21.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12201 2023-10-17T01:30:21.656 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202 2023-10-17T01:30:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12202_fsm 2023-10-17T01:30:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12202_vm 2023-10-17T01:30:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12204 2023-10-17T01:30:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12206 2023-10-17T01:30:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207 2023-10-17T01:30:21.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 12207_fsm 2023-10-17T01:30:21.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12207_vm 2023-10-17T01:30:21.658 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12209 2023-10-17T01:30:21.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12211 2023-10-17T01:30:21.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12212 2023-10-17T01:30:21.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 12214 2023-10-17T01:30:21.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 12216 2023-10-17T01:30:21.659 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 17 01:15 1247 2023-10-17T01:30:21.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:15 1247_fsm 2023-10-17T01:30:21.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 1247_vm 2023-10-17T01:30:21.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 17 01:15 1249 2023-10-17T01:30:21.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:15 1249_fsm 2023-10-17T01:30:21.660 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 1249_vm 2023-10-17T01:30:21.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 17 01:14 1255 2023-10-17T01:30:21.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1255_fsm 2023-10-17T01:30:21.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1255_vm 2023-10-17T01:30:21.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:30 1259 2023-10-17T01:30:21.661 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1259_fsm 2023-10-17T01:30:21.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 1259_vm 2023-10-17T01:30:21.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417 2023-10-17T01:30:21.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1417_vm 2023-10-17T01:30:21.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418 2023-10-17T01:30:21.662 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 1418_vm 2023-10-17T01:30:21.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10280960 Oct 17 01:30 16387 2023-10-17T01:30:21.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:30 16387_fsm 2023-10-17T01:30:21.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:30 16387_vm 2023-10-17T01:30:21.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2752512 Oct 17 01:30 16393 2023-10-17T01:30:21.663 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:30 16393_fsm 2023-10-17T01:30:21.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:30 16393_vm 2023-10-17T01:30:21.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 460677120 Oct 17 01:30 16396 2023-10-17T01:30:21.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 17 01:29 16396_fsm 2023-10-17T01:30:21.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:18 16396_vm 2023-10-17T01:30:21.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 17 01:30 16397 2023-10-17T01:30:21.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 458752 Oct 17 01:30 16399 2023-10-17T01:30:21.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Oct 17 01:30 16401 2023-10-17T01:30:21.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 85032960 Oct 17 01:30 16403 2023-10-17T01:30:21.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:30 16403_fsm 2023-10-17T01:30:21.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 174 2023-10-17T01:30:21.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 175 2023-10-17T01:30:21.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2187 2023-10-17T01:30:21.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224 2023-10-17T01:30:21.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2224_vm 2023-10-17T01:30:21.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328 2023-10-17T01:30:21.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2328_vm 2023-10-17T01:30:21.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336 2023-10-17T01:30:21.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2336_vm 2023-10-17T01:30:21.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2337 2023-10-17T01:30:21.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2600 2023-10-17T01:30:21.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2600_fsm 2023-10-17T01:30:21.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2600_vm 2023-10-17T01:30:21.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601 2023-10-17T01:30:21.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2601_fsm 2023-10-17T01:30:21.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2601_vm 2023-10-17T01:30:21.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2602 2023-10-17T01:30:21.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2602_fsm 2023-10-17T01:30:21.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2602_vm 2023-10-17T01:30:21.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2603 2023-10-17T01:30:21.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2603_fsm 2023-10-17T01:30:21.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2603_vm 2023-10-17T01:30:21.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604 2023-10-17T01:30:21.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2604_vm 2023-10-17T01:30:21.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2605 2023-10-17T01:30:21.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2605_fsm 2023-10-17T01:30:21.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2605_vm 2023-10-17T01:30:21.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 2606 2023-10-17T01:30:21.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2606_fsm 2023-10-17T01:30:21.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 2606_vm 2023-10-17T01:30:21.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607 2023-10-17T01:30:21.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2607_fsm 2023-10-17T01:30:21.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2607_vm 2023-10-17T01:30:21.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Oct 17 01:15 2608 2023-10-17T01:30:21.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:15 2608_fsm 2023-10-17T01:30:21.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 2608_vm 2023-10-17T01:30:21.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 17 01:14 2609 2023-10-17T01:30:21.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2609_fsm 2023-10-17T01:30:21.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2609_vm 2023-10-17T01:30:21.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:15 2610 2023-10-17T01:30:21.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:15 2610_fsm 2023-10-17T01:30:21.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 2610_vm 2023-10-17T01:30:21.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611 2023-10-17T01:30:21.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2611_vm 2023-10-17T01:30:21.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612 2023-10-17T01:30:21.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2612_fsm 2023-10-17T01:30:21.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2612_vm 2023-10-17T01:30:21.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613 2023-10-17T01:30:21.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2613_vm 2023-10-17T01:30:21.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615 2023-10-17T01:30:21.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2615_fsm 2023-10-17T01:30:21.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2615_vm 2023-10-17T01:30:21.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616 2023-10-17T01:30:21.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2616_fsm 2023-10-17T01:30:21.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2616_vm 2023-10-17T01:30:21.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 17 01:14 2617 2023-10-17T01:30:21.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2617_fsm 2023-10-17T01:30:21.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2617_vm 2023-10-17T01:30:21.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 17 01:14 2618 2023-10-17T01:30:21.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2618_fsm 2023-10-17T01:30:21.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2618_vm 2023-10-17T01:30:21.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Oct 17 01:30 2619 2023-10-17T01:30:21.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:19 2619_fsm 2023-10-17T01:30:21.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 2619_vm 2023-10-17T01:30:21.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620 2023-10-17T01:30:21.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2620_vm 2023-10-17T01:30:21.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2650 2023-10-17T01:30:21.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2651 2023-10-17T01:30:21.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2652 2023-10-17T01:30:21.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2653 2023-10-17T01:30:21.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2654 2023-10-17T01:30:21.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2655 2023-10-17T01:30:21.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2656 2023-10-17T01:30:21.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2657 2023-10-17T01:30:21.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Oct 17 01:15 2658 2023-10-17T01:30:21.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:15 2659 2023-10-17T01:30:21.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2660 2023-10-17T01:30:21.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2661 2023-10-17T01:30:21.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:15 2662 2023-10-17T01:30:21.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:15 2663 2023-10-17T01:30:21.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2664 2023-10-17T01:30:21.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2665 2023-10-17T01:30:21.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2666 2023-10-17T01:30:21.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2667 2023-10-17T01:30:21.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2668 2023-10-17T01:30:21.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2669 2023-10-17T01:30:21.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2670 2023-10-17T01:30:21.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 17 01:15 2673 2023-10-17T01:30:21.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 17 01:15 2674 2023-10-17T01:30:21.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 17 01:14 2675 2023-10-17T01:30:21.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2678 2023-10-17T01:30:21.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2679 2023-10-17T01:30:21.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2680 2023-10-17T01:30:21.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2681 2023-10-17T01:30:21.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2682 2023-10-17T01:30:21.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2683 2023-10-17T01:30:21.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2684 2023-10-17T01:30:21.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2685 2023-10-17T01:30:21.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2686 2023-10-17T01:30:21.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2687 2023-10-17T01:30:21.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:14 2688 2023-10-17T01:30:21.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 17 01:14 2689 2023-10-17T01:30:21.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 17 01:14 2690 2023-10-17T01:30:21.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 17 01:14 2691 2023-10-17T01:30:21.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2692 2023-10-17T01:30:21.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2693 2023-10-17T01:30:21.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:19 2696 2023-10-17T01:30:21.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2699 2023-10-17T01:30:21.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2701 2023-10-17T01:30:21.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2702 2023-10-17T01:30:21.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:15 2703 2023-10-17T01:30:21.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:15 2704 2023-10-17T01:30:21.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2753 2023-10-17T01:30:21.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2753_fsm 2023-10-17T01:30:21.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2753_vm 2023-10-17T01:30:21.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2754 2023-10-17T01:30:21.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2755 2023-10-17T01:30:21.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2756 2023-10-17T01:30:21.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 2757 2023-10-17T01:30:21.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830 2023-10-17T01:30:21.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2830_vm 2023-10-17T01:30:21.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2831 2023-10-17T01:30:21.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832 2023-10-17T01:30:21.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2832_vm 2023-10-17T01:30:21.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2833 2023-10-17T01:30:21.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834 2023-10-17T01:30:21.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2834_vm 2023-10-17T01:30:21.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2835 2023-10-17T01:30:21.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836 2023-10-17T01:30:21.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2836_vm 2023-10-17T01:30:21.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2837 2023-10-17T01:30:21.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 17 01:14 2838 2023-10-17T01:30:21.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2838_fsm 2023-10-17T01:30:21.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2838_vm 2023-10-17T01:30:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2839 2023-10-17T01:30:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840 2023-10-17T01:30:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2840_fsm 2023-10-17T01:30:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2840_vm 2023-10-17T01:30:21.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2841 2023-10-17T01:30:21.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995 2023-10-17T01:30:21.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2995_vm 2023-10-17T01:30:21.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2996 2023-10-17T01:30:21.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079 2023-10-17T01:30:21.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3079_fsm 2023-10-17T01:30:21.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3079_vm 2023-10-17T01:30:21.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3080 2023-10-17T01:30:21.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3081 2023-10-17T01:30:21.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3085 2023-10-17T01:30:21.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118 2023-10-17T01:30:21.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3118_vm 2023-10-17T01:30:21.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3119 2023-10-17T01:30:21.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3164 2023-10-17T01:30:21.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256 2023-10-17T01:30:21.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3256_vm 2023-10-17T01:30:21.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3257 2023-10-17T01:30:21.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3258 2023-10-17T01:30:21.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350 2023-10-17T01:30:21.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3350_vm 2023-10-17T01:30:21.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3351 2023-10-17T01:30:21.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3379 2023-10-17T01:30:21.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3380 2023-10-17T01:30:21.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381 2023-10-17T01:30:21.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3381_vm 2023-10-17T01:30:21.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3394 2023-10-17T01:30:21.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3394_fsm 2023-10-17T01:30:21.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3394_vm 2023-10-17T01:30:21.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3395 2023-10-17T01:30:21.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439 2023-10-17T01:30:21.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3439_vm 2023-10-17T01:30:21.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3440 2023-10-17T01:30:21.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 17 01:15 3455 2023-10-17T01:30:21.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3456 2023-10-17T01:30:21.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3456_fsm 2023-10-17T01:30:21.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3456_vm 2023-10-17T01:30:21.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466 2023-10-17T01:30:21.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3466_vm 2023-10-17T01:30:21.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3467 2023-10-17T01:30:21.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3468 2023-10-17T01:30:21.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501 2023-10-17T01:30:21.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3501_vm 2023-10-17T01:30:21.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3502 2023-10-17T01:30:21.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3503 2023-10-17T01:30:21.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3534 2023-10-17T01:30:21.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541 2023-10-17T01:30:21.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3541_fsm 2023-10-17T01:30:21.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3541_vm 2023-10-17T01:30:21.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3542 2023-10-17T01:30:21.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3574 2023-10-17T01:30:21.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3575 2023-10-17T01:30:21.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576 2023-10-17T01:30:21.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3576_vm 2023-10-17T01:30:21.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596 2023-10-17T01:30:21.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3596_vm 2023-10-17T01:30:21.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3597 2023-10-17T01:30:21.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598 2023-10-17T01:30:21.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3598_vm 2023-10-17T01:30:21.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3599 2023-10-17T01:30:21.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600 2023-10-17T01:30:21.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3600_fsm 2023-10-17T01:30:21.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3600_vm 2023-10-17T01:30:21.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601 2023-10-17T01:30:21.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3601_fsm 2023-10-17T01:30:21.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3601_vm 2023-10-17T01:30:21.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602 2023-10-17T01:30:21.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3602_fsm 2023-10-17T01:30:21.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3602_vm 2023-10-17T01:30:21.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3603 2023-10-17T01:30:21.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3603_fsm 2023-10-17T01:30:21.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3603_vm 2023-10-17T01:30:21.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3604 2023-10-17T01:30:21.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3605 2023-10-17T01:30:21.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3606 2023-10-17T01:30:21.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3607 2023-10-17T01:30:21.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3608 2023-10-17T01:30:21.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 17 01:14 3609 2023-10-17T01:30:21.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3712 2023-10-17T01:30:21.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764 2023-10-17T01:30:21.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 3764_fsm 2023-10-17T01:30:21.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3764_vm 2023-10-17T01:30:21.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3766 2023-10-17T01:30:21.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 3767 2023-10-17T01:30:21.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3997 2023-10-17T01:30:21.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 5002 2023-10-17T01:30:21.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 548 2023-10-17T01:30:21.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 549 2023-10-17T01:30:21.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102 2023-10-17T01:30:21.716 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6102_vm 2023-10-17T01:30:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104 2023-10-17T01:30:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6104_vm 2023-10-17T01:30:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106 2023-10-17T01:30:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6106_vm 2023-10-17T01:30:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6110 2023-10-17T01:30:21.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6111 2023-10-17T01:30:21.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6112 2023-10-17T01:30:21.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6113 2023-10-17T01:30:21.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6117 2023-10-17T01:30:21.718 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826 2023-10-17T01:30:21.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 826_vm 2023-10-17T01:30:21.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 827 2023-10-17T01:30:21.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 828 2023-10-17T01:30:21.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:30:21.719 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Oct 17 01:14 pg_internal.init 2023-10-17T01:30:21.720 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:14 PG_VERSION 2023-10-17T01:30:21.720 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.720 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-10-17T01:30:21.720 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.720 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:15 . 2023-10-17T01:30:21.721 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 17 01:15 .. 2023-10-17T01:30:21.721 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.721 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/global: 2023-10-17T01:30:21.721 INFO:teuthology.orchestra.run.smithi045.stdout:total 574 2023-10-17T01:30:21.721 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 17 01:14 . 2023-10-17T01:30:21.721 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1136 2023-10-17T01:30:21.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1136_fsm 2023-10-17T01:30:21.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1136_vm 2023-10-17T01:30:21.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 1137 2023-10-17T01:30:21.722 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1213 2023-10-17T01:30:21.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1213_fsm 2023-10-17T01:30:21.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1213_vm 2023-10-17T01:30:21.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1214 2023-10-17T01:30:21.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1214_fsm 2023-10-17T01:30:21.723 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1214_vm 2023-10-17T01:30:21.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 1232 2023-10-17T01:30:21.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 1233 2023-10-17T01:30:21.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1260 2023-10-17T01:30:21.724 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1260_fsm 2023-10-17T01:30:21.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1260_vm 2023-10-17T01:30:21.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1261 2023-10-17T01:30:21.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1261_fsm 2023-10-17T01:30:21.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1261_vm 2023-10-17T01:30:21.725 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1262 2023-10-17T01:30:21.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 1262_fsm 2023-10-17T01:30:21.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 1262_vm 2023-10-17T01:30:21.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2396 2023-10-17T01:30:21.726 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 17 01:14 2396_fsm 2023-10-17T01:30:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2396_vm 2023-10-17T01:30:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2397 2023-10-17T01:30:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2671 2023-10-17T01:30:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2672 2023-10-17T01:30:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2676 2023-10-17T01:30:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2677 2023-10-17T01:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2694 2023-10-17T01:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2695 2023-10-17T01:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2697 2023-10-17T01:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 17 01:14 2698 2023-10-17T01:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2846 2023-10-17T01:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2846_vm 2023-10-17T01:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2847 2023-10-17T01:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2964 2023-10-17T01:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2964_vm 2023-10-17T01:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2965 2023-10-17T01:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2966 2023-10-17T01:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 2966_vm 2023-10-17T01:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 2967 2023-10-17T01:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3592 2023-10-17T01:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 3592_vm 2023-10-17T01:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 3593 2023-10-17T01:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 4060 2023-10-17T01:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 4060_vm 2023-10-17T01:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 4061 2023-10-17T01:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6000 2023-10-17T01:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6000_vm 2023-10-17T01:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6001 2023-10-17T01:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6002 2023-10-17T01:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6100 2023-10-17T01:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 6100_vm 2023-10-17T01:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6114 2023-10-17T01:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 6115 2023-10-17T01:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:30 pg_control 2023-10-17T01:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 17 01:14 pg_filenode.map 2023-10-17T01:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Oct 17 01:14 pg_internal.init 2023-10-17T01:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/log: 2023-10-17T01:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout:total 2 2023-10-17T01:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1332 Oct 17 01:30 postgresql-Tue.log 2023-10-17T01:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-17T01:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-17T01:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-17T01:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout:total 1 2023-10-17T01:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:30 . 2023-10-17T01:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 mappings 2023-10-17T01:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 17 01:30 replorigin_checkpoint 2023-10-17T01:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 snapshots 2023-10-17T01:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-17T01:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:30 .. 2023-10-17T01:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-17T01:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:30 .. 2023-10-17T01:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-17T01:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 . 2023-10-17T01:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 members 2023-10-17T01:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 offsets 2023-10-17T01:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-17T01:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 .. 2023-10-17T01:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-17T01:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:14 .. 2023-10-17T01:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:15 0000 2023-10-17T01:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-17T01:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout:total 8 2023-10-17T01:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:14 . 2023-10-17T01:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 17 01:14 0000 2023-10-17T01:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-17T01:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-17T01:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-17T01:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-17T01:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout:total 13 2023-10-17T01:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 17 01:30 . 2023-10-17T01:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Oct 17 01:30 db_0.stat 2023-10-17T01:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Oct 17 01:30 db_12347.stat 2023-10-17T01:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Oct 17 01:30 global.stat 2023-10-17T01:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-17T01:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:30 . 2023-10-17T01:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-17T01:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout:total 208 2023-10-17T01:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 17 01:30 . 2023-10-17T01:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Oct 17 01:30 0018 2023-10-17T01:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-17T01:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-17T01:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-17T01:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout:total 1032192 2023-10-17T01:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 17 01:30 . 2023-10-17T01:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 0000000100000001000000DC 2023-10-17T01:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 0000000100000001000000DD 2023-10-17T01:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 0000000100000001000000DE 2023-10-17T01:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 0000000100000001000000DF 2023-10-17T01:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 0000000100000001000000E0 2023-10-17T01:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E1 2023-10-17T01:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E2 2023-10-17T01:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E3 2023-10-17T01:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E4 2023-10-17T01:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E5 2023-10-17T01:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E6 2023-10-17T01:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:30 0000000100000001000000E7 2023-10-17T01:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:26 0000000100000001000000E8 2023-10-17T01:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000E9 2023-10-17T01:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000EA 2023-10-17T01:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000EB 2023-10-17T01:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 0000000100000001000000EC 2023-10-17T01:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 0000000100000001000000ED 2023-10-17T01:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000EE 2023-10-17T01:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000EF 2023-10-17T01:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F0 2023-10-17T01:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F1 2023-10-17T01:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F2 2023-10-17T01:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F3 2023-10-17T01:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F4 2023-10-17T01:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 0000000100000001000000F5 2023-10-17T01:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F6 2023-10-17T01:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F7 2023-10-17T01:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F8 2023-10-17T01:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000F9 2023-10-17T01:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 0000000100000001000000FA 2023-10-17T01:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 0000000100000001000000FB 2023-10-17T01:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000FC 2023-10-17T01:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000FD 2023-10-17T01:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000FE 2023-10-17T01:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 0000000100000001000000FF 2023-10-17T01:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000000 2023-10-17T01:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:27 000000010000000200000001 2023-10-17T01:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000002 2023-10-17T01:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000003 2023-10-17T01:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000004 2023-10-17T01:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000005 2023-10-17T01:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000006 2023-10-17T01:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000007 2023-10-17T01:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000008 2023-10-17T01:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000009 2023-10-17T01:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 00000001000000020000000A 2023-10-17T01:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 00000001000000020000000B 2023-10-17T01:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 00000001000000020000000C 2023-10-17T01:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 00000001000000020000000D 2023-10-17T01:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 00000001000000020000000E 2023-10-17T01:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 00000001000000020000000F 2023-10-17T01:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000010 2023-10-17T01:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000011 2023-10-17T01:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000012 2023-10-17T01:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000013 2023-10-17T01:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000014 2023-10-17T01:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000015 2023-10-17T01:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000016 2023-10-17T01:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 000000010000000200000017 2023-10-17T01:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000018 2023-10-17T01:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:28 000000010000000200000019 2023-10-17T01:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 17 01:29 00000001000000020000001A 2023-10-17T01:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 archive_status 2023-10-17T01:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-17T01:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:total 0 2023-10-17T01:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 17 01:14 . 2023-10-17T01:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 17 01:30 .. 2023-10-17T01:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-17T01:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:total 400 2023-10-17T01:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 17 01:23 . 2023-10-17T01:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 17 01:30 .. 2023-10-17T01:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Oct 17 01:23 0000 2023-10-17T01:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Oct 17 01:30 0001 2023-10-17T01:30:21.777 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-17T01:30:21.789 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:30:21.803 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-17T01:30:21.813 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-17T01:30:21.813 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_ab9fe43522bc41de59746200b95f65d4fbcccffe/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-10-17T01:30:21.813 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-17T01:30:21.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:30:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.059+0000 7f632b5ff700 1 -- 172.21.15.45:0/3828180607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6324101f40 msgr2=0x7f63241023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.059+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3828180607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6324101f40 0x7f63241023a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f63140099c0 tx=0x7f6314009cd0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.059+0000 7f632b5ff700 1 -- 172.21.15.45:0/3828180607 shutdown_connections 2023-10-17T01:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.059+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3828180607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63241028e0 0x7f6324109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.059+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3828180607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 0x7f6324101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.059+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3828180607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6324101f40 0x7f63241023a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.060+0000 7f632b5ff700 1 -- 172.21.15.45:0/3828180607 >> 172.21.15.45:0/3828180607 conn(0x7f63240fc280 msgr2=0x7f63240fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.060+0000 7f632b5ff700 1 -- 172.21.15.45:0/3828180607 shutdown_connections 2023-10-17T01:30:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.060+0000 7f632b5ff700 1 -- 172.21.15.45:0/3828180607 wait complete. 2023-10-17T01:30:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 Processor -- start 2023-10-17T01:30:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 -- start start 2023-10-17T01:30:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 0x7f6324198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 0x7f63241991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63241028e0 0x7f63241a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a07b0 con 0x7f6324100d50 2023-10-17T01:30:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0920 con 0x7f63241028e0 2023-10-17T01:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632b5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0ae0 con 0x7f6324101f40 2023-10-17T01:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f632939b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 0x7f6324198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f6328b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 0x7f63241991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f6328b9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 0x7f63241991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60416/0 (socket says 172.21.15.45:60416) 2023-10-17T01:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f6328b9a700 1 -- 172.21.15.45:0/3091047043 learned_addr learned my addr 172.21.15.45:0/3091047043 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.061+0000 7f6329b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63241028e0 0x7f63241a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6328b9a700 1 -- 172.21.15.45:0/3091047043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63241028e0 msgr2=0x7f63241a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6328b9a700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63241028e0 0x7f63241a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6328b9a700 1 -- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 msgr2=0x7f6324198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6328b9a700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 0x7f6324198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6328b9a700 1 -- 172.21.15.45:0/3091047043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63140095f0 con 0x7f6324101f40 2023-10-17T01:30:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6329b9c700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63241028e0 0x7f63241a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:30:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f632939b700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 0x7f6324198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:30:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f6328b9a700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 0x7f63241991b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6314009990 tx=0x7f631400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401d070 con 0x7f6324101f40 2023-10-17T01:30:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63241a0d60 con 0x7f6324101f40 2023-10-17T01:30:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6314005030 con 0x7f6324101f40 2023-10-17T01:30:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314004a70 con 0x7f6324101f40 2023-10-17T01:30:22.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.062+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63241a1190 con 0x7f6324101f40 2023-10-17T01:30:22.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.063+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632410c5b0 con 0x7f6324101f40 2023-10-17T01:30:22.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.065+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f6314005410 con 0x7f6324101f40 2023-10-17T01:30:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.067+0000 7f631a7fc700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f63100a0b70 0x7f63100a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.067+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f63140cd340 con 0x7f6324101f40 2023-10-17T01:30:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.067+0000 7f632939b700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f63100a0b70 0x7f63100a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.068+0000 7f632939b700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f63100a0b70 0x7f63100a3030 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f632419a140 tx=0x7f6320009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.070+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f6314096d30 con 0x7f6324101f40 2023-10-17T01:30:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.298+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63241a1740 con 0x7f6324101f40 2023-10-17T01:30:22.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.300+0000 7f631a7fc700 1 -- 172.21.15.45:0/3091047043 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f631409a350 con 0x7f6324101f40 2023-10-17T01:30:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:22.302 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:30:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f63100a0b70 msgr2=0x7f63100a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f63100a0b70 0x7f63100a3030 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f632419a140 tx=0x7f6320009260 comp rx=0 tx=0).stop 2023-10-17T01:30:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 msgr2=0x7f63241991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 0x7f63241991b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6314009990 tx=0x7f631400f690 comp rx=0 tx=0).stop 2023-10-17T01:30:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 shutdown_connections 2023-10-17T01:30:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f63100a0b70 0x7f63100a3030 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63241028e0 0x7f63241a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6324100d50 0x7f6324198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 --2- 172.21.15.45:0/3091047043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6324101f40 0x7f63241991b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.302+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 >> 172.21.15.45:0/3091047043 conn(0x7f63240fc280 msgr2=0x7f63240fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.303+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 shutdown_connections 2023-10-17T01:30:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.303+0000 7f632b5ff700 1 -- 172.21.15.45:0/3091047043 wait complete. 2023-10-17T01:30:22.323 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:30:22.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:30:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/177591397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb041028d0 msgr2=0x7fdb04109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/177591397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb041028d0 0x7fdb04109180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00b0d0 tx=0x7fdafc00b3e0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/177591397 shutdown_connections 2023-10-17T01:30:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/177591397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb041028d0 0x7fdb04109180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/177591397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb04100d40 0x7fdb04101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/177591397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04101f30 0x7fdb04102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/177591397 >> 172.21.15.45:0/177591397 conn(0x7fdb040fc290 msgr2=0x7fdb040fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/177591397 shutdown_connections 2023-10-17T01:30:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.534+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/177591397 wait complete. 2023-10-17T01:30:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.535+0000 7fdb0b1ff700 1 Processor -- start 2023-10-17T01:30:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.535+0000 7fdb0b1ff700 1 -- start start 2023-10-17T01:30:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0b1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 0x7fdb04198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb04101f30 0x7fdb04199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0b1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb041028d0 0x7fdb041a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0b1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041a0980 con 0x7fdb041028d0 2023-10-17T01:30:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041a0ac0 con 0x7fdb04101f30 2023-10-17T01:30:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0b1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041a0c50 con 0x7fdb04100d40 2023-10-17T01:30:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 0x7fdb04198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 0x7fdb04198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60436/0 (socket says 172.21.15.45:60436) 2023-10-17T01:30:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 -- 172.21.15.45:0/2037539280 learned_addr learned my addr 172.21.15.45:0/2037539280 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb03fff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb04101f30 0x7fdb04199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb0979c700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb041028d0 0x7fdb041a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 -- 172.21.15.45:0/2037539280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb04101f30 msgr2=0x7fdb04199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb04101f30 0x7fdb04199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 -- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb041028d0 msgr2=0x7fdb041a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb041028d0 0x7fdb041a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.536+0000 7fdb08f9b700 1 -- 172.21.15.45:0/2037539280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009d00 con 0x7fdb04100d40 2023-10-17T01:30:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.537+0000 7fdb08f9b700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 0x7fdb04198e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800e970 tx=0x7fdaf800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.537+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8009790 con 0x7fdb04100d40 2023-10-17T01:30:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.537+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb041a0f30 con 0x7fdb04100d40 2023-10-17T01:30:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.537+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaf800c7e0 con 0x7fdb04100d40 2023-10-17T01:30:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.537+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8003c70 con 0x7fdb04100d40 2023-10-17T01:30:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.537+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb041a1350 con 0x7fdb04100d40 2023-10-17T01:30:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.538+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb041933a0 con 0x7fdb04100d40 2023-10-17T01:30:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.540+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fdaf800cc50 con 0x7fdb04100d40 2023-10-17T01:30:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.542+0000 7fdb01ffb700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdaf40a0b20 0x7fdaf40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.542+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdaf8014070 con 0x7fdb04100d40 2023-10-17T01:30:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.542+0000 7fdb03fff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdaf40a0b20 0x7fdaf40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.542+0000 7fdb03fff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdaf40a0b20 0x7fdaf40a2fe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009be0 tx=0x7fdaf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.546+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fdaf8095e10 con 0x7fdb04100d40 2023-10-17T01:30:22.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:22 smithi045 ceph-mon[98474]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 2.16k op/s 2023-10-17T01:30:22.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:22 smithi060 ceph-mon[134136]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 2.16k op/s 2023-10-17T01:30:22.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:22 smithi181 ceph-mon[134520]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 2.16k op/s 2023-10-17T01:30:22.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.769+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb0410c650 con 0x7fdb04100d40 2023-10-17T01:30:22.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.769+0000 7fdb01ffb700 1 -- 172.21.15.45:0/2037539280 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdaf8003dd0 con 0x7fdb04100d40 2023-10-17T01:30:22.771 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:30:22.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdaf40a0b20 msgr2=0x7fdaf40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdaf40a0b20 0x7fdaf40a2fe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009be0 tx=0x7fdaf0009260 comp rx=0 tx=0).stop 2023-10-17T01:30:22.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 msgr2=0x7fdb04198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:22.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 0x7fdb04198e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800e970 tx=0x7fdaf800ec80 comp rx=0 tx=0).stop 2023-10-17T01:30:22.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 shutdown_connections 2023-10-17T01:30:22.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fdaf40a0b20 0x7fdaf40a2fe0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb041028d0 0x7fdb041a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb04100d40 0x7fdb04198e00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 --2- 172.21.15.45:0/2037539280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb04101f30 0x7fdb04199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:22.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 >> 172.21.15.45:0/2037539280 conn(0x7fdb040fc290 msgr2=0x7fdb04107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:22.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 shutdown_connections 2023-10-17T01:30:22.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.772+0000 7fdb0b1ff700 1 -- 172.21.15.45:0/2037539280 wait complete. 2023-10-17T01:30:22.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:30:23.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 -- 172.21.15.45:0/2742389146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c100d40 msgr2=0x7f148c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 --2- 172.21.15.45:0/2742389146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c100d40 0x7f148c101160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f14800099c0 tx=0x7f1480009cd0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 -- 172.21.15.45:0/2742389146 shutdown_connections 2023-10-17T01:30:23.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 --2- 172.21.15.45:0/2742389146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c1028d0 0x7f148c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 --2- 172.21.15.45:0/2742389146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c100d40 0x7f148c101160 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 --2- 172.21.15.45:0/2742389146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c101f30 0x7f148c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:22.999+0000 7f1492fff700 1 -- 172.21.15.45:0/2742389146 >> 172.21.15.45:0/2742389146 conn(0x7f148c0fc290 msgr2=0x7f148c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:23.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.000+0000 7f1492fff700 1 -- 172.21.15.45:0/2742389146 shutdown_connections 2023-10-17T01:30:23.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.000+0000 7f1492fff700 1 -- 172.21.15.45:0/2742389146 wait complete. 2023-10-17T01:30:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 Processor -- start 2023-10-17T01:30:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 -- start start 2023-10-17T01:30:23.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c100d40 0x7f148c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c101f30 0x7f148c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c1028d0 0x7f148c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1a0800 con 0x7f148c1028d0 2023-10-17T01:30:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1a0970 con 0x7f148c101f30 2023-10-17T01:30:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1492fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1a0b30 con 0x7f148c100d40 2023-10-17T01:30:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1490d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c100d40 0x7f148c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f148bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c101f30 0x7f148c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1490d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c100d40 0x7f148c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60466/0 (socket says 172.21.15.45:60466) 2023-10-17T01:30:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f1490d9b700 1 -- 172.21.15.45:0/1484853516 learned_addr learned my addr 172.21.15.45:0/1484853516 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.001+0000 7f149159c700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c1028d0 0x7f148c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f148bfff700 1 -- 172.21.15.45:0/1484853516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c100d40 msgr2=0x7f148c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f148bfff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c100d40 0x7f148c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f148bfff700 1 -- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c1028d0 msgr2=0x7f148c1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f148bfff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c1028d0 0x7f148c1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f148bfff700 1 -- 172.21.15.45:0/1484853516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800095f0 con 0x7f148c101f30 2023-10-17T01:30:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f149159c700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c1028d0 0x7f148c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:30:23.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f148bfff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c101f30 0x7f148c1991d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f147800e9f0 tx=0x7f147800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:23.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147800eee0 con 0x7f148c101f30 2023-10-17T01:30:23.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147800cbd0 con 0x7f148c101f30 2023-10-17T01:30:23.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478018680 con 0x7f148c101f30 2023-10-17T01:30:23.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c1a0e10 con 0x7f148c101f30 2023-10-17T01:30:23.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.002+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c1a1290 con 0x7f148c101f30 2023-10-17T01:30:23.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.003+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c193390 con 0x7f148c101f30 2023-10-17T01:30:23.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.006+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f147800cd40 con 0x7f148c101f30 2023-10-17T01:30:23.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.007+0000 7f1489ffb700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f147c0a0ba0 0x7f147c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.007+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1478014070 con 0x7f148c101f30 2023-10-17T01:30:23.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.007+0000 7f1490d9b700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f147c0a0ba0 0x7f147c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.008+0000 7f1490d9b700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f147c0a0ba0 0x7f147c0a3060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:23.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.010+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1478095ec0 con 0x7f148c101f30 2023-10-17T01:30:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.221+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f148c02ce80 con 0x7f148c101f30 2023-10-17T01:30:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.221+0000 7f1489ffb700 1 -- 172.21.15.45:0/1484853516 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14780994e0 con 0x7f148c101f30 2023-10-17T01:30:23.223 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:30:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f147c0a0ba0 msgr2=0x7f147c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f147c0a0ba0 0x7f147c0a3060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480005fb0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c101f30 msgr2=0x7f148c1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c101f30 0x7f148c1991d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f147800e9f0 tx=0x7f147800ed00 comp rx=0 tx=0).stop 2023-10-17T01:30:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 shutdown_connections 2023-10-17T01:30:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f147c0a0ba0 0x7f147c0a3060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148c1028d0 0x7f148c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f148c100d40 0x7f148c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 --2- 172.21.15.45:0/1484853516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148c101f30 0x7f148c1991d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 >> 172.21.15.45:0/1484853516 conn(0x7f148c0fc290 msgr2=0x7f148c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 shutdown_connections 2023-10-17T01:30:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.224+0000 7f1492fff700 1 -- 172.21.15.45:0/1484853516 wait complete. 2023-10-17T01:30:23.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:30:23.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 -- 172.21.15.45:0/3282202847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 msgr2=0x7f7304101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 --2- 172.21.15.45:0/3282202847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304101160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009990 tx=0x7f72f8009ca0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 -- 172.21.15.45:0/3282202847 shutdown_connections 2023-10-17T01:30:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 --2- 172.21.15.45:0/3282202847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 0x7f7304109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 --2- 172.21.15.45:0/3282202847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304101160 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 --2- 172.21.15.45:0/3282202847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7304101f30 0x7f7304102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 -- 172.21.15.45:0/3282202847 >> 172.21.15.45:0/3282202847 conn(0x7f73040fc290 msgr2=0x7f73040fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 -- 172.21.15.45:0/3282202847 shutdown_connections 2023-10-17T01:30:23.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.457+0000 7f730adff700 1 -- 172.21.15.45:0/3282202847 wait complete. 2023-10-17T01:30:23.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.459+0000 7f730adff700 1 Processor -- start 2023-10-17T01:30:23.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.459+0000 7f730adff700 1 -- start start 2023-10-17T01:30:23.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.459+0000 7f730adff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.459+0000 7f730adff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7304101f30 0x7f7304196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 0x7f730419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730adff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730419e610 con 0x7f7304101f30 2023-10-17T01:30:23.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730419e750 con 0x7f73041028d0 2023-10-17T01:30:23.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730adff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730419e8e0 con 0x7f7304100d40 2023-10-17T01:30:23.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f7308b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f7308b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60474/0 (socket says 172.21.15.45:60474) 2023-10-17T01:30:23.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f7308b9b700 1 -- 172.21.15.45:0/3073131077 learned_addr learned my addr 172.21.15.45:0/3073131077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:23.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 0x7f730419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f7303fff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7304101f30 0x7f7304196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 -- 172.21.15.45:0/3073131077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 msgr2=0x7f7304196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 -- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7304101f30 msgr2=0x7f7304196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7304101f30 0x7f7304196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 -- 172.21.15.45:0/3073131077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80095f0 con 0x7f73041028d0 2023-10-17T01:30:23.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f7308b9b700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:30:23.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.460+0000 7f730939c700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 0x7f730419e040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00efc0 tx=0x7f72fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:23.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.461+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc018070 con 0x7f73041028d0 2023-10-17T01:30:23.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.461+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72fc00f040 con 0x7f73041028d0 2023-10-17T01:30:23.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.461+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730419eb60 con 0x7f73041028d0 2023-10-17T01:30:23.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.461+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00cd40 con 0x7f73041028d0 2023-10-17T01:30:23.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.461+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730419f070 con 0x7f73041028d0 2023-10-17T01:30:23.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.462+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730410c5a0 con 0x7f73041028d0 2023-10-17T01:30:23.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.464+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f72fc003a00 con 0x7f73041028d0 2023-10-17T01:30:23.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.465+0000 7f7301ffb700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f72f40a0b20 0x7f72f40a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.465+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f72fc0cbd40 con 0x7f73041028d0 2023-10-17T01:30:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.465+0000 7f7308b9b700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f72f40a0b20 0x7f72f40a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.466+0000 7f7308b9b700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f72f40a0b20 0x7f72f40a2fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f72f800b2e0 tx=0x7f72f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.469+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f72fc095730 con 0x7f73041028d0 2023-10-17T01:30:23.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:23 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:30:23.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:23 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3091047043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:30:23.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:23 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2037539280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:30:23.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:23 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1484853516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:30:23.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:23 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:30:23.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:23 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3091047043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:30:23.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:23 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2037539280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:30:23.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:23 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1484853516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:30:23.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:23 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:30:23.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:23 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3091047043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:30:23.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:23 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2037539280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:30:23.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:23 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1484853516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:30:23.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.683+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7304107c80 con 0x7f73041028d0 2023-10-17T01:30:23.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.684+0000 7f7301ffb700 1 -- 172.21.15.45:0/3073131077 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72fc098d50 con 0x7f73041028d0 2023-10-17T01:30:23.686 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:30:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f72f40a0b20 msgr2=0x7f72f40a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f72f40a0b20 0x7f72f40a2fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f72f800b2e0 tx=0x7f72f8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 msgr2=0x7f730419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 0x7f730419e040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00efc0 tx=0x7f72fc00c3a0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 shutdown_connections 2023-10-17T01:30:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f72f40a0b20 0x7f72f40a2fe0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73041028d0 0x7f730419e040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304100d40 0x7f7304196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 --2- 172.21.15.45:0/3073131077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7304101f30 0x7f7304196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 >> 172.21.15.45:0/3073131077 conn(0x7f73040fc290 msgr2=0x7f73040fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 shutdown_connections 2023-10-17T01:30:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.686+0000 7f730adff700 1 -- 172.21.15.45:0/3073131077 wait complete. 2023-10-17T01:30:23.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-17T01:30:23.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.909+0000 7f33153ff700 1 -- 172.21.15.45:0/1409331874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104180 msgr2=0x7f33101045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.909+0000 7f33153ff700 1 --2- 172.21.15.45:0/1409331874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104180 0x7f33101045e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f32f80099c0 tx=0x7f32f8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 -- 172.21.15.45:0/1409331874 shutdown_connections 2023-10-17T01:30:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 --2- 172.21.15.45:0/1409331874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104b20 0x7f331010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 --2- 172.21.15.45:0/1409331874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f33101033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 --2- 172.21.15.45:0/1409331874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104180 0x7f33101045e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 -- 172.21.15.45:0/1409331874 >> 172.21.15.45:0/1409331874 conn(0x7f33100fe500 msgr2=0x7f3310100940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 -- 172.21.15.45:0/1409331874 shutdown_connections 2023-10-17T01:30:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.910+0000 7f33153ff700 1 -- 172.21.15.45:0/1409331874 wait complete. 2023-10-17T01:30:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 Processor -- start 2023-10-17T01:30:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 -- start start 2023-10-17T01:30:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f3310198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104180 0x7f3310199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104b20 0x7f33101a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101a0770 con 0x7f3310104180 2023-10-17T01:30:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101a08e0 con 0x7f3310104b20 2023-10-17T01:30:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.911+0000 7f33153ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101a0aa0 con 0x7f3310102f90 2023-10-17T01:30:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f3310198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f3310198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:60488/0 (socket says 172.21.15.45:60488) 2023-10-17T01:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 -- 172.21.15.45:0/2576697002 learned_addr learned my addr 172.21.15.45:0/2576697002 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f3307fff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104180 0x7f3310199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330f7fe700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104b20 0x7f33101a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 -- 172.21.15.45:0/2576697002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104b20 msgr2=0x7f33101a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104b20 0x7f33101a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 -- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104180 msgr2=0x7f3310199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104180 0x7f3310199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 -- 172.21.15.45:0/2576697002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80095f0 con 0x7f3310102f90 2023-10-17T01:30:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f3307fff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104180 0x7f3310199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:30:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.912+0000 7f330effd700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f3310198c30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f330000e970 tx=0x7f330000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.913+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300009790 con 0x7f3310102f90 2023-10-17T01:30:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.913+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f330000c7e0 con 0x7f3310102f90 2023-10-17T01:30:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.913+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33101a0d80 con 0x7f3310102f90 2023-10-17T01:30:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.913+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300003c70 con 0x7f3310102f90 2023-10-17T01:30:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.913+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33101a11d0 con 0x7f3310102f90 2023-10-17T01:30:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.914+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331010e7f0 con 0x7f3310102f90 2023-10-17T01:30:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.916+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f330000cc50 con 0x7f3310102f90 2023-10-17T01:30:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.917+0000 7f330cff9700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32fc0a0b70 0x7f32fc0a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:30:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.917+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3300014070 con 0x7f3310102f90 2023-10-17T01:30:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.917+0000 7f3307fff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32fc0a0b70 0x7f32fc0a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:30:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.918+0000 7f3307fff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32fc0a0b70 0x7f32fc0a3030 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b580 tx=0x7f32f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:30:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:23.921+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f3300095e30 con 0x7f3310102f90 2023-10-17T01:30:24.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.193+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f331010e9b0 con 0x7f3310102f90 2023-10-17T01:30:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.203+0000 7f330cff9700 1 -- 172.21.15.45:0/2576697002 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+731614 (secure 0 0 0) 0x7f3300003dd0 con 0x7f3310102f90 2023-10-17T01:30:24.214 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:30:24.214 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:30:24.214 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 18, 2023-10-17T01:30:24.214 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 14150, 2023-10-17T01:30:24.215 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2023-10-17T01:30:24.215 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2023-10-17T01:30:24.215 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-10-17T01:30:24.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-10-17T01:30:24.216 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2023-10-17T01:30:24.216 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 869456926 2023-10-17T01:30:24.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-17T01:30:24.217 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2023-10-17T01:30:24.217 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 869456926 2023-10-17T01:30:24.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.217 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:24.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.218 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/869456926", 2023-10-17T01:30:24.218 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2023-10-17T01:06:32.349062+0000", 2023-10-17T01:30:24.218 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540138322906710015, 2023-10-17T01:30:24.218 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2023-10-17T01:30:24.218 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2023-10-17T01:30:24.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.219 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24109, 2023-10-17T01:30:24.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2023-10-17T01:30:24.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540138322906710015, 2023-10-17T01:30:24.219 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2023-10-17T01:30:24.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2023-10-17T01:30:24.220 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.220 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.220 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.221 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:24.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:24.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:24.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2023-10-17T01:30:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2023-10-17T01:30:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2023-10-17T01:30:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2023-10-17T01:30:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2023-10-17T01:30:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2023-10-17T01:30:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2023-10-17T01:30:24.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2023-10-17T01:30:24.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.243 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2023-10-17T01:30:24.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2023-10-17T01:30:24.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2023-10-17T01:30:24.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.246 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2023-10-17T01:30:24.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2023-10-17T01:30:24.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2023-10-17T01:30:24.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.248 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2023-10-17T01:30:24.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2023-10-17T01:30:24.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2023-10-17T01:30:24.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2023-10-17T01:30:24.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.251 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.251 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2023-10-17T01:30:24.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2023-10-17T01:30:24.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2023-10-17T01:30:24.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2023-10-17T01:30:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2023-10-17T01:30:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-17T01:30:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2023-10-17T01:30:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2023-10-17T01:30:24.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2023-10-17T01:30:24.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2023-10-17T01:30:24.260 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.260 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.260 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.260 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2023-10-17T01:30:24.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2023-10-17T01:30:24.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2023-10-17T01:30:24.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.263 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2023-10-17T01:30:24.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2023-10-17T01:30:24.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2023-10-17T01:30:24.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2023-10-17T01:30:24.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-17T01:30:24.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.266 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2023-10-17T01:30:24.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2023-10-17T01:30:24.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:24.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:24.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-10-17T01:30:24.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-10-17T01:30:24.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-17T01:30:24.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-17T01:30:24.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.268 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2023-10-17T01:30:24.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2023-10-17T01:30:24.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:24.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2023-10-17T01:30:24.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:24.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2023-10-17T01:30:24.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-17T01:30:24.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.271 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2023-10-17T01:30:24.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2023-10-17T01:30:24.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2023-10-17T01:30:24.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-17T01:30:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-17T01:30:24.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2023-10-17T01:30:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2023-10-17T01:30:24.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2023-10-17T01:30:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2023-10-17T01:30:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2023-10-17T01:30:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2023-10-17T01:30:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-17T01:30:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2023-10-17T01:30:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2023-10-17T01:30:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2023-10-17T01:30:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2023-10-17T01:30:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-17T01:30:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.279 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2023-10-17T01:30:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2023-10-17T01:30:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-10-17T01:30:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-10-17T01:30:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-17T01:30:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-17T01:30:24.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.284 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.292 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.293 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2023-10-17T01:30:24.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2023-10-17T01:30:24.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:24.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:24.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-17T01:30:24.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.298 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-10-17T01:30:24.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-10-17T01:30:24.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2023-10-17T01:30:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2023-10-17T01:30:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2023-10-17T01:30:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2023-10-17T01:30:24.301 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2023-10-17T01:30:24.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.302 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2023-10-17T01:30:24.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2023-10-17T01:30:24.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-17T01:30:24.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.304 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:24.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:24.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:24.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-17T01:30:24.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.306 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.307 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2023-10-17T01:30:24.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2023-10-17T01:30:24.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:24.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:24.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-17T01:30:24.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-17T01:30:24.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.310 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2023-10-17T01:30:24.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2023-10-17T01:30:24.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:24.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-17T01:30:24.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2023-10-17T01:30:24.313 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.313 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.313 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.313 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2023-10-17T01:30:24.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2023-10-17T01:30:24.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:24.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-10-17T01:30:24.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-17T01:30:24.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.316 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2023-10-17T01:30:24.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2023-10-17T01:30:24.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2023-10-17T01:30:24.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-17T01:30:24.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.319 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2023-10-17T01:30:24.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2023-10-17T01:30:24.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2023-10-17T01:30:24.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-17T01:30:24.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.322 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2023-10-17T01:30:24.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2023-10-17T01:30:24.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-17T01:30:24.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-17T01:30:24.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.324 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2023-10-17T01:30:24.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2023-10-17T01:30:24.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:24.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-17T01:30:24.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.327 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2023-10-17T01:30:24.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2023-10-17T01:30:24.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:24.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2023-10-17T01:30:24.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-17T01:30:24.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.330 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2023-10-17T01:30:24.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2023-10-17T01:30:24.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-17T01:30:24.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.332 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2023-10-17T01:30:24.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2023-10-17T01:30:24.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-17T01:30:24.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.335 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2023-10-17T01:30:24.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2023-10-17T01:30:24.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-17T01:30:24.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.337 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2023-10-17T01:30:24.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2023-10-17T01:30:24.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-17T01:30:24.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:24.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.340 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2023-10-17T01:30:24.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2023-10-17T01:30:24.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph:v18", 2023-10-17T01:30:24.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2023-10-17T01:30:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.343 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2023-10-17T01:30:24.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2023-10-17T01:30:24.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-17T01:30:24.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2023-10-17T01:30:24.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.345 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2023-10-17T01:30:24.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2023-10-17T01:30:24.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-17T01:30:24.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:24.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.348 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2023-10-17T01:30:24.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2023-10-17T01:30:24.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-17T01:30:24.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2023-10-17T01:30:24.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2023-10-17T01:30:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2023-10-17T01:30:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-17T01:30:24.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2023-10-17T01:30:24.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.353 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2023-10-17T01:30:24.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2023-10-17T01:30:24.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-17T01:30:24.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2023-10-17T01:30:24.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.356 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2023-10-17T01:30:24.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2023-10-17T01:30:24.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-17T01:30:24.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2023-10-17T01:30:24.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.359 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2023-10-17T01:30:24.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2023-10-17T01:30:24.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-17T01:30:24.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2023-10-17T01:30:24.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.362 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2023-10-17T01:30:24.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2023-10-17T01:30:24.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-17T01:30:24.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2023-10-17T01:30:24.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.365 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2023-10-17T01:30:24.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2023-10-17T01:30:24.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-17T01:30:24.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:24.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.368 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2023-10-17T01:30:24.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2023-10-17T01:30:24.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-10-17T01:30:24.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2023-10-17T01:30:24.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.370 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2023-10-17T01:30:24.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2023-10-17T01:30:24.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-17T01:30:24.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:24.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.373 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2023-10-17T01:30:24.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2023-10-17T01:30:24.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-17T01:30:24.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2023-10-17T01:30:24.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.375 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2023-10-17T01:30:24.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2023-10-17T01:30:24.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-17T01:30:24.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2023-10-17T01:30:24.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.378 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2023-10-17T01:30:24.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2023-10-17T01:30:24.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2023-10-17T01:30:24.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.381 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2023-10-17T01:30:24.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2023-10-17T01:30:24.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:24.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-17T01:30:24.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2023-10-17T01:30:24.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2023-10-17T01:30:24.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2023-10-17T01:30:24.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-17T01:30:24.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2023-10-17T01:30:24.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2023-10-17T01:30:24.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2023-10-17T01:30:24.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-17T01:30:24.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2023-10-17T01:30:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2023-10-17T01:30:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2023-10-17T01:30:24.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2023-10-17T01:30:24.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.392 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2023-10-17T01:30:24.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2023-10-17T01:30:24.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-17T01:30:24.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.394 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2023-10-17T01:30:24.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2023-10-17T01:30:24.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:24.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2023-10-17T01:30:24.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.397 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2023-10-17T01:30:24.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2023-10-17T01:30:24.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2023-10-17T01:30:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2023-10-17T01:30:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2023-10-17T01:30:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-17T01:30:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2023-10-17T01:30:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2023-10-17T01:30:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-17T01:30:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-17T01:30:24.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.414 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.414 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.414 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.414 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.415 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.415 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.416 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.419 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-17T01:30:24.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-17T01:30:24.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-17T01:30:24.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.421 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-17T01:30:24.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-17T01:30:24.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2023-10-17T01:30:24.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-17T01:30:24.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2023-10-17T01:30:24.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2023-10-17T01:30:24.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2023-10-17T01:30:24.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2023-10-17T01:30:24.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2023-10-17T01:30:24.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-17T01:30:24.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.429 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2023-10-17T01:30:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2023-10-17T01:30:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2023-10-17T01:30:24.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-10-17T01:30:24.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-10-17T01:30:24.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2023-10-17T01:30:24.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.434 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2023-10-17T01:30:24.434 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2023-10-17T01:30:24.434 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2023-10-17T01:30:24.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.435 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2023-10-17T01:30:24.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2023-10-17T01:30:24.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-17T01:30:24.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-17T01:30:24.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.438 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2023-10-17T01:30:24.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2023-10-17T01:30:24.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-17T01:30:24.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.441 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2023-10-17T01:30:24.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2023-10-17T01:30:24.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-17T01:30:24.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.443 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2023-10-17T01:30:24.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2023-10-17T01:30:24.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-17T01:30:24.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.448 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2023-10-17T01:30:24.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2023-10-17T01:30:24.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-17T01:30:24.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.450 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2023-10-17T01:30:24.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2023-10-17T01:30:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-17T01:30:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2023-10-17T01:30:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2023-10-17T01:30:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2023-10-17T01:30:24.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2023-10-17T01:30:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2023-10-17T01:30:24.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2023-10-17T01:30:24.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-17T01:30:24.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.458 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2023-10-17T01:30:24.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2023-10-17T01:30:24.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-17T01:30:24.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.461 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2023-10-17T01:30:24.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2023-10-17T01:30:24.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-17T01:30:24.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.464 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2023-10-17T01:30:24.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2023-10-17T01:30:24.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2023-10-17T01:30:24.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.466 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2023-10-17T01:30:24.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2023-10-17T01:30:24.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-17T01:30:24.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.469 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2023-10-17T01:30:24.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2023-10-17T01:30:24.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-17T01:30:24.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2023-10-17T01:30:24.472 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.472 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.473 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.473 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.475 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.475 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.475 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.475 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.475 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.476 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.476 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.481 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.481 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.481 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.482 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.482 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.482 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.482 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.483 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.486 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2023-10-17T01:30:24.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2023-10-17T01:30:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2023-10-17T01:30:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-17T01:30:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.489 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2023-10-17T01:30:24.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2023-10-17T01:30:24.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2023-10-17T01:30:24.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-17T01:30:24.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2023-10-17T01:30:24.492 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.493 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.493 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.493 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-17T01:30:24.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-17T01:30:24.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.496 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2023-10-17T01:30:24.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-17T01:30:24.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.499 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-17T01:30:24.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-17T01:30:24.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.501 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2023-10-17T01:30:24.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2023-10-17T01:30:24.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.504 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-17T01:30:24.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-17T01:30:24.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.507 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2023-10-17T01:30:24.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-17T01:30:24.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.509 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-17T01:30:24.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-17T01:30:24.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.512 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-17T01:30:24.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-17T01:30:24.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.514 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.514 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.514 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.514 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-17T01:30:24.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-17T01:30:24.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.517 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-17T01:30:24.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-17T01:30:24.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.520 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-17T01:30:24.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-17T01:30:24.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.522 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-17T01:30:24.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-17T01:30:24.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.525 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-17T01:30:24.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-17T01:30:24.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.528 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-17T01:30:24.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-17T01:30:24.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.530 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2023-10-17T01:30:24.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-17T01:30:24.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:24.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.533 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-17T01:30:24.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-17T01:30:24.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.536 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2023-10-17T01:30:24.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2023-10-17T01:30:24.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.538 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2023-10-17T01:30:24.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-17T01:30:24.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:24.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.541 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-17T01:30:24.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-17T01:30:24.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.544 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-17T01:30:24.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-17T01:30:24.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.546 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-17T01:30:24.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-17T01:30:24.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.549 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-17T01:30:24.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-17T01:30:24.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.552 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2023-10-17T01:30:24.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-17T01:30:24.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.554 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-17T01:30:24.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-17T01:30:24.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.557 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-17T01:30:24.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-17T01:30:24.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-17T01:30:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-17T01:30:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-17T01:30:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-17T01:30:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-17T01:30:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-17T01:30:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-17T01:30:24.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-17T01:30:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-17T01:30:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-17T01:30:24.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.572 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-17T01:30:24.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-17T01:30:24.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.575 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2023-10-17T01:30:24.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-17T01:30:24.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.578 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-17T01:30:24.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-17T01:30:24.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-17T01:30:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.580 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-17T01:30:24.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-17T01:30:24.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.583 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-17T01:30:24.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-17T01:30:24.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2023-10-17T01:30:24.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.585 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-17T01:30:24.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-17T01:30:24.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2023-10-17T01:30:24.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.588 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2023-10-17T01:30:24.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-17T01:30:24.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.591 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-17T01:30:24.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-17T01:30:24.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:24.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.593 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2023-10-17T01:30:24.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-17T01:30:24.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.596 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2023-10-17T01:30:24.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-17T01:30:24.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.599 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2023-10-17T01:30:24.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-17T01:30:24.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.601 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-17T01:30:24.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-17T01:30:24.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:24.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.604 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-17T01:30:24.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-17T01:30:24.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.607 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-17T01:30:24.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-17T01:30:24.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:24.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.609 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2023-10-17T01:30:24.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2023-10-17T01:30:24.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.612 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2023-10-17T01:30:24.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2023-10-17T01:30:24.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.615 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2023-10-17T01:30:24.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2023-10-17T01:30:24.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2023-10-17T01:30:24.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.618 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2023-10-17T01:30:24.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2023-10-17T01:30:24.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2023-10-17T01:30:24.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.620 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-10-17T01:30:24.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-10-17T01:30:24.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.623 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.624 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.625 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.625 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.625 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.625 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.625 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.625 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.627 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.629 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.631 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.631 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.631 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.632 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.632 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.632 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.632 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.633 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.636 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2023-10-17T01:30:24.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2023-10-17T01:30:24.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2023-10-17T01:30:24.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.639 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2023-10-17T01:30:24.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2023-10-17T01:30:24.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.641 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:24.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:24.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-10-17T01:30:24.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.644 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:24.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:24.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2023-10-17T01:30:24.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.646 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-10-17T01:30:24.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-10-17T01:30:24.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.649 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2023-10-17T01:30:24.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2023-10-17T01:30:24.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2023-10-17T01:30:24.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.652 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-10-17T01:30:24.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-10-17T01:30:24.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2023-10-17T01:30:24.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.654 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.654 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2023-10-17T01:30:24.654 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.654 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.655 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-10-17T01:30:24.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-10-17T01:30:24.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-10-17T01:30:24.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-10-17T01:30:24.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-10-17T01:30:24.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.658 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2023-10-17T01:30:24.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2023-10-17T01:30:24.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-10-17T01:30:24.661 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.661 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.662 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2023-10-17T01:30:24.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2023-10-17T01:30:24.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2023-10-17T01:30:24.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.664 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.666 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.666 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.667 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.667 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.667 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.667 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.667 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.668 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.671 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.673 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.673 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.673 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.673 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.674 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.674 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.674 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.675 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.677 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.678 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2023-10-17T01:30:24.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2023-10-17T01:30:24.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2023-10-17T01:30:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-17T01:30:24.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.681 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2023-10-17T01:30:24.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2023-10-17T01:30:24.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2023-10-17T01:30:24.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-17T01:30:24.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.683 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2023-10-17T01:30:24.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2023-10-17T01:30:24.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2023-10-17T01:30:24.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2023-10-17T01:30:24.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.686 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2023-10-17T01:30:24.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2023-10-17T01:30:24.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-10-17T01:30:24.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-17T01:30:24.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.689 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2023-10-17T01:30:24.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2023-10-17T01:30:24.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-17T01:30:24.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.691 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:24.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:24.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:24.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2023-10-17T01:30:24.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.694 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2023-10-17T01:30:24.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2023-10-17T01:30:24.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2023-10-17T01:30:24.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-17T01:30:24.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2023-10-17T01:30:24.698 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.698 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.698 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.698 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.700 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.700 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.700 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.701 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.701 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.701 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.701 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.702 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.704 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.705 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.707 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.707 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.707 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.708 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.708 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.708 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.708 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.709 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.712 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2023-10-17T01:30:24.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2023-10-17T01:30:24.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-10-17T01:30:24.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.715 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2023-10-17T01:30:24.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2023-10-17T01:30:24.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2023-10-17T01:30:24.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.717 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:24.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:24.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:24.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.720 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.720 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2023-10-17T01:30:24.721 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2023-10-17T01:30:24.721 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2023-10-17T01:30:24.721 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.721 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2023-10-17T01:30:24.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2023-10-17T01:30:24.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.722 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2023-10-17T01:30:24.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-17T01:30:24.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.724 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2023-10-17T01:30:24.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2023-10-17T01:30:24.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2023-10-17T01:30:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-17T01:30:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.727 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2023-10-17T01:30:24.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2023-10-17T01:30:24.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2023-10-17T01:30:24.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.729 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:24.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:24.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2023-10-17T01:30:24.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2023-10-17T01:30:24.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-17T01:30:24.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.732 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.734 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.734 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.734 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.734 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.734 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.735 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.735 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.736 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.740 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.741 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.741 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.741 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.741 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.741 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.742 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.743 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.745 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2023-10-17T01:30:24.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2023-10-17T01:30:24.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.746 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2023-10-17T01:30:24.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.748 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2023-10-17T01:30:24.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2023-10-17T01:30:24.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2023-10-17T01:30:24.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2023-10-17T01:30:24.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.751 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-10-17T01:30:24.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-10-17T01:30:24.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2023-10-17T01:30:24.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-17T01:30:24.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.753 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2023-10-17T01:30:24.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2023-10-17T01:30:24.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:24.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:24.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2023-10-17T01:30:24.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-17T01:30:24.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.756 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2023-10-17T01:30:24.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2023-10-17T01:30:24.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2023-10-17T01:30:24.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.759 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2023-10-17T01:30:24.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2023-10-17T01:30:24.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.760 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2023-10-17T01:30:24.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-17T01:30:24.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2023-10-17T01:30:24.762 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.763 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.763 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.763 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.765 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.765 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.765 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.765 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.765 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.766 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.766 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.767 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.770 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.771 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.771 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.772 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.772 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.772 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.772 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.772 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.774 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2023-10-17T01:30:24.777 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.777 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.777 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.778 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.779 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.779 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.780 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.780 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.780 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.780 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.780 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.782 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.784 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.786 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.786 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.786 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.786 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.786 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.786 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.787 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.788 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2023-10-17T01:30:24.791 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.792 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.792 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.792 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2023-10-17T01:30:24.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2023-10-17T01:30:24.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2023-10-17T01:30:24.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-17T01:30:24.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.795 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2023-10-17T01:30:24.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2023-10-17T01:30:24.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:24.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2023-10-17T01:30:24.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-17T01:30:24.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.800 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.800 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.800 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.801 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.804 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.805 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.806 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.806 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.806 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.806 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.806 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.807 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.808 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2023-10-17T01:30:24.811 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.811 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.812 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.812 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2023-10-17T01:30:24.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2023-10-17T01:30:24.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2023-10-17T01:30:24.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2023-10-17T01:30:24.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-17T01:30:24.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.814 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.816 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.818 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.821 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.823 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.823 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.823 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.823 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.823 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.824 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.824 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2023-10-17T01:30:24.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2023-10-17T01:30:24.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2023-10-17T01:30:24.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-17T01:30:24.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.830 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2023-10-17T01:30:24.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2023-10-17T01:30:24.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2023-10-17T01:30:24.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2023-10-17T01:30:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.833 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2023-10-17T01:30:24.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2023-10-17T01:30:24.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2023-10-17T01:30:24.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2023-10-17T01:30:24.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.835 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2023-10-17T01:30:24.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2023-10-17T01:30:24.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2023-10-17T01:30:24.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.838 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2023-10-17T01:30:24.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2023-10-17T01:30:24.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2023-10-17T01:30:24.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-17T01:30:24.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-17T01:30:24.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2023-10-17T01:30:24.841 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.842 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.842 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.842 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.844 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.844 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.844 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.844 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.845 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.845 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.846 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.849 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.851 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.851 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.851 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.851 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.851 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.852 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.852 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.853 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2023-10-17T01:30:24.856 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.857 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.857 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.857 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.859 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.859 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.859 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.859 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.860 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.860 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.860 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.861 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.864 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.865 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.866 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.866 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.866 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.866 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.866 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.867 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2023-10-17T01:30:24.871 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.871 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.871 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.872 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.873 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.873 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.874 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.874 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.874 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.874 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.878 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.880 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.880 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.880 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.881 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.881 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.881 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.882 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-10-17T01:30:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2023-10-17T01:30:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2023-10-17T01:30:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-17T01:30:24.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.889 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.890 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.891 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.891 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.891 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.891 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.891 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.891 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.893 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.895 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.897 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.897 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.897 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.898 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.898 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.898 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.898 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.899 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.902 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2023-10-17T01:30:24.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-10-17T01:30:24.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.904 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-10-17T01:30:24.904 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2023-10-17T01:30:24.904 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2023-10-17T01:30:24.904 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2023-10-17T01:30:24.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2023-10-17T01:30:24.906 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.906 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.907 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.907 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.908 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.909 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.909 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.909 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.909 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.909 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.910 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.911 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.916 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.916 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.916 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.916 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.916 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.917 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2023-10-17T01:30:24.921 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.921 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.921 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.922 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.923 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.923 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.924 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.924 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.924 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.924 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.925 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.931 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.931 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2023-10-17T01:30:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.938 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.938 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.938 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.938 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.938 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.939 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.939 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.940 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.944 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.944 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.945 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.945 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.945 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.945 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.945 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.946 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.949 INFO:teuthology.orchestra.run.smithi045.stdout: "noautoscale": { 2023-10-17T01:30:24.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "noautoscale", 2023-10-17T01:30:24.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "global autoscale flag", 2023-10-17T01:30:24.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-10-17T01:30:24.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.952 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:24.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:24.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:24.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.954 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2023-10-17T01:30:24.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2023-10-17T01:30:24.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:24.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-10-17T01:30:24.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2023-10-17T01:30:24.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2023-10-17T01:30:24.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-17T01:30:24.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2023-10-17T01:30:24.958 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.958 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.958 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.958 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2023-10-17T01:30:24.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2023-10-17T01:30:24.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-17T01:30:24.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-10-17T01:30:24.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-10-17T01:30:24.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.964 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.965 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.966 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.966 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.966 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.966 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.966 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.966 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.968 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.970 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.972 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.972 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.972 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.973 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.973 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.973 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.973 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.974 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:24.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2023-10-17T01:30:24.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2023-10-17T01:30:24.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:24.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2023-10-17T01:30:24.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2023-10-17T01:30:24.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.980 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:24.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:24.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:24.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:24.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2023-10-17T01:30:24.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:24.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:24.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2023-10-17T01:30:24.983 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:24.983 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:24.983 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:24.983 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2023-10-17T01:30:24.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2023-10-17T01:30:24.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:24.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.985 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.986 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2023-10-17T01:30:24.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2023-10-17T01:30:24.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:24.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-17T01:30:24.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-17T01:30:24.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.989 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:24.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:24.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:24.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.990 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.991 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.991 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.991 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.991 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.991 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.992 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.993 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:24.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:24.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:24.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:24.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:24.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.995 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:24.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:24.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:24.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:24.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:24.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:24.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:24.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:24.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:24.997 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:24.997 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:24.997 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:24.998 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:24.998 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:24.998 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:24.998 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:24.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:24.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:24.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:24.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:24.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:24.999 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.002 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2023-10-17T01:30:25.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2023-10-17T01:30:25.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.005 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-17T01:30:25.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-17T01:30:25.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2023-10-17T01:30:25.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.007 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2023-10-17T01:30:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2023-10-17T01:30:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2023-10-17T01:30:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:25.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-10-17T01:30:25.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-17T01:30:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2023-10-17T01:30:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-17T01:30:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2023-10-17T01:30:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2023-10-17T01:30:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2023-10-17T01:30:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-10-17T01:30:25.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-10-17T01:30:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2023-10-17T01:30:25.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2023-10-17T01:30:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2023-10-17T01:30:25.020 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-10-17T01:30:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-10-17T01:30:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-10-17T01:30:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-10-17T01:30:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-10-17T01:30:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-10-17T01:30:25.025 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.025 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.025 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.028 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.028 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.028 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.029 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.032 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.034 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.034 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.034 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.034 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.034 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.035 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.035 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.036 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2023-10-17T01:30:25.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2023-10-17T01:30:25.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2023-10-17T01:30:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2023-10-17T01:30:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2023-10-17T01:30:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2023-10-17T01:30:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2023-10-17T01:30:25.047 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.048 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.048 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2023-10-17T01:30:25.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2023-10-17T01:30:25.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.051 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-10-17T01:30:25.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-10-17T01:30:25.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.053 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.065 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.068 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:25.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:25.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.071 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:25.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:25.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2023-10-17T01:30:25.074 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.074 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.074 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.075 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.076 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.076 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.077 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.077 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.077 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.077 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.077 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.079 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.081 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.083 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.083 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.083 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.083 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.084 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.084 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.084 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.085 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2023-10-17T01:30:25.089 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.089 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.089 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.089 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2023-10-17T01:30:25.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2023-10-17T01:30:25.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2023-10-17T01:30:25.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-17T01:30:25.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.092 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.093 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.094 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.094 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.094 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.094 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.094 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.095 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.098 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.100 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.100 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.100 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.101 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.101 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.101 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.101 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.105 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2023-10-17T01:30:25.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2023-10-17T01:30:25.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2023-10-17T01:30:25.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-17T01:30:25.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.107 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.108 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2023-10-17T01:30:25.108 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.109 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.109 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.109 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.111 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.111 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.111 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.111 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.111 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.112 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.112 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.113 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.116 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.117 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.118 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.118 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.118 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.118 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.118 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.119 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.120 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.122 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2023-10-17T01:30:25.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2023-10-17T01:30:25.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.125 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2023-10-17T01:30:25.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2023-10-17T01:30:25.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2023-10-17T01:30:25.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.128 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2023-10-17T01:30:25.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2023-10-17T01:30:25.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.130 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2023-10-17T01:30:25.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2023-10-17T01:30:25.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2023-10-17T01:30:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2023-10-17T01:30:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2023-10-17T01:30:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2023-10-17T01:30:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.138 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2023-10-17T01:30:25.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2023-10-17T01:30:25.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.141 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.141 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2023-10-17T01:30:25.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2023-10-17T01:30:25.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.144 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2023-10-17T01:30:25.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2023-10-17T01:30:25.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:25.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2023-10-17T01:30:25.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.147 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2023-10-17T01:30:25.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2023-10-17T01:30:25.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.149 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2023-10-17T01:30:25.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2023-10-17T01:30:25.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.152 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2023-10-17T01:30:25.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2023-10-17T01:30:25.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.155 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-10-17T01:30:25.155 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.156 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.156 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.156 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2023-10-17T01:30:25.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2023-10-17T01:30:25.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2023-10-17T01:30:25.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.159 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2023-10-17T01:30:25.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2023-10-17T01:30:25.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2023-10-17T01:30:25.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.161 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.163 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.163 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.163 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.164 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.164 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.164 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.164 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.168 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.170 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.170 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.170 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.170 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.171 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.171 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.171 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.172 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.175 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2023-10-17T01:30:25.175 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.176 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.176 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.176 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.178 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.178 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.178 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.179 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.179 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.179 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.179 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.180 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.183 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.185 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.185 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.185 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.185 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.186 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.186 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.186 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.187 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2023-10-17T01:30:25.191 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.191 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.191 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.191 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.193 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.193 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.193 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.193 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.194 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.194 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.194 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.195 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.198 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.200 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.200 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.200 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.200 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.201 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.201 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.201 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.202 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2023-10-17T01:30:25.206 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.206 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.206 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.206 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2023-10-17T01:30:25.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2023-10-17T01:30:25.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-17T01:30:25.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.209 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:25.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:25.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2023-10-17T01:30:25.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.211 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.213 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.213 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.213 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.214 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.214 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.214 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.214 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.215 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.218 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.220 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.220 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.220 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.221 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.221 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.221 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.221 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.222 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2023-10-17T01:30:25.226 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.226 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.226 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.226 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2023-10-17T01:30:25.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2023-10-17T01:30:25.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-17T01:30:25.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.229 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2023-10-17T01:30:25.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2023-10-17T01:30:25.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-17T01:30:25.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.232 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2023-10-17T01:30:25.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2023-10-17T01:30:25.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-17T01:30:25.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.234 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2023-10-17T01:30:25.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2023-10-17T01:30:25.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-17T01:30:25.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.237 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2023-10-17T01:30:25.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2023-10-17T01:30:25.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-17T01:30:25.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.239 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2023-10-17T01:30:25.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2023-10-17T01:30:25.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.242 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2023-10-17T01:30:25.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2023-10-17T01:30:25.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.245 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2023-10-17T01:30:25.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2023-10-17T01:30:25.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-17T01:30:25.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-10-17T01:30:25.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-10-17T01:30:25.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.250 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:25.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:25.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2023-10-17T01:30:25.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2023-10-17T01:30:25.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.253 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2023-10-17T01:30:25.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2023-10-17T01:30:25.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2023-10-17T01:30:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.255 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2023-10-17T01:30:25.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2023-10-17T01:30:25.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.258 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2023-10-17T01:30:25.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2023-10-17T01:30:25.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.261 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.265 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.265 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.265 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.266 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.266 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.266 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.266 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.267 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.270 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.272 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.272 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.272 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.272 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.273 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.273 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.273 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.274 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.277 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2023-10-17T01:30:25.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2023-10-17T01:30:25.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.279 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2023-10-17T01:30:25.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2023-10-17T01:30:25.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.282 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2023-10-17T01:30:25.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2023-10-17T01:30:25.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-17T01:30:25.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2023-10-17T01:30:25.285 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.285 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.286 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.286 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.287 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.288 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.288 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.288 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.288 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.288 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.289 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.294 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.294 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.294 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.296 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-10-17T01:30:25.300 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.300 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.300 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.300 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.302 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.302 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.302 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.303 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.303 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.303 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.303 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.304 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.307 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.309 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.309 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.309 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.309 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.309 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.309 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.310 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.311 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2023-10-17T01:30:25.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2023-10-17T01:30:25.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2023-10-17T01:30:25.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-17T01:30:25.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2023-10-17T01:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2023-10-17T01:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:25.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-17T01:30:25.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2023-10-17T01:30:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2023-10-17T01:30:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2023-10-17T01:30:25.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2023-10-17T01:30:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2023-10-17T01:30:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2023-10-17T01:30:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:25.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.330 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.336 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2023-10-17T01:30:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2023-10-17T01:30:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2023-10-17T01:30:25.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2023-10-17T01:30:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2023-10-17T01:30:25.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.347 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2023-10-17T01:30:25.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2023-10-17T01:30:25.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-17T01:30:25.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24113, 2023-10-17T01:30:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2023-10-17T01:30:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540138322906710015, 2023-10-17T01:30:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2023-10-17T01:30:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2023-10-17T01:30:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.356 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.357 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.361 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.363 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.365 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2023-10-17T01:30:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2023-10-17T01:30:25.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2023-10-17T01:30:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2023-10-17T01:30:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2023-10-17T01:30:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2023-10-17T01:30:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2023-10-17T01:30:25.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2023-10-17T01:30:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2023-10-17T01:30:25.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2023-10-17T01:30:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2023-10-17T01:30:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2023-10-17T01:30:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2023-10-17T01:30:25.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2023-10-17T01:30:25.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2023-10-17T01:30:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2023-10-17T01:30:25.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2023-10-17T01:30:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2023-10-17T01:30:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2023-10-17T01:30:25.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2023-10-17T01:30:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2023-10-17T01:30:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2023-10-17T01:30:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-17T01:30:25.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2023-10-17T01:30:25.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2023-10-17T01:30:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2023-10-17T01:30:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2023-10-17T01:30:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2023-10-17T01:30:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2023-10-17T01:30:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2023-10-17T01:30:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2023-10-17T01:30:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2023-10-17T01:30:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2023-10-17T01:30:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2023-10-17T01:30:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-17T01:30:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2023-10-17T01:30:25.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2023-10-17T01:30:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:25.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-10-17T01:30:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-10-17T01:30:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-17T01:30:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-17T01:30:25.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2023-10-17T01:30:25.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2023-10-17T01:30:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2023-10-17T01:30:25.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2023-10-17T01:30:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-17T01:30:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2023-10-17T01:30:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2023-10-17T01:30:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2023-10-17T01:30:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-17T01:30:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-17T01:30:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2023-10-17T01:30:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2023-10-17T01:30:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2023-10-17T01:30:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2023-10-17T01:30:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2023-10-17T01:30:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2023-10-17T01:30:25.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-17T01:30:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2023-10-17T01:30:25.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2023-10-17T01:30:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2023-10-17T01:30:25.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2023-10-17T01:30:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-17T01:30:25.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2023-10-17T01:30:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2023-10-17T01:30:25.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-10-17T01:30:25.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-10-17T01:30:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-17T01:30:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-17T01:30:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.413 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2023-10-17T01:30:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2023-10-17T01:30:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-17T01:30:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-10-17T01:30:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-10-17T01:30:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2023-10-17T01:30:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2023-10-17T01:30:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2023-10-17T01:30:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2023-10-17T01:30:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2023-10-17T01:30:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2023-10-17T01:30:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2023-10-17T01:30:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-17T01:30:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-17T01:30:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2023-10-17T01:30:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2023-10-17T01:30:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-17T01:30:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-17T01:30:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2023-10-17T01:30:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2023-10-17T01:30:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-17T01:30:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2023-10-17T01:30:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2023-10-17T01:30:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2023-10-17T01:30:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-10-17T01:30:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-17T01:30:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2023-10-17T01:30:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2023-10-17T01:30:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2023-10-17T01:30:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-17T01:30:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2023-10-17T01:30:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2023-10-17T01:30:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2023-10-17T01:30:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-17T01:30:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2023-10-17T01:30:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2023-10-17T01:30:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-17T01:30:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-17T01:30:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2023-10-17T01:30:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2023-10-17T01:30:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-17T01:30:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2023-10-17T01:30:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2023-10-17T01:30:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2023-10-17T01:30:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-17T01:30:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2023-10-17T01:30:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2023-10-17T01:30:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-17T01:30:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2023-10-17T01:30:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2023-10-17T01:30:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-17T01:30:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.464 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2023-10-17T01:30:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2023-10-17T01:30:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-17T01:30:25.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2023-10-17T01:30:25.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2023-10-17T01:30:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-17T01:30:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2023-10-17T01:30:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2023-10-17T01:30:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph:v18", 2023-10-17T01:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2023-10-17T01:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2023-10-17T01:30:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2023-10-17T01:30:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-17T01:30:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2023-10-17T01:30:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2023-10-17T01:30:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2023-10-17T01:30:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-17T01:30:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:25.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.479 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2023-10-17T01:30:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2023-10-17T01:30:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-17T01:30:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2023-10-17T01:30:25.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.482 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2023-10-17T01:30:25.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2023-10-17T01:30:25.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-17T01:30:25.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2023-10-17T01:30:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2023-10-17T01:30:25.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2023-10-17T01:30:25.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-17T01:30:25.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.486 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2023-10-17T01:30:25.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.487 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2023-10-17T01:30:25.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2023-10-17T01:30:25.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-17T01:30:25.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2023-10-17T01:30:25.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.490 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2023-10-17T01:30:25.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2023-10-17T01:30:25.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-17T01:30:25.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2023-10-17T01:30:25.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.492 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2023-10-17T01:30:25.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2023-10-17T01:30:25.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-17T01:30:25.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2023-10-17T01:30:25.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.495 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2023-10-17T01:30:25.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2023-10-17T01:30:25.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-17T01:30:25.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:25.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.497 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2023-10-17T01:30:25.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2023-10-17T01:30:25.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-10-17T01:30:25.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2023-10-17T01:30:25.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.500 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2023-10-17T01:30:25.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2023-10-17T01:30:25.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-17T01:30:25.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:25.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.503 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2023-10-17T01:30:25.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2023-10-17T01:30:25.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-17T01:30:25.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2023-10-17T01:30:25.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.505 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2023-10-17T01:30:25.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2023-10-17T01:30:25.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-17T01:30:25.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2023-10-17T01:30:25.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2023-10-17T01:30:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2023-10-17T01:30:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2023-10-17T01:30:25.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.510 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2023-10-17T01:30:25.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2023-10-17T01:30:25.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:25.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-17T01:30:25.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2023-10-17T01:30:25.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2023-10-17T01:30:25.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2023-10-17T01:30:25.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-17T01:30:25.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2023-10-17T01:30:25.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2023-10-17T01:30:25.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2023-10-17T01:30:25.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-17T01:30:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2023-10-17T01:30:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2023-10-17T01:30:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2023-10-17T01:30:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2023-10-17T01:30:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2023-10-17T01:30:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2023-10-17T01:30:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-17T01:30:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.526 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2023-10-17T01:30:25.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2023-10-17T01:30:25.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:25.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2023-10-17T01:30:25.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.529 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2023-10-17T01:30:25.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2023-10-17T01:30:25.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:25.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2023-10-17T01:30:25.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.531 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2023-10-17T01:30:25.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2023-10-17T01:30:25.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-17T01:30:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.534 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.536 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.536 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.536 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.536 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.536 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.537 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.537 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.538 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2023-10-17T01:30:25.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2023-10-17T01:30:25.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-17T01:30:25.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.541 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-17T01:30:25.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.543 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.545 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.545 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.545 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.546 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.546 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.546 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.546 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.547 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.550 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-17T01:30:25.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-17T01:30:25.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-17T01:30:25.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.553 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-17T01:30:25.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-17T01:30:25.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2023-10-17T01:30:25.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-17T01:30:25.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2023-10-17T01:30:25.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2023-10-17T01:30:25.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.557 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.557 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2023-10-17T01:30:25.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2023-10-17T01:30:25.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2023-10-17T01:30:25.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-17T01:30:25.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.561 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2023-10-17T01:30:25.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2023-10-17T01:30:25.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2023-10-17T01:30:25.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.564 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-10-17T01:30:25.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-10-17T01:30:25.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2023-10-17T01:30:25.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.565 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2023-10-17T01:30:25.565 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2023-10-17T01:30:25.566 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2023-10-17T01:30:25.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.567 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2023-10-17T01:30:25.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2023-10-17T01:30:25.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-17T01:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-17T01:30:25.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.569 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2023-10-17T01:30:25.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2023-10-17T01:30:25.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-17T01:30:25.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.572 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2023-10-17T01:30:25.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2023-10-17T01:30:25.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-17T01:30:25.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.575 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2023-10-17T01:30:25.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2023-10-17T01:30:25.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-17T01:30:25.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.577 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2023-10-17T01:30:25.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2023-10-17T01:30:25.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-17T01:30:25.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.580 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2023-10-17T01:30:25.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2023-10-17T01:30:25.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-17T01:30:25.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2023-10-17T01:30:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2023-10-17T01:30:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2023-10-17T01:30:25.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2023-10-17T01:30:25.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2023-10-17T01:30:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2023-10-17T01:30:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-17T01:30:25.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.588 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2023-10-17T01:30:25.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2023-10-17T01:30:25.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-17T01:30:25.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.590 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2023-10-17T01:30:25.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2023-10-17T01:30:25.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-17T01:30:25.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2023-10-17T01:30:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2023-10-17T01:30:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2023-10-17T01:30:25.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.596 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2023-10-17T01:30:25.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2023-10-17T01:30:25.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-17T01:30:25.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.598 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2023-10-17T01:30:25.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2023-10-17T01:30:25.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-17T01:30:25.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.601 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.601 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2023-10-17T01:30:25.602 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.602 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.602 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.602 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.604 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.604 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.604 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.605 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.605 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.605 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.605 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.606 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.609 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.611 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.611 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.611 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.611 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.611 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.611 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.612 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.613 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2023-10-17T01:30:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2023-10-17T01:30:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2023-10-17T01:30:25.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-17T01:30:25.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.618 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2023-10-17T01:30:25.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2023-10-17T01:30:25.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2023-10-17T01:30:25.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-17T01:30:25.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.621 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.621 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2023-10-17T01:30:25.622 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.622 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.622 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.622 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-17T01:30:25.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-17T01:30:25.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.625 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2023-10-17T01:30:25.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-17T01:30:25.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.628 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-17T01:30:25.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-17T01:30:25.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.630 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2023-10-17T01:30:25.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2023-10-17T01:30:25.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.633 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-17T01:30:25.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-17T01:30:25.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.635 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2023-10-17T01:30:25.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-17T01:30:25.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-17T01:30:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-17T01:30:25.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-17T01:30:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-17T01:30:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.643 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-17T01:30:25.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-17T01:30:25.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.646 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-17T01:30:25.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-17T01:30:25.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.649 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-17T01:30:25.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-17T01:30:25.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.651 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-17T01:30:25.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-17T01:30:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.654 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-17T01:30:25.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-17T01:30:25.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.656 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-17T01:30:25.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-17T01:30:25.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.659 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2023-10-17T01:30:25.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-17T01:30:25.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:25.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.662 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-17T01:30:25.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-17T01:30:25.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.664 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2023-10-17T01:30:25.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2023-10-17T01:30:25.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.667 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2023-10-17T01:30:25.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-17T01:30:25.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.669 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-17T01:30:25.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-17T01:30:25.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.672 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-17T01:30:25.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-17T01:30:25.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-17T01:30:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-17T01:30:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.679 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-17T01:30:25.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-17T01:30:25.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.681 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2023-10-17T01:30:25.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-17T01:30:25.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.684 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-17T01:30:25.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-17T01:30:25.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.687 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-17T01:30:25.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-17T01:30:25.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.689 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-17T01:30:25.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-17T01:30:25.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.692 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-17T01:30:25.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-17T01:30:25.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.694 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-17T01:30:25.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-17T01:30:25.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.697 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-17T01:30:25.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-17T01:30:25.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.700 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-17T01:30:25.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-17T01:30:25.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.702 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-17T01:30:25.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-17T01:30:25.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.704 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.704 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2023-10-17T01:30:25.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-17T01:30:25.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.707 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-17T01:30:25.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-17T01:30:25.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-17T01:30:25.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.709 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-17T01:30:25.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-17T01:30:25.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.712 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-17T01:30:25.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-17T01:30:25.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2023-10-17T01:30:25.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.715 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-17T01:30:25.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-17T01:30:25.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2023-10-17T01:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.717 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2023-10-17T01:30:25.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-17T01:30:25.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.720 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-17T01:30:25.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-17T01:30:25.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.723 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2023-10-17T01:30:25.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-17T01:30:25.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.725 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2023-10-17T01:30:25.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-17T01:30:25.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2023-10-17T01:30:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-17T01:30:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.730 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-17T01:30:25.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-17T01:30:25.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:25.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.733 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-17T01:30:25.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-17T01:30:25.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.735 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-17T01:30:25.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-17T01:30:25.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:25.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.738 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2023-10-17T01:30:25.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2023-10-17T01:30:25.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.741 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2023-10-17T01:30:25.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2023-10-17T01:30:25.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2023-10-17T01:30:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2023-10-17T01:30:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2023-10-17T01:30:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2023-10-17T01:30:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2023-10-17T01:30:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2023-10-17T01:30:25.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.749 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-10-17T01:30:25.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-10-17T01:30:25.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.753 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.753 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.753 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.754 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.754 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.754 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.754 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.755 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.758 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2023-10-17T01:30:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2023-10-17T01:30:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2023-10-17T01:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2023-10-17T01:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2023-10-17T01:30:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.770 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.770 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:25.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:25.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-10-17T01:30:25.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.773 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:25.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:25.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2023-10-17T01:30:25.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.775 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-10-17T01:30:25.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-10-17T01:30:25.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.778 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2023-10-17T01:30:25.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2023-10-17T01:30:25.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2023-10-17T01:30:25.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.781 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-10-17T01:30:25.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-10-17T01:30:25.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2023-10-17T01:30:25.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.782 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.783 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2023-10-17T01:30:25.783 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.784 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-10-17T01:30:25.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-10-17T01:30:25.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-10-17T01:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-10-17T01:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-10-17T01:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.787 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2023-10-17T01:30:25.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2023-10-17T01:30:25.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.789 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.789 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-10-17T01:30:25.790 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.790 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.790 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2023-10-17T01:30:25.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2023-10-17T01:30:25.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2023-10-17T01:30:25.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.793 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.795 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.795 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.795 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.795 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.796 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.796 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.796 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.800 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.801 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.802 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.802 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.802 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.802 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.802 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.803 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.804 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2023-10-17T01:30:25.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2023-10-17T01:30:25.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2023-10-17T01:30:25.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-17T01:30:25.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.809 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2023-10-17T01:30:25.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2023-10-17T01:30:25.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2023-10-17T01:30:25.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-17T01:30:25.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.812 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2023-10-17T01:30:25.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2023-10-17T01:30:25.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2023-10-17T01:30:25.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2023-10-17T01:30:25.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.814 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2023-10-17T01:30:25.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2023-10-17T01:30:25.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-10-17T01:30:25.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-17T01:30:25.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.817 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2023-10-17T01:30:25.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2023-10-17T01:30:25.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:25.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-17T01:30:25.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.820 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:25.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:25.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:25.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2023-10-17T01:30:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.822 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2023-10-17T01:30:25.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2023-10-17T01:30:25.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2023-10-17T01:30:25.823 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-17T01:30:25.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2023-10-17T01:30:25.826 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.826 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.826 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.828 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.828 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.828 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.828 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.829 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.829 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.829 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.830 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.833 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.836 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.837 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.840 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2023-10-17T01:30:25.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2023-10-17T01:30:25.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-10-17T01:30:25.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.842 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2023-10-17T01:30:25.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2023-10-17T01:30:25.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2023-10-17T01:30:25.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.845 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:25.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:25.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:25.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2023-10-17T01:30:25.848 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2023-10-17T01:30:25.849 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2023-10-17T01:30:25.849 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.849 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2023-10-17T01:30:25.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2023-10-17T01:30:25.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2023-10-17T01:30:25.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-17T01:30:25.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.852 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2023-10-17T01:30:25.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2023-10-17T01:30:25.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2023-10-17T01:30:25.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-17T01:30:25.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2023-10-17T01:30:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2023-10-17T01:30:25.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2023-10-17T01:30:25.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.857 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:25.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:25.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:25.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2023-10-17T01:30:25.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2023-10-17T01:30:25.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-17T01:30:25.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.861 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.861 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.861 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.862 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.862 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.862 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.862 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.863 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.866 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.868 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.868 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.868 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.868 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.868 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.869 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.870 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.873 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2023-10-17T01:30:25.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2023-10-17T01:30:25.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.874 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2023-10-17T01:30:25.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.876 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2023-10-17T01:30:25.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2023-10-17T01:30:25.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2023-10-17T01:30:25.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2023-10-17T01:30:25.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.878 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-10-17T01:30:25.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-10-17T01:30:25.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2023-10-17T01:30:25.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-17T01:30:25.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.882 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2023-10-17T01:30:25.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2023-10-17T01:30:25.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:25.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:25.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2023-10-17T01:30:25.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-17T01:30:25.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.885 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2023-10-17T01:30:25.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2023-10-17T01:30:25.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2023-10-17T01:30:25.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.888 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2023-10-17T01:30:25.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2023-10-17T01:30:25.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2023-10-17T01:30:25.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-17T01:30:25.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2023-10-17T01:30:25.891 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.891 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.891 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.895 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.898 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.900 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.900 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.900 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.900 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.900 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.900 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.901 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.902 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2023-10-17T01:30:25.905 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.905 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.905 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.906 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.907 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.908 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.908 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.908 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.908 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.908 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.908 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.910 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.912 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.914 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.914 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.914 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.915 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.915 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.915 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.916 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2023-10-17T01:30:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2023-10-17T01:30:25.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2023-10-17T01:30:25.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2023-10-17T01:30:25.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-17T01:30:25.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.923 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2023-10-17T01:30:25.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2023-10-17T01:30:25.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:25.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:25.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2023-10-17T01:30:25.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-17T01:30:25.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.926 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.927 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.927 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.928 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.928 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.928 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.928 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.928 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.934 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.934 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.934 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.935 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.935 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.935 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.935 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2023-10-17T01:30:25.940 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.940 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.940 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.940 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2023-10-17T01:30:25.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2023-10-17T01:30:25.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2023-10-17T01:30:25.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2023-10-17T01:30:25.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-17T01:30:25.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.945 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.945 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.945 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.945 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.946 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.946 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.946 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.947 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.950 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.951 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.952 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.952 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.952 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.952 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.952 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.952 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.954 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2023-10-17T01:30:25.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2023-10-17T01:30:25.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2023-10-17T01:30:25.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-17T01:30:25.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.959 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2023-10-17T01:30:25.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2023-10-17T01:30:25.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2023-10-17T01:30:25.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2023-10-17T01:30:25.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.961 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2023-10-17T01:30:25.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2023-10-17T01:30:25.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:25.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2023-10-17T01:30:25.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2023-10-17T01:30:25.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.964 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2023-10-17T01:30:25.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2023-10-17T01:30:25.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2023-10-17T01:30:25.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.967 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2023-10-17T01:30:25.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2023-10-17T01:30:25.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2023-10-17T01:30:25.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-17T01:30:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-17T01:30:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2023-10-17T01:30:25.970 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.970 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.971 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.971 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.972 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.973 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.973 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.973 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.973 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.973 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.974 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.975 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.977 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.979 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.979 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.979 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.980 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.980 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.980 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.980 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.981 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:25.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2023-10-17T01:30:25.985 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:25.985 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:25.985 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:25.985 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:25.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:25.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:25.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.987 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.987 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.987 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.988 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.988 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.988 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.988 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.990 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:25.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:25.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.992 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:25.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:25.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:25.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:25.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:25.994 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:25.994 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:25.994 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:25.995 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:25.995 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:25.995 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:25.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:25.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.996 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:25.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:25.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:25.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:25.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:25.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:25.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:25.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:25.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:25.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:25.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2023-10-17T01:30:26.000 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.000 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.000 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.000 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.002 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.002 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.002 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.003 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.003 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.003 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.003 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.007 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.009 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.009 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.009 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-10-17T01:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2023-10-17T01:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2023-10-17T01:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-17T01:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2023-10-17T01:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-10-17T01:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-10-17T01:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2023-10-17T01:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2023-10-17T01:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2023-10-17T01:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2023-10-17T01:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2023-10-17T01:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2023-10-17T01:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "noautoscale": { 2023-10-17T01:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "noautoscale", 2023-10-17T01:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "global autoscale flag", 2023-10-17T01:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-10-17T01:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2023-10-17T01:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2023-10-17T01:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-10-17T01:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2023-10-17T01:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2023-10-17T01:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-17T01:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2023-10-17T01:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2023-10-17T01:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2023-10-17T01:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-17T01:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-10-17T01:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-10-17T01:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2023-10-17T01:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2023-10-17T01:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2023-10-17T01:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2023-10-17T01:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2023-10-17T01:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2023-10-17T01:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2023-10-17T01:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2023-10-17T01:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2023-10-17T01:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2023-10-17T01:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-17T01:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-17T01:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.131 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.133 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.134 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2023-10-17T01:30:26.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2023-10-17T01:30:26.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.137 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-17T01:30:26.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-17T01:30:26.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2023-10-17T01:30:26.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.140 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2023-10-17T01:30:26.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2023-10-17T01:30:26.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2023-10-17T01:30:26.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.143 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:26.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:26.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-10-17T01:30:26.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-17T01:30:26.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.145 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:26.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:26.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2023-10-17T01:30:26.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-17T01:30:26.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.148 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2023-10-17T01:30:26.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2023-10-17T01:30:26.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2023-10-17T01:30:26.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.151 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-10-17T01:30:26.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-10-17T01:30:26.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2023-10-17T01:30:26.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2023-10-17T01:30:26.153 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2023-10-17T01:30:26.153 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.154 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-10-17T01:30:26.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-10-17T01:30:26.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-10-17T01:30:26.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-10-17T01:30:26.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-10-17T01:30:26.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.156 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-10-17T01:30:26.157 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.160 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.160 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.160 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.160 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.161 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.161 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.161 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.162 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.165 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.167 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.167 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.167 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.167 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.168 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.168 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.168 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.169 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2023-10-17T01:30:26.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2023-10-17T01:30:26.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.172 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.175 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2023-10-17T01:30:26.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2023-10-17T01:30:26.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.177 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2023-10-17T01:30:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2023-10-17T01:30:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2023-10-17T01:30:26.181 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.181 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.181 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2023-10-17T01:30:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2023-10-17T01:30:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.184 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-10-17T01:30:26.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-10-17T01:30:26.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.187 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.189 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.189 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.189 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.189 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.189 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.191 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.194 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.195 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.196 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.196 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.196 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.196 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.196 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.197 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.198 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.200 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:26.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:26.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.203 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:26.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:26.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.206 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2023-10-17T01:30:26.207 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.207 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.207 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.207 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.209 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.209 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.209 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.210 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.210 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.210 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.210 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.211 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.214 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.214 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.216 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.216 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.216 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.217 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.217 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.217 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.217 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.219 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2023-10-17T01:30:26.222 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.223 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.223 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.223 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2023-10-17T01:30:26.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2023-10-17T01:30:26.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2023-10-17T01:30:26.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-17T01:30:26.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.227 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.227 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.228 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.228 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.228 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.228 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.232 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.234 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.234 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.234 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.235 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.235 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.235 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.235 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.236 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.239 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2023-10-17T01:30:26.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2023-10-17T01:30:26.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2023-10-17T01:30:26.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.240 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-17T01:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2023-10-17T01:30:26.242 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.243 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.243 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.243 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.245 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.245 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.245 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.245 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.246 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.246 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.246 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.247 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.250 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.251 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.252 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.252 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.252 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.252 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.252 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.253 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.254 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.257 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2023-10-17T01:30:26.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2023-10-17T01:30:26.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.260 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2023-10-17T01:30:26.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2023-10-17T01:30:26.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2023-10-17T01:30:26.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.263 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2023-10-17T01:30:26.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2023-10-17T01:30:26.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.265 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2023-10-17T01:30:26.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2023-10-17T01:30:26.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.268 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2023-10-17T01:30:26.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2023-10-17T01:30:26.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.271 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2023-10-17T01:30:26.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2023-10-17T01:30:26.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.273 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2023-10-17T01:30:26.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2023-10-17T01:30:26.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.276 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2023-10-17T01:30:26.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2023-10-17T01:30:26.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.278 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2023-10-17T01:30:26.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2023-10-17T01:30:26.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:26.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2023-10-17T01:30:26.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.281 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2023-10-17T01:30:26.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2023-10-17T01:30:26.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.284 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2023-10-17T01:30:26.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2023-10-17T01:30:26.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.285 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.287 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2023-10-17T01:30:26.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2023-10-17T01:30:26.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-10-17T01:30:26.290 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.290 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.291 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.291 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2023-10-17T01:30:26.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2023-10-17T01:30:26.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2023-10-17T01:30:26.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.294 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2023-10-17T01:30:26.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2023-10-17T01:30:26.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2023-10-17T01:30:26.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.296 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.298 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.298 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.299 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.299 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.299 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.299 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.299 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.300 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.303 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.305 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.305 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.305 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.305 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.306 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.306 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.307 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2023-10-17T01:30:26.311 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.311 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.311 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.312 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.313 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.313 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.314 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.314 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.314 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.314 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.314 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.316 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.318 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.320 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.320 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.320 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.320 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.321 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.321 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.321 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.322 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2023-10-17T01:30:26.326 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.326 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.326 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.326 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.328 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.328 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.328 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.329 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.329 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.329 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.329 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.340 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.340 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.340 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.340 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.342 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2023-10-17T01:30:26.346 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.346 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.346 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.346 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2023-10-17T01:30:26.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2023-10-17T01:30:26.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-17T01:30:26.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.349 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:26.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:26.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2023-10-17T01:30:26.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.352 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.353 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.354 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.354 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.354 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.354 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.361 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.361 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.361 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.361 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.361 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.363 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2023-10-17T01:30:26.366 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.366 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.367 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.367 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2023-10-17T01:30:26.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2023-10-17T01:30:26.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-17T01:30:26.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.369 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2023-10-17T01:30:26.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2023-10-17T01:30:26.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-17T01:30:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2023-10-17T01:30:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2023-10-17T01:30:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-17T01:30:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2023-10-17T01:30:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2023-10-17T01:30:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-17T01:30:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2023-10-17T01:30:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2023-10-17T01:30:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-17T01:30:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2023-10-17T01:30:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2023-10-17T01:30:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2023-10-17T01:30:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2023-10-17T01:30:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2023-10-17T01:30:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2023-10-17T01:30:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-17T01:30:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-10-17T01:30:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-10-17T01:30:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2023-10-17T01:30:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2023-10-17T01:30:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2023-10-17T01:30:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2023-10-17T01:30:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2023-10-17T01:30:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2023-10-17T01:30:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2023-10-17T01:30:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2023-10-17T01:30:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2023-10-17T01:30:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2023-10-17T01:30:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2023-10-17T01:30:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2023-10-17T01:30:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2023-10-17T01:30:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2023-10-17T01:30:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2023-10-17T01:30:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-17T01:30:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2023-10-17T01:30:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-10-17T01:30:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2023-10-17T01:30:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2023-10-17T01:30:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2023-10-17T01:30:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-17T01:30:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2023-10-17T01:30:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2023-10-17T01:30:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-17T01:30:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2023-10-17T01:30:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2023-10-17T01:30:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2023-10-17T01:30:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2023-10-17T01:30:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2023-10-17T01:30:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2023-10-17T01:30:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2023-10-17T01:30:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2023-10-17T01:30:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2023-10-17T01:30:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2023-10-17T01:30:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2023-10-17T01:30:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2023-10-17T01:30:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2023-10-17T01:30:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-17T01:30:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2023-10-17T01:30:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-10-17T01:30:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2023-10-17T01:30:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2023-10-17T01:30:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2023-10-17T01:30:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "restful" 2023-10-17T01:30:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2023-10-17T01:30:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2023-10-17T01:30:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2023-10-17T01:30:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2023-10-17T01:30:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2023-10-17T01:30:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2023-10-17T01:30:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2023-10-17T01:30:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2023-10-17T01:30:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2023-10-17T01:30:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2023-10-17T01:30:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2023-10-17T01:30:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2023-10-17T01:30:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2023-10-17T01:30:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2023-10-17T01:30:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2023-10-17T01:30:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2023-10-17T01:30:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2023-10-17T01:30:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2023-10-17T01:30:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2023-10-17T01:30:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2023-10-17T01:30:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2023-10-17T01:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2023-10-17T01:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2023-10-17T01:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2023-10-17T01:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2023-10-17T01:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-17T01:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2023-10-17T01:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2023-10-17T01:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2023-10-17T01:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2023-10-17T01:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2023-10-17T01:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2023-10-17T01:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2023-10-17T01:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2023-10-17T01:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2023-10-17T01:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2023-10-17T01:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2023-10-17T01:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-17T01:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2023-10-17T01:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2023-10-17T01:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-10-17T01:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-10-17T01:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-17T01:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-17T01:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2023-10-17T01:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2023-10-17T01:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2023-10-17T01:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2023-10-17T01:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-17T01:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2023-10-17T01:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2023-10-17T01:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2023-10-17T01:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-17T01:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-17T01:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2023-10-17T01:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2023-10-17T01:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2023-10-17T01:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2023-10-17T01:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2023-10-17T01:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2023-10-17T01:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-17T01:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2023-10-17T01:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2023-10-17T01:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2023-10-17T01:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2023-10-17T01:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-17T01:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2023-10-17T01:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2023-10-17T01:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2023-10-17T01:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2023-10-17T01:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-17T01:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-17T01:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2023-10-17T01:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2023-10-17T01:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-17T01:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-10-17T01:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-10-17T01:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2023-10-17T01:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2023-10-17T01:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2023-10-17T01:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap" 2023-10-17T01:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2023-10-17T01:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2023-10-17T01:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2023-10-17T01:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-17T01:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-17T01:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2023-10-17T01:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2023-10-17T01:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-17T01:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-17T01:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2023-10-17T01:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2023-10-17T01:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-17T01:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2023-10-17T01:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2023-10-17T01:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2023-10-17T01:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-10-17T01:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-17T01:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2023-10-17T01:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2023-10-17T01:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2023-10-17T01:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-17T01:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2023-10-17T01:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2023-10-17T01:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2023-10-17T01:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-17T01:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2023-10-17T01:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2023-10-17T01:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-17T01:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-17T01:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2023-10-17T01:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2023-10-17T01:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-17T01:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2023-10-17T01:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2023-10-17T01:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2023-10-17T01:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-17T01:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2023-10-17T01:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2023-10-17T01:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-17T01:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2023-10-17T01:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2023-10-17T01:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-17T01:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2023-10-17T01:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2023-10-17T01:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-17T01:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2023-10-17T01:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2023-10-17T01:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-17T01:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2023-10-17T01:30:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2023-10-17T01:30:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph:v18", 2023-10-17T01:30:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2023-10-17T01:30:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2023-10-17T01:30:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2023-10-17T01:30:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-17T01:30:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2023-10-17T01:30:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2023-10-17T01:30:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2023-10-17T01:30:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-17T01:30:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2023-10-17T01:30:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2023-10-17T01:30:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-17T01:30:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2023-10-17T01:30:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2023-10-17T01:30:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2023-10-17T01:30:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-17T01:30:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2023-10-17T01:30:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2023-10-17T01:30:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2023-10-17T01:30:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-17T01:30:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2023-10-17T01:30:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2023-10-17T01:30:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2023-10-17T01:30:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-17T01:30:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2023-10-17T01:30:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2023-10-17T01:30:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2023-10-17T01:30:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-17T01:30:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2023-10-17T01:30:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2023-10-17T01:30:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2023-10-17T01:30:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-17T01:30:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2023-10-17T01:30:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2023-10-17T01:30:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2023-10-17T01:30:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-17T01:30:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2023-10-17T01:30:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2023-10-17T01:30:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-10-17T01:30:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2023-10-17T01:30:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2023-10-17T01:30:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2023-10-17T01:30:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-17T01:30:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2023-10-17T01:30:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2023-10-17T01:30:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2023-10-17T01:30:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-17T01:30:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2023-10-17T01:30:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2023-10-17T01:30:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2023-10-17T01:30:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-17T01:30:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2023-10-17T01:30:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2023-10-17T01:30:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2023-10-17T01:30:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2023-10-17T01:30:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2023-10-17T01:30:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2023-10-17T01:30:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-17T01:30:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2023-10-17T01:30:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2023-10-17T01:30:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2023-10-17T01:30:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-17T01:30:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2023-10-17T01:30:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2023-10-17T01:30:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2023-10-17T01:30:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-17T01:30:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2023-10-17T01:30:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2023-10-17T01:30:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2023-10-17T01:30:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2023-10-17T01:30:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2023-10-17T01:30:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2023-10-17T01:30:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-17T01:30:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2023-10-17T01:30:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2023-10-17T01:30:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2023-10-17T01:30:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2023-10-17T01:30:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2023-10-17T01:30:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2023-10-17T01:30:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2023-10-17T01:30:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2023-10-17T01:30:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-17T01:30:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2023-10-17T01:30:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2023-10-17T01:30:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-17T01:30:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-17T01:30:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-17T01:30:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-17T01:30:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-17T01:30:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-17T01:30:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-17T01:30:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2023-10-17T01:30:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-17T01:30:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2023-10-17T01:30:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2023-10-17T01:30:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2023-10-17T01:30:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2023-10-17T01:30:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2023-10-17T01:30:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-17T01:30:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2023-10-17T01:30:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2023-10-17T01:30:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2023-10-17T01:30:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2023-10-17T01:30:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2023-10-17T01:30:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2023-10-17T01:30:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2023-10-17T01:30:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2023-10-17T01:30:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2023-10-17T01:30:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2023-10-17T01:30:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2023-10-17T01:30:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-17T01:30:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-17T01:30:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2023-10-17T01:30:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2023-10-17T01:30:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-17T01:30:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2023-10-17T01:30:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2023-10-17T01:30:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-17T01:30:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2023-10-17T01:30:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2023-10-17T01:30:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-17T01:30:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2023-10-17T01:30:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2023-10-17T01:30:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-17T01:30:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2023-10-17T01:30:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2023-10-17T01:30:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-17T01:30:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2023-10-17T01:30:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2023-10-17T01:30:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2023-10-17T01:30:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2023-10-17T01:30:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2023-10-17T01:30:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2023-10-17T01:30:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-17T01:30:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2023-10-17T01:30:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2023-10-17T01:30:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-17T01:30:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2023-10-17T01:30:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2023-10-17T01:30:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-17T01:30:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2023-10-17T01:30:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2023-10-17T01:30:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2023-10-17T01:30:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2023-10-17T01:30:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2023-10-17T01:30:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-17T01:30:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2023-10-17T01:30:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2023-10-17T01:30:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-17T01:30:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2023-10-17T01:30:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2023-10-17T01:30:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2023-10-17T01:30:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2023-10-17T01:30:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-17T01:30:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2023-10-17T01:30:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2023-10-17T01:30:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2023-10-17T01:30:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-17T01:30:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2023-10-17T01:30:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-17T01:30:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-17T01:30:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2023-10-17T01:30:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-17T01:30:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-17T01:30:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-17T01:30:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2023-10-17T01:30:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2023-10-17T01:30:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-17T01:30:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-17T01:30:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2023-10-17T01:30:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-17T01:30:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-17T01:30:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-17T01:30:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-17T01:30:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-17T01:30:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-17T01:30:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-17T01:30:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-17T01:30:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-17T01:30:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-17T01:30:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-17T01:30:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-17T01:30:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-17T01:30:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-17T01:30:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-17T01:30:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-17T01:30:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-17T01:30:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2023-10-17T01:30:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-17T01:30:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-17T01:30:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-17T01:30:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2023-10-17T01:30:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2023-10-17T01:30:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2023-10-17T01:30:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-17T01:30:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-17T01:30:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-17T01:30:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-17T01:30:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-17T01:30:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-17T01:30:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-17T01:30:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-17T01:30:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-17T01:30:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2023-10-17T01:30:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-17T01:30:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-17T01:30:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-17T01:30:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-17T01:30:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-17T01:30:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-17T01:30:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-17T01:30:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-17T01:30:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-17T01:30:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-17T01:30:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-17T01:30:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-17T01:30:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-17T01:30:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-17T01:30:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-17T01:30:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-17T01:30:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-17T01:30:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2023-10-17T01:30:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-17T01:30:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-17T01:30:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-17T01:30:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-17T01:30:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-17T01:30:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-17T01:30:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-17T01:30:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-17T01:30:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2023-10-17T01:30:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-17T01:30:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-17T01:30:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2023-10-17T01:30:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2023-10-17T01:30:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-17T01:30:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-17T01:30:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-17T01:30:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2023-10-17T01:30:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2023-10-17T01:30:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-17T01:30:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2023-10-17T01:30:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-17T01:30:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2023-10-17T01:30:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-17T01:30:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-17T01:30:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-17T01:30:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-17T01:30:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-17T01:30:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-17T01:30:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-17T01:30:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2023-10-17T01:30:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2023-10-17T01:30:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2023-10-17T01:30:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2023-10-17T01:30:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2023-10-17T01:30:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2023-10-17T01:30:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2023-10-17T01:30:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2023-10-17T01:30:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2023-10-17T01:30:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2023-10-17T01:30:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-10-17T01:30:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-10-17T01:30:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2023-10-17T01:30:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2023-10-17T01:30:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2023-10-17T01:30:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2023-10-17T01:30:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2023-10-17T01:30:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-10-17T01:30:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2023-10-17T01:30:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-10-17T01:30:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-10-17T01:30:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2023-10-17T01:30:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2023-10-17T01:30:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2023-10-17T01:30:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-10-17T01:30:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-10-17T01:30:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2023-10-17T01:30:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2023-10-17T01:30:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-10-17T01:30:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-10-17T01:30:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-10-17T01:30:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-10-17T01:30:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-10-17T01:30:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2023-10-17T01:30:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2023-10-17T01:30:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-10-17T01:30:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2023-10-17T01:30:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2023-10-17T01:30:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2023-10-17T01:30:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2023-10-17T01:30:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2023-10-17T01:30:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2023-10-17T01:30:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-17T01:30:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2023-10-17T01:30:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2023-10-17T01:30:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2023-10-17T01:30:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-17T01:30:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2023-10-17T01:30:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2023-10-17T01:30:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2023-10-17T01:30:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2023-10-17T01:30:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2023-10-17T01:30:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2023-10-17T01:30:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-10-17T01:30:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-17T01:30:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2023-10-17T01:30:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2023-10-17T01:30:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-17T01:30:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2023-10-17T01:30:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2023-10-17T01:30:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2023-10-17T01:30:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2023-10-17T01:30:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-17T01:30:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2023-10-17T01:30:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2023-10-17T01:30:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2023-10-17T01:30:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2023-10-17T01:30:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2023-10-17T01:30:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2023-10-17T01:30:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2023-10-17T01:30:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2023-10-17T01:30:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2023-10-17T01:30:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2023-10-17T01:30:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2023-10-17T01:30:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2023-10-17T01:30:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2023-10-17T01:30:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2023-10-17T01:30:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-17T01:30:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2023-10-17T01:30:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2023-10-17T01:30:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2023-10-17T01:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-17T01:30:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2023-10-17T01:30:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2023-10-17T01:30:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2023-10-17T01:30:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2023-10-17T01:30:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2023-10-17T01:30:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-17T01:30:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2023-10-17T01:30:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2023-10-17T01:30:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2023-10-17T01:30:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2023-10-17T01:30:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2023-10-17T01:30:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2023-10-17T01:30:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2023-10-17T01:30:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2023-10-17T01:30:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2023-10-17T01:30:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2023-10-17T01:30:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-17T01:30:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2023-10-17T01:30:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2023-10-17T01:30:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2023-10-17T01:30:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-17T01:30:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2023-10-17T01:30:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2023-10-17T01:30:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2023-10-17T01:30:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2023-10-17T01:30:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2023-10-17T01:30:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2023-10-17T01:30:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-17T01:30:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2023-10-17T01:30:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2023-10-17T01:30:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2023-10-17T01:30:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2023-10-17T01:30:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2023-10-17T01:30:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2023-10-17T01:30:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-17T01:30:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2023-10-17T01:30:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2023-10-17T01:30:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2023-10-17T01:30:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-17T01:30:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2023-10-17T01:30:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2023-10-17T01:30:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2023-10-17T01:30:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2023-10-17T01:30:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2023-10-17T01:30:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-17T01:30:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2023-10-17T01:30:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2023-10-17T01:30:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2023-10-17T01:30:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-17T01:30:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2023-10-17T01:30:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2023-10-17T01:30:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2023-10-17T01:30:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2023-10-17T01:30:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2023-10-17T01:30:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2023-10-17T01:30:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2023-10-17T01:30:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2023-10-17T01:30:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2023-10-17T01:30:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2023-10-17T01:30:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2023-10-17T01:30:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2023-10-17T01:30:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2023-10-17T01:30:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2023-10-17T01:30:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-17T01:30:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-17T01:30:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2023-10-17T01:30:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2023-10-17T01:30:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2023-10-17T01:30:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-10-17T01:30:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2023-10-17T01:30:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2023-10-17T01:30:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-17T01:30:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2023-10-17T01:30:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2023-10-17T01:30:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2023-10-17T01:30:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2023-10-17T01:30:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2023-10-17T01:30:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2023-10-17T01:30:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2023-10-17T01:30:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2023-10-17T01:30:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2023-10-17T01:30:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "noautoscale": { 2023-10-17T01:30:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "noautoscale", 2023-10-17T01:30:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "global autoscale flag", 2023-10-17T01:30:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-10-17T01:30:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2023-10-17T01:30:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2023-10-17T01:30:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2023-10-17T01:30:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2023-10-17T01:30:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2023-10-17T01:30:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-17T01:30:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2023-10-17T01:30:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2023-10-17T01:30:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2023-10-17T01:30:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-17T01:30:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-10-17T01:30:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-10-17T01:30:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2023-10-17T01:30:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2023-10-17T01:30:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2023-10-17T01:30:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2023-10-17T01:30:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2023-10-17T01:30:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2023-10-17T01:30:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2023-10-17T01:30:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2023-10-17T01:30:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2023-10-17T01:30:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2023-10-17T01:30:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2023-10-17T01:30:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2023-10-17T01:30:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2023-10-17T01:30:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-17T01:30:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-17T01:30:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2023-10-17T01:30:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2023-10-17T01:30:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-17T01:30:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-17T01:30:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2023-10-17T01:30:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2023-10-17T01:30:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2023-10-17T01:30:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2023-10-17T01:30:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2023-10-17T01:30:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-17T01:30:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2023-10-17T01:30:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-17T01:30:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2023-10-17T01:30:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2023-10-17T01:30:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2023-10-17T01:30:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2023-10-17T01:30:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2023-10-17T01:30:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2023-10-17T01:30:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2023-10-17T01:30:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2023-10-17T01:30:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2023-10-17T01:30:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2023-10-17T01:30:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2023-10-17T01:30:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2023-10-17T01:30:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-10-17T01:30:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2023-10-17T01:30:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2023-10-17T01:30:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2023-10-17T01:30:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2023-10-17T01:30:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2023-10-17T01:30:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2023-10-17T01:30:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2023-10-17T01:30:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2023-10-17T01:30:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2023-10-17T01:30:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2023-10-17T01:30:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2023-10-17T01:30:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2023-10-17T01:30:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2023-10-17T01:30:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2023-10-17T01:30:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2023-10-17T01:30:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2023-10-17T01:30:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2023-10-17T01:30:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2023-10-17T01:30:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "drive_group_interval": { 2023-10-17T01:30:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "drive_group_interval", 2023-10-17T01:30:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300.0", 2023-10-17T01:30:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-17T01:30:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2023-10-17T01:30:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2023-10-17T01:30:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2023-10-17T01:30:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-17T01:30:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2023-10-17T01:30:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2023-10-17T01:30:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2023-10-17T01:30:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2023-10-17T01:30:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2023-10-17T01:30:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2023-10-17T01:30:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2023-10-17T01:30:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2023-10-17T01:30:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2023-10-17T01:30:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2023-10-17T01:30:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2023-10-17T01:30:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2023-10-17T01:30:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2023-10-17T01:30:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2023-10-17T01:30:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2023-10-17T01:30:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2023-10-17T01:30:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2023-10-17T01:30:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2023-10-17T01:30:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2023-10-17T01:30:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2023-10-17T01:30:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2023-10-17T01:30:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2023-10-17T01:30:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2023-10-17T01:30:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2023-10-17T01:30:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2023-10-17T01:30:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2023-10-17T01:30:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2023-10-17T01:30:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2023-10-17T01:30:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2023-10-17T01:30:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2023-10-17T01:30:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2023-10-17T01:30:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2023-10-17T01:30:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2023-10-17T01:30:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2023-10-17T01:30:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2023-10-17T01:30:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2023-10-17T01:30:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2023-10-17T01:30:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2023-10-17T01:30:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2023-10-17T01:30:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2023-10-17T01:30:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2023-10-17T01:30:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-17T01:30:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2023-10-17T01:30:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2023-10-17T01:30:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2023-10-17T01:30:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2023-10-17T01:30:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-17T01:30:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2023-10-17T01:30:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2023-10-17T01:30:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-17T01:30:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2023-10-17T01:30:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2023-10-17T01:30:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2023-10-17T01:30:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-17T01:30:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2023-10-17T01:30:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2023-10-17T01:30:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-17T01:30:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2023-10-17T01:30:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2023-10-17T01:30:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-17T01:30:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2023-10-17T01:30:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2023-10-17T01:30:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2023-10-17T01:30:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2023-10-17T01:30:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2023-10-17T01:30:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2023-10-17T01:30:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-17T01:30:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2023-10-17T01:30:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2023-10-17T01:30:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2023-10-17T01:30:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2023-10-17T01:30:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2023-10-17T01:30:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2023-10-17T01:30:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2023-10-17T01:30:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2023-10-17T01:30:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2023-10-17T01:30:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2023-10-17T01:30:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2023-10-17T01:30:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2023-10-17T01:30:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2023-10-17T01:30:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2023-10-17T01:30:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2023-10-17T01:30:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2023-10-17T01:30:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2023-10-17T01:30:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-17T01:30:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2023-10-17T01:30:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-10-17T01:30:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2023-10-17T01:30:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2023-10-17T01:30:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2023-10-17T01:30:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-17T01:30:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2023-10-17T01:30:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2023-10-17T01:30:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2023-10-17T01:30:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-17T01:30:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2023-10-17T01:30:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2023-10-17T01:30:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2023-10-17T01:30:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2023-10-17T01:30:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2023-10-17T01:30:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2023-10-17T01:30:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2023-10-17T01:30:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2023-10-17T01:30:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2023-10-17T01:30:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2023-10-17T01:30:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2023-10-17T01:30:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2023-10-17T01:30:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2023-10-17T01:30:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2023-10-17T01:30:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2023-10-17T01:30:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2023-10-17T01:30:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2023-10-17T01:30:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2023-10-17T01:30:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2023-10-17T01:30:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2023-10-17T01:30:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2023-10-17T01:30:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2023-10-17T01:30:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2023-10-17T01:30:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2023-10-17T01:30:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2023-10-17T01:30:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2023-10-17T01:30:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2023-10-17T01:30:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2023-10-17T01:30:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2023-10-17T01:30:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2023-10-17T01:30:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2023-10-17T01:30:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2023-10-17T01:30:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2023-10-17T01:30:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2023-10-17T01:30:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2023-10-17T01:30:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2023-10-17T01:30:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2023-10-17T01:30:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2023-10-17T01:30:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2023-10-17T01:30:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2023-10-17T01:30:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2023-10-17T01:30:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2023-10-17T01:30:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2023-10-17T01:30:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2023-10-17T01:30:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2023-10-17T01:30:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-17T01:30:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2023-10-17T01:30:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2023-10-17T01:30:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2023-10-17T01:30:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2023-10-17T01:30:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2023-10-17T01:30:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2023-10-17T01:30:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2023-10-17T01:30:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2023-10-17T01:30:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2023-10-17T01:30:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2023-10-17T01:30:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2023-10-17T01:30:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-10-17T01:30:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-10-17T01:30:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-10-17T01:30:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-10-17T01:30:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-10-17T01:30:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-10-17T01:30:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-10-17T01:30:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-10-17T01:30:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-10-17T01:30:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-10-17T01:30:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2023-10-17T01:30:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-10-17T01:30:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-10-17T01:30:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-10-17T01:30:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-10-17T01:30:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-10-17T01:30:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-10-17T01:30:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-10-17T01:30:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-10-17T01:30:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-10-17T01:30:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-10-17T01:30:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2023-10-17T01:30:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-10-17T01:30:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-10-17T01:30:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-10-17T01:30:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-10-17T01:30:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-10-17T01:30:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-10-17T01:30:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-10-17T01:30:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-10-17T01:30:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-10-17T01:30:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-10-17T01:30:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-17T01:30:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2023-10-17T01:30:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2023-10-17T01:30:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2023-10-17T01:30:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2023-10-17T01:30:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2023-10-17T01:30:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2023-10-17T01:30:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2023-10-17T01:30:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2023-10-17T01:30:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2023-10-17T01:30:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2023-10-17T01:30:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2023-10-17T01:30:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 3, 2023-10-17T01:30:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2023-10-17T01:30:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2023-10-17T01:30:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-10-17T01:30:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-10-17T01:30:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-10-17T01:30:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4118900047 2023-10-17T01:30:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2023-10-17T01:30:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-10-17T01:30:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-10-17T01:30:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-10-17T01:30:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4229440561 2023-10-17T01:30:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2023-10-17T01:30:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-10-17T01:30:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-10-17T01:30:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-10-17T01:30:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1235220098 2023-10-17T01:30:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2023-10-17T01:30:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2023-10-17T01:30:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-17T01:30:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2023-10-17T01:30:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2023-10-17T01:30:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 55059091 2023-10-17T01:30:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-17T01:30:27.696 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:30:27.698 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:24 smithi181 ceph-mon[134520]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2023-10-17T01:30:27.698 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:24 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/3073131077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:30:27.699 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:24 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2576697002' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-17T01:30:27.699 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:26 smithi181 ceph-mon[134520]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 MiB/s wr, 1.83k op/s 2023-10-17T01:30:27.699 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:26 smithi181 ceph-mon[134520]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:30:27.699 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:24 smithi060 ceph-mon[134136]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2023-10-17T01:30:27.699 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:24 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/3073131077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:30:27.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:24 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2576697002' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-17T01:30:27.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:26 smithi060 ceph-mon[134136]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 MiB/s wr, 1.83k op/s 2023-10-17T01:30:27.700 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:26 smithi060 ceph-mon[134136]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:30:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32fc0a0b70 msgr2=0x7f32fc0a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:27.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32fc0a0b70 0x7f32fc0a3030 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b580 tx=0x7f32f8005fb0 comp rx=0 tx=0).stop 2023-10-17T01:30:27.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 msgr2=0x7f3310198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:30:27.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f3310198c30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f330000e970 tx=0x7f330000ec80 comp rx=0 tx=0).stop 2023-10-17T01:30:27.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 shutdown_connections 2023-10-17T01:30:27.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f32fc0a0b70 0x7f32fc0a3030 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:27.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3310104b20 0x7f33101a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:27.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3310102f90 0x7f3310198c30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:27.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 --2- 172.21.15.45:0/2576697002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3310104180 0x7f3310199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:30:27.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 >> 172.21.15.45:0/2576697002 conn(0x7f33100fe500 msgr2=0x7f33101006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:30:27.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 shutdown_connections 2023-10-17T01:30:27.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:30:24.211+0000 7f33153ff700 1 -- 172.21.15.45:0/2576697002 wait complete. 2023-10-17T01:30:27.709 DEBUG:tasks.check_counter:Getting stats from a 2023-10-17T01:30:27.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-17T01:30:27.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:24 smithi045 ceph-mon[98474]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2023-10-17T01:30:27.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:24 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/3073131077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:30:27.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:24 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2576697002' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-17T01:30:27.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:26 smithi045 ceph-mon[98474]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 MiB/s wr, 1.83k op/s 2023-10-17T01:30:27.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:26 smithi045 ceph-mon[98474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-17T01:30:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:30:28.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:28 smithi045 ceph-mon[98474]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2023-10-17T01:30:28.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:28 smithi060 ceph-mon[134136]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2023-10-17T01:30:28.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:28 smithi181 ceph-mon[134520]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2023-10-17T01:30:30.049 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:30:30.050 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:30.050 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2252, 2023-10-17T01:30:30.050 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2252, 2023-10-17T01:30:30.050 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1761858, 2023-10-17T01:30:30.051 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 699577, 2023-10-17T01:30:30.051 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2023-10-17T01:30:30.051 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2023-10-17T01:30:30.051 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.323140049, 2023-10-17T01:30:30.051 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.084907432, 2023-10-17T01:30:30.052 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.073037901, 2023-10-17T01:30:30.052 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.128953484, 2023-10-17T01:30:30.052 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:30.052 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2251, 2023-10-17T01:30:30.052 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.352364528, 2023-10-17T01:30:30.053 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000156536 2023-10-17T01:30:30.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.053 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:30.053 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.053 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.054 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.054 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-17T01:30:30.054 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2023-10-17T01:30:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1256, 2023-10-17T01:30:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1233, 2023-10-17T01:30:30.056 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 859206, 2023-10-17T01:30:30.056 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 456093, 2023-10-17T01:30:30.056 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2023-10-17T01:30:30.056 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:30.056 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.221980287, 2023-10-17T01:30:30.057 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.067153466, 2023-10-17T01:30:30.057 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.054496922, 2023-10-17T01:30:30.057 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.075202962, 2023-10-17T01:30:30.057 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:30.057 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1232, 2023-10-17T01:30:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.209972850, 2023-10-17T01:30:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000170432 2023-10-17T01:30:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.058 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 626576, 2023-10-17T01:30:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 162752 2023-10-17T01:30:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3, 2023-10-17T01:30:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 217, 2023-10-17T01:30:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 588, 2023-10-17T01:30:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 528823, 2023-10-17T01:30:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2023-10-17T01:30:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.030841402, 2023-10-17T01:30:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.024675179, 2023-10-17T01:30:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002970361, 2023-10-17T01:30:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000029594, 2023-10-17T01:30:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 215, 2023-10-17T01:30:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.044963561, 2023-10-17T01:30:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000209132 2023-10-17T01:30:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-17T01:30:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 528512 2023-10-17T01:30:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-10-17T01:30:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2023-10-17T01:30:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-10-17T01:30:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2023-10-17T01:30:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2023-10-17T01:30:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2023-10-17T01:30:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5236, 2023-10-17T01:30:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.253995428, 2023-10-17T01:30:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000048509 2023-10-17T01:30:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2023-10-17T01:30:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2023-10-17T01:30:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2023-10-17T01:30:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 0, 2023-10-17T01:30:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 0, 2023-10-17T01:30:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2023-10-17T01:30:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2023-10-17T01:30:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2023-10-17T01:30:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 0, 2023-10-17T01:30:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2023-10-17T01:30:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2023-10-17T01:30:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 227, 2023-10-17T01:30:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 6, 2023-10-17T01:30:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 206, 2023-10-17T01:30:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 15, 2023-10-17T01:30:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 218, 2023-10-17T01:30:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2023-10-17T01:30:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2023-10-17T01:30:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2023-10-17T01:30:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 8, 2023-10-17T01:30:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 0, 2023-10-17T01:30:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 0, 2023-10-17T01:30:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2023-10-17T01:30:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2023-10-17T01:30:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2023-10-17T01:30:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2023-10-17T01:30:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2023-10-17T01:30:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2023-10-17T01:30:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2023-10-17T01:30:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2023-10-17T01:30:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2023-10-17T01:30:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2023-10-17T01:30:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2023-10-17T01:30:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2023-10-17T01:30:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2023-10-17T01:30:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2023-10-17T01:30:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:30:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:30:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2023-10-17T01:30:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:30:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 0, 2023-10-17T01:30:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 0, 2023-10-17T01:30:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 0, 2023-10-17T01:30:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 0, 2023-10-17T01:30:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 0, 2023-10-17T01:30:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 0 2023-10-17T01:30:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2023-10-17T01:30:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2023-10-17T01:30:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2023-10-17T01:30:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2023-10-17T01:30:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2023-10-17T01:30:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2023-10-17T01:30:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 0, 2023-10-17T01:30:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 0, 2023-10-17T01:30:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2023-10-17T01:30:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2023-10-17T01:30:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2023-10-17T01:30:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2023-10-17T01:30:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2023-10-17T01:30:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2023-10-17T01:30:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2023-10-17T01:30:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 0, 2023-10-17T01:30:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2023-10-17T01:30:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2023-10-17T01:30:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 925, 2023-10-17T01:30:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2023-10-17T01:30:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2023-10-17T01:30:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 0, 2023-10-17T01:30:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2023-10-17T01:30:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2023-10-17T01:30:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 1, 2023-10-17T01:30:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2023-10-17T01:30:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2023-10-17T01:30:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2023-10-17T01:30:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 5290571, 2023-10-17T01:30:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 5289384, 2023-10-17T01:30:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2023-10-17T01:30:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 925 2023-10-17T01:30:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2023-10-17T01:30:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 220, 2023-10-17T01:30:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 222, 2023-10-17T01:30:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 2, 2023-10-17T01:30:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 16, 2023-10-17T01:30:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 16, 2023-10-17T01:30:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2023-10-17T01:30:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 227, 2023-10-17T01:30:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 227, 2023-10-17T01:30:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 0, 2023-10-17T01:30:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2023-10-17T01:30:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 0, 2023-10-17T01:30:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 0, 2023-10-17T01:30:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 50532, 2023-10-17T01:30:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 207156 2023-10-17T01:30:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2023-10-17T01:30:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 0, 2023-10-17T01:30:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2023-10-17T01:30:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 0, 2023-10-17T01:30:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 0, 2023-10-17T01:30:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2023-10-17T01:30:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2023-10-17T01:30:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2023-10-17T01:30:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2023-10-17T01:30:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2023-10-17T01:30:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2023-10-17T01:30:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2023-10-17T01:30:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.106 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.106 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2023-10-17T01:30:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2023-10-17T01:30:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2023-10-17T01:30:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.110 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2023-10-17T01:30:30.110 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.110 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.110 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.111 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2023-10-17T01:30:30.111 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.111 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.111 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.112 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2023-10-17T01:30:30.112 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.112 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.112 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.113 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2023-10-17T01:30:30.113 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.113 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.113 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.114 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:30.114 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.114 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.114 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.115 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2023-10-17T01:30:30.115 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.115 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.115 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.115 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:30.116 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.116 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.116 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2023-10-17T01:30:30.118 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.118 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.118 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.118 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:30.119 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.119 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.119 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.119 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:30.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.120 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.120 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2023-10-17T01:30:30.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.121 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.121 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.121 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2023-10-17T01:30:30.121 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.122 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.122 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.122 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2023-10-17T01:30:30.122 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.123 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.123 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.123 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2023-10-17T01:30:30.123 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.124 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.124 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.124 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:30.124 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:30.125 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:30.125 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.125 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.125 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2023-10-17T01:30:30.126 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2023-10-17T01:30:30.126 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2023-10-17T01:30:30.126 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2023-10-17T01:30:30.126 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2023-10-17T01:30:30.127 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2023-10-17T01:30:30.127 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 0, 2023-10-17T01:30:30.127 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 0, 2023-10-17T01:30:30.127 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 2553 2023-10-17T01:30:30.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-10-17T01:30:30.128 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:30.128 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:30.128 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:30.128 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:30.129 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:30.129 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:30.129 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:30.129 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:30.129 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:30.130 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:30.130 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:30.130 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:30.130 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:30.130 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:30.130 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:30.131 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:30.131 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:30.131 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:30.131 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:30.131 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:30.132 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:30.132 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:30.132 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:30.132 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:30.132 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:30.133 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:30.133 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:30.133 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:30.133 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:30.133 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:30.133 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:30.134 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-10-17T01:30:30.134 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:30.134 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:30.134 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:30.134 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:30.135 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 198242, 2023-10-17T01:30:30.135 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 262, 2023-10-17T01:30:30.135 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:30.135 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:30.135 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-10-17T01:30:30.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-10-17T01:30:30.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:30.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:30.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:30.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:30.137 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:30.137 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 156, 2023-10-17T01:30:30.137 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:30.137 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:30.137 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:30.137 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-10-17T01:30:30.138 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 669065, 2023-10-17T01:30:30.138 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 5720, 2023-10-17T01:30:30.138 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:30.138 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-10-17T01:30:30.138 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-10-17T01:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2023-10-17T01:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2023-10-17T01:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2023-10-17T01:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 3192, 2023-10-17T01:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 21, 2023-10-17T01:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 3, 2023-10-17T01:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 3192, 2023-10-17T01:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2023-10-17T01:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3192, 2023-10-17T01:30:30.141 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 9.034863644, 2023-10-17T01:30:30.141 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002830471 2023-10-17T01:30:30.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.141 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2023-10-17T01:30:30.141 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2023-10-17T01:30:30.142 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3192, 2023-10-17T01:30:30.142 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 3193 2023-10-17T01:30:30.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.142 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 3192, 2023-10-17T01:30:30.142 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 3192, 2023-10-17T01:30:30.143 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 0, 2023-10-17T01:30:30.143 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2023-10-17T01:30:30.143 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2023-10-17T01:30:30.143 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 1974, 2023-10-17T01:30:30.143 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 0, 2023-10-17T01:30:30.143 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 1216, 2023-10-17T01:30:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 0, 2023-10-17T01:30:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 0, 2023-10-17T01:30:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2023-10-17T01:30:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2023-10-17T01:30:30.144 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2023-10-17T01:30:30.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2023-10-17T01:30:30.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 0, 2023-10-17T01:30:30.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2023-10-17T01:30:30.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:30.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:30.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1, 2023-10-17T01:30:30.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 0, 2023-10-17T01:30:30.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:30.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:30.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:30.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2023-10-17T01:30:30.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2023-10-17T01:30:30.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2023-10-17T01:30:30.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:30.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2023-10-17T01:30:30.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:30.148 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2023-10-17T01:30:30.148 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2023-10-17T01:30:30.148 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2023-10-17T01:30:30.148 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2023-10-17T01:30:30.148 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2023-10-17T01:30:30.149 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2023-10-17T01:30:30.149 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2023-10-17T01:30:30.149 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2023-10-17T01:30:30.149 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2023-10-17T01:30:30.149 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2023-10-17T01:30:30.149 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2023-10-17T01:30:30.150 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2023-10-17T01:30:30.150 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2023-10-17T01:30:30.150 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2023-10-17T01:30:30.150 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2023-10-17T01:30:30.150 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2023-10-17T01:30:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2023-10-17T01:30:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 85, 2023-10-17T01:30:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2023-10-17T01:30:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 2, 2023-10-17T01:30:30.151 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 5, 2023-10-17T01:30:30.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 7, 2023-10-17T01:30:30.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 2, 2023-10-17T01:30:30.152 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2023-10-17T01:30:30.152 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 0, 2023-10-17T01:30:30.152 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 2, 2023-10-17T01:30:30.153 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 0 2023-10-17T01:30:30.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.153 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2023-10-17T01:30:30.153 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 0, 2023-10-17T01:30:30.153 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:30.153 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2023-10-17T01:30:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2023-10-17T01:30:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2023-10-17T01:30:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2023-10-17T01:30:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.155 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2023-10-17T01:30:30.155 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:30.155 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 0, 2023-10-17T01:30:30.155 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2023-10-17T01:30:30.155 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 0, 2023-10-17T01:30:30.156 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 0, 2023-10-17T01:30:30.156 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:30.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.156 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:30.156 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:30.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-10-17T01:30:30.157 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:30.157 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 3507, 2023-10-17T01:30:30.157 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 2287037, 2023-10-17T01:30:30.157 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:30.157 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 3507, 2023-10-17T01:30:30.158 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:30.158 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:30.158 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3507, 2023-10-17T01:30:30.158 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 2287037, 2023-10-17T01:30:30.159 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:30.159 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.159 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.159 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.160 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:30.160 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:30.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-10-17T01:30:30.160 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:30.160 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:30.161 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:30.161 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:30.161 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:30.161 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 3192, 2023-10-17T01:30:30.161 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 1096272, 2023-10-17T01:30:30.162 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3192, 2023-10-17T01:30:30.162 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1096272, 2023-10-17T01:30:30.162 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:30.162 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.162 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.163 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.163 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:30.163 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:30.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2023-10-17T01:30:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:30.164 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 3192, 2023-10-17T01:30:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 3192, 2023-10-17T01:30:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3192, 2023-10-17T01:30:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 3192, 2023-10-17T01:30:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:30.165 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.166 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.166 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.166 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:30.167 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:30.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2023-10-17T01:30:30.167 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:30.167 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:30.167 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:30.168 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:30.168 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:30.168 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:30.168 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:30.168 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2023-10-17T01:30:30.168 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2023-10-17T01:30:30.169 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:30.169 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.169 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.169 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:30.170 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x55a827566640": { 2023-10-17T01:30:30.170 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:30.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2023-10-17T01:30:30.170 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:30.170 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:30.171 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:30.171 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:30.171 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:30.171 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:30.171 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:30.172 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2023-10-17T01:30:30.172 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2023-10-17T01:30:30.172 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:30.172 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:30.172 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:30.172 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:30.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:30.173 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:30:30.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:30 smithi045 ceph-mon[98474]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.6 MiB/s wr, 753 op/s 2023-10-17T01:30:30.559 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-10-17T01:30:30.560 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-10-17T01:30:30.560 DEBUG:tasks.check_counter:Getting stats from d 2023-10-17T01:30:30.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-17T01:30:30.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:30 smithi060 ceph-mon[134136]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.6 MiB/s wr, 753 op/s 2023-10-17T01:30:30.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:30 smithi181 ceph-mon[134520]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.6 MiB/s wr, 753 op/s 2023-10-17T01:30:31.305 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:30:32.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:32 smithi045 ceph-mon[98474]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.5 MiB/s wr, 482 op/s 2023-10-17T01:30:32.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:32 smithi060 ceph-mon[134136]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.5 MiB/s wr, 482 op/s 2023-10-17T01:30:32.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:32 smithi181 ceph-mon[134520]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.5 MiB/s wr, 482 op/s 2023-10-17T01:30:32.874 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:30:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2023-10-17T01:30:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2023-10-17T01:30:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2944, 2023-10-17T01:30:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2023-10-17T01:30:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-10-17T01:30:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2023-10-17T01:30:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.004416003, 2023-10-17T01:30:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000099186, 2023-10-17T01:30:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001609758, 2023-10-17T01:30:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-17T01:30:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2023-10-17T01:30:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001604101, 2023-10-17T01:30:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000534700 2023-10-17T01:30:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-17T01:30:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2023-10-17T01:30:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2023-10-17T01:30:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 220, 2023-10-17T01:30:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2023-10-17T01:30:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 190784, 2023-10-17T01:30:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-10-17T01:30:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2023-10-17T01:30:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.029540467, 2023-10-17T01:30:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.022315244, 2023-10-17T01:30:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.003076632, 2023-10-17T01:30:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-17T01:30:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 218, 2023-10-17T01:30:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.036874033, 2023-10-17T01:30:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000169146 2023-10-17T01:30:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-17T01:30:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 190784 2023-10-17T01:30:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 291, 2023-10-17T01:30:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 288, 2023-10-17T01:30:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 491552, 2023-10-17T01:30:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 164528, 2023-10-17T01:30:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2023-10-17T01:30:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2023-10-17T01:30:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.085144346, 2023-10-17T01:30:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.051515981, 2023-10-17T01:30:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.020870900, 2023-10-17T01:30:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.008997094, 2023-10-17T01:30:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 287, 2023-10-17T01:30:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.047365684, 2023-10-17T01:30:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000165037 2023-10-17T01:30:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 491552, 2023-10-17T01:30:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 164528 2023-10-17T01:30:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-10-17T01:30:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2023-10-17T01:30:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-10-17T01:30:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-10-17T01:30:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-10-17T01:30:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 147920, 2023-10-17T01:30:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 27, 2023-10-17T01:30:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-10-17T01:30:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-10-17T01:30:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2023-10-17T01:30:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2023-10-17T01:30:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-10-17T01:30:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2023-10-17T01:30:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2023-10-17T01:30:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-10-17T01:30:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-10-17T01:30:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-10-17T01:30:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 293, 2023-10-17T01:30:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 450441, 2023-10-17T01:30:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 293, 2023-10-17T01:30:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 293, 2023-10-17T01:30:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 450441, 2023-10-17T01:30:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:32.911 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:30:33.402 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-17T01:30:33.403 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-17T01:30:33.403 DEBUG:tasks.check_counter:Getting stats from g 2023-10-17T01:30:33.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-17T01:30:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:30:34.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:34 smithi045 ceph-mon[98474]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 643 KiB/s wr, 74 op/s 2023-10-17T01:30:34.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:34 smithi060 ceph-mon[134136]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 643 KiB/s wr, 74 op/s 2023-10-17T01:30:34.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:34 smithi181 ceph-mon[134520]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 643 KiB/s wr, 74 op/s 2023-10-17T01:30:35.726 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:30:35.726 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:35.726 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1039, 2023-10-17T01:30:35.727 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1041, 2023-10-17T01:30:35.727 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 349049, 2023-10-17T01:30:35.727 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 323130, 2023-10-17T01:30:35.727 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 7, 2023-10-17T01:30:35.727 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2023-10-17T01:30:35.728 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.196592778, 2023-10-17T01:30:35.728 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.088833431, 2023-10-17T01:30:35.728 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.035257613, 2023-10-17T01:30:35.728 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.053533835, 2023-10-17T01:30:35.728 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:35.728 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1039, 2023-10-17T01:30:35.729 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.205428924, 2023-10-17T01:30:35.729 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000197717 2023-10-17T01:30:35.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.729 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:35.729 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.730 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.730 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.730 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:30:35.730 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:30:35.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.731 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:35.731 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2191, 2023-10-17T01:30:35.731 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2407, 2023-10-17T01:30:35.731 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3014021, 2023-10-17T01:30:35.732 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 1126459, 2023-10-17T01:30:35.732 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2023-10-17T01:30:35.732 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2023-10-17T01:30:35.732 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.302245599, 2023-10-17T01:30:35.733 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.096436490, 2023-10-17T01:30:35.733 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.067209572, 2023-10-17T01:30:35.733 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.118342786, 2023-10-17T01:30:35.733 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:35.733 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2405, 2023-10-17T01:30:35.734 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.395792118, 2023-10-17T01:30:35.734 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000164570 2023-10-17T01:30:35.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.734 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:35.734 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.735 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.735 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.736 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-17T01:30:35.737 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 445312 2023-10-17T01:30:35.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.737 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:35.737 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 322, 2023-10-17T01:30:35.737 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 297, 2023-10-17T01:30:35.738 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 631412, 2023-10-17T01:30:35.738 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 165328, 2023-10-17T01:30:35.738 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2023-10-17T01:30:35.738 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:35.738 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.079193242, 2023-10-17T01:30:35.739 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.039722823, 2023-10-17T01:30:35.739 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.028419767, 2023-10-17T01:30:35.739 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.009540526, 2023-10-17T01:30:35.739 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:35.739 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 297, 2023-10-17T01:30:35.740 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.064589405, 2023-10-17T01:30:35.740 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000217472 2023-10-17T01:30:35.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.740 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:35.740 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.741 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.741 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.741 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 630064, 2023-10-17T01:30:35.741 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 164384 2023-10-17T01:30:35.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.742 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-10-17T01:30:35.742 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2023-10-17T01:30:35.742 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-10-17T01:30:35.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.743 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2023-10-17T01:30:35.743 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2023-10-17T01:30:35.743 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2023-10-17T01:30:35.743 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5334, 2023-10-17T01:30:35.744 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.223001262, 2023-10-17T01:30:35.744 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000041807 2023-10-17T01:30:35.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.745 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:35.745 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2023-10-17T01:30:35.745 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2023-10-17T01:30:35.745 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.745 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.745 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2023-10-17T01:30:35.746 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 0, 2023-10-17T01:30:35.747 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 0, 2023-10-17T01:30:35.747 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2023-10-17T01:30:35.747 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.747 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.747 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.748 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2023-10-17T01:30:35.748 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2023-10-17T01:30:35.748 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:35.748 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:35.749 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 0, 2023-10-17T01:30:35.749 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2023-10-17T01:30:35.749 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2023-10-17T01:30:35.749 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 404, 2023-10-17T01:30:35.749 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 32, 2023-10-17T01:30:35.749 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 341, 2023-10-17T01:30:35.750 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 31, 2023-10-17T01:30:35.750 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 357, 2023-10-17T01:30:35.750 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2023-10-17T01:30:35.750 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2023-10-17T01:30:35.750 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2023-10-17T01:30:35.751 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 8, 2023-10-17T01:30:35.751 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 0, 2023-10-17T01:30:35.751 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 0, 2023-10-17T01:30:35.751 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2023-10-17T01:30:35.751 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2023-10-17T01:30:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2023-10-17T01:30:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2023-10-17T01:30:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2023-10-17T01:30:35.753 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2023-10-17T01:30:35.753 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2023-10-17T01:30:35.753 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2023-10-17T01:30:35.753 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2023-10-17T01:30:35.753 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:35.754 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2023-10-17T01:30:35.754 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:35.754 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2023-10-17T01:30:35.754 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2023-10-17T01:30:35.754 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2023-10-17T01:30:35.755 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2023-10-17T01:30:35.755 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:35.755 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:35.755 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:35.755 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:30:35.756 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:35.756 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:30:35.756 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2023-10-17T01:30:35.756 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:30:35.756 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 0, 2023-10-17T01:30:35.757 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 0, 2023-10-17T01:30:35.757 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:35.757 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:35.757 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:35.757 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 0, 2023-10-17T01:30:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 0, 2023-10-17T01:30:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 0, 2023-10-17T01:30:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 0 2023-10-17T01:30:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2023-10-17T01:30:35.759 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2023-10-17T01:30:35.759 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:35.759 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2023-10-17T01:30:35.759 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2023-10-17T01:30:35.759 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2023-10-17T01:30:35.759 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2023-10-17T01:30:35.760 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:35.760 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:35.760 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 0, 2023-10-17T01:30:35.760 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 0, 2023-10-17T01:30:35.760 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:35.761 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2023-10-17T01:30:35.761 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:35.761 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:35.761 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:35.761 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2023-10-17T01:30:35.761 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2023-10-17T01:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2023-10-17T01:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2023-10-17T01:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:35.763 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2023-10-17T01:30:35.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.763 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2023-10-17T01:30:35.763 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 0, 2023-10-17T01:30:35.763 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2023-10-17T01:30:35.764 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2023-10-17T01:30:35.764 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 1555, 2023-10-17T01:30:35.764 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2023-10-17T01:30:35.764 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2023-10-17T01:30:35.764 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 0, 2023-10-17T01:30:35.765 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2023-10-17T01:30:35.765 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2023-10-17T01:30:35.765 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 2, 2023-10-17T01:30:35.765 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2023-10-17T01:30:35.765 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2023-10-17T01:30:35.766 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2023-10-17T01:30:35.766 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 6652753, 2023-10-17T01:30:35.766 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 6651615, 2023-10-17T01:30:35.766 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2023-10-17T01:30:35.766 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.767 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.767 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.767 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 1555 2023-10-17T01:30:35.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2023-10-17T01:30:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 148, 2023-10-17T01:30:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 384, 2023-10-17T01:30:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 236, 2023-10-17T01:30:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 19, 2023-10-17T01:30:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 20, 2023-10-17T01:30:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 1, 2023-10-17T01:30:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 404, 2023-10-17T01:30:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 411, 2023-10-17T01:30:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 7, 2023-10-17T01:30:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2023-10-17T01:30:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 0, 2023-10-17T01:30:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 0, 2023-10-17T01:30:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 49768, 2023-10-17T01:30:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 207156 2023-10-17T01:30:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2023-10-17T01:30:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 0, 2023-10-17T01:30:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2023-10-17T01:30:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 0, 2023-10-17T01:30:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 0, 2023-10-17T01:30:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2023-10-17T01:30:35.772 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2023-10-17T01:30:35.772 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.772 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.772 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.773 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2023-10-17T01:30:35.773 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.773 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.773 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.774 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:35.774 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.774 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.774 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.775 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2023-10-17T01:30:35.775 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.775 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.775 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.776 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2023-10-17T01:30:35.776 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.776 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.776 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.777 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:35.777 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.777 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.777 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.777 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2023-10-17T01:30:35.778 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.778 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.778 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.778 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2023-10-17T01:30:35.779 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.779 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.779 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.779 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:35.780 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.780 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.780 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.780 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2023-10-17T01:30:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.782 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.782 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.782 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2023-10-17T01:30:35.782 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.783 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.783 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.783 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2023-10-17T01:30:35.783 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.784 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.784 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.784 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2023-10-17T01:30:35.784 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.785 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.785 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.785 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2023-10-17T01:30:35.785 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.785 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.786 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.786 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2023-10-17T01:30:35.786 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.786 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.787 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.787 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2023-10-17T01:30:35.787 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.787 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.788 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.788 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:35.788 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.788 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.789 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.789 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2023-10-17T01:30:35.789 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.789 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.789 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.790 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:35.790 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.790 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.790 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.791 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:35.791 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.791 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.792 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.792 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2023-10-17T01:30:35.792 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.792 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.792 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.793 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:35.793 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.793 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.793 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.794 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:35.794 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.794 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.794 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.795 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2023-10-17T01:30:35.795 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.795 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.795 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.796 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2023-10-17T01:30:35.796 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.796 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.796 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.797 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2023-10-17T01:30:35.797 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.797 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.797 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.798 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2023-10-17T01:30:35.798 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.798 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.798 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.799 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:35.799 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:35.799 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:35.799 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.799 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.800 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2023-10-17T01:30:35.800 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2023-10-17T01:30:35.801 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2023-10-17T01:30:35.801 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2023-10-17T01:30:35.801 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2023-10-17T01:30:35.801 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2023-10-17T01:30:35.801 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 0, 2023-10-17T01:30:35.801 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 0, 2023-10-17T01:30:35.802 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 2559 2023-10-17T01:30:35.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.802 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-10-17T01:30:35.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:35.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:35.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:35.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:35.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:35.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:35.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:35.804 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:35.804 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:35.804 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:35.804 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:35.804 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:35.804 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:35.805 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:35.805 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:35.805 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:35.805 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:35.805 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:35.806 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:35.806 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:35.806 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:35.806 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:35.806 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:35.807 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:35.807 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:35.807 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:35.807 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:35.807 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:35.808 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:35.808 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:35.808 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:35.808 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-10-17T01:30:35.808 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:35.809 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:35.809 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:35.809 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:35.809 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 202606, 2023-10-17T01:30:35.809 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 193, 2023-10-17T01:30:35.809 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:35.810 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:35.810 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-10-17T01:30:35.810 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-10-17T01:30:35.810 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:35.810 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:35.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:35.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:35.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:35.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 156, 2023-10-17T01:30:35.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:35.812 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:35.812 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:35.812 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-10-17T01:30:35.812 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 638215, 2023-10-17T01:30:35.812 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 5854, 2023-10-17T01:30:35.812 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:35.813 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-10-17T01:30:35.813 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:35.813 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-10-17T01:30:35.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.813 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2023-10-17T01:30:35.814 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2023-10-17T01:30:35.814 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2023-10-17T01:30:35.814 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 3232, 2023-10-17T01:30:35.814 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 21, 2023-10-17T01:30:35.814 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 3, 2023-10-17T01:30:35.815 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 3232, 2023-10-17T01:30:35.815 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2023-10-17T01:30:35.815 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3232, 2023-10-17T01:30:35.815 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 10.284844747, 2023-10-17T01:30:35.815 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003182192 2023-10-17T01:30:35.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.816 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2023-10-17T01:30:35.816 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2023-10-17T01:30:35.816 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3232, 2023-10-17T01:30:35.816 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 3233 2023-10-17T01:30:35.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.817 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 3232, 2023-10-17T01:30:35.817 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 3232, 2023-10-17T01:30:35.817 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 0, 2023-10-17T01:30:35.817 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2023-10-17T01:30:35.817 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2023-10-17T01:30:35.818 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 1973, 2023-10-17T01:30:35.818 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 0, 2023-10-17T01:30:35.818 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 1257, 2023-10-17T01:30:35.818 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 0, 2023-10-17T01:30:35.818 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 0, 2023-10-17T01:30:35.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2023-10-17T01:30:35.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2023-10-17T01:30:35.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2023-10-17T01:30:35.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2023-10-17T01:30:35.819 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 0, 2023-10-17T01:30:35.820 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2023-10-17T01:30:35.820 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:35.820 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:35.820 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1, 2023-10-17T01:30:35.820 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 0, 2023-10-17T01:30:35.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:35.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:35.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:35.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2023-10-17T01:30:35.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2023-10-17T01:30:35.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2023-10-17T01:30:35.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:35.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2023-10-17T01:30:35.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:35.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2023-10-17T01:30:35.823 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2023-10-17T01:30:35.823 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2023-10-17T01:30:35.823 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2023-10-17T01:30:35.823 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2023-10-17T01:30:35.823 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2023-10-17T01:30:35.824 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2023-10-17T01:30:35.824 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2023-10-17T01:30:35.824 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2023-10-17T01:30:35.824 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2023-10-17T01:30:35.824 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2023-10-17T01:30:35.824 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2023-10-17T01:30:35.825 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2023-10-17T01:30:35.825 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2023-10-17T01:30:35.825 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2023-10-17T01:30:35.825 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2023-10-17T01:30:35.825 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2023-10-17T01:30:35.826 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 85, 2023-10-17T01:30:35.826 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2023-10-17T01:30:35.826 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 2, 2023-10-17T01:30:35.826 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 6, 2023-10-17T01:30:35.826 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 8, 2023-10-17T01:30:35.827 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 2, 2023-10-17T01:30:35.827 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2023-10-17T01:30:35.827 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 0, 2023-10-17T01:30:35.827 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 2, 2023-10-17T01:30:35.827 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 0 2023-10-17T01:30:35.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.828 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2023-10-17T01:30:35.828 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 0, 2023-10-17T01:30:35.828 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:35.828 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2023-10-17T01:30:35.828 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2023-10-17T01:30:35.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.829 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2023-10-17T01:30:35.829 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2023-10-17T01:30:35.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.829 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2023-10-17T01:30:35.830 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:35.830 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 0, 2023-10-17T01:30:35.830 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2023-10-17T01:30:35.830 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 0, 2023-10-17T01:30:35.830 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 0, 2023-10-17T01:30:35.831 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:35.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.831 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:35.831 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:35.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-10-17T01:30:35.832 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:35.832 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 3548, 2023-10-17T01:30:35.832 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 3656116, 2023-10-17T01:30:35.832 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:35.832 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 3548, 2023-10-17T01:30:35.832 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3548, 2023-10-17T01:30:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 3656116, 2023-10-17T01:30:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-10-17T01:30:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 3232, 2023-10-17T01:30:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 2458454, 2023-10-17T01:30:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3232, 2023-10-17T01:30:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 2458454, 2023-10-17T01:30:35.837 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:35.837 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.837 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.837 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.838 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:35.838 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:35.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2023-10-17T01:30:35.838 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:35.839 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:35.839 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:35.839 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:35.839 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:35.839 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 3232, 2023-10-17T01:30:35.840 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 3232, 2023-10-17T01:30:35.840 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3232, 2023-10-17T01:30:35.840 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 3232, 2023-10-17T01:30:35.840 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:35.840 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.841 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.841 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.841 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:35.842 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:35.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2023-10-17T01:30:35.842 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:35.842 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:35.842 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:35.843 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:35.843 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:35.843 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:35.843 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:35.843 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2023-10-17T01:30:35.844 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2023-10-17T01:30:35.844 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:35.844 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.844 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.844 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:35.845 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x5648611525a0": { 2023-10-17T01:30:35.845 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:35.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2023-10-17T01:30:35.846 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:35.846 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2023-10-17T01:30:35.846 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2023-10-17T01:30:35.846 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:35.846 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:35.847 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:35.847 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:35.847 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2023-10-17T01:30:35.847 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2023-10-17T01:30:35.847 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:35.847 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:35.848 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:35.848 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:35.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:35.848 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:30:36.246 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-10-17T01:30:36.247 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-10-17T01:30:36.247 DEBUG:tasks.check_counter:Getting stats from j 2023-10-17T01:30:36.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-17T01:30:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:36 smithi045 ceph-mon[98474]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-10-17T01:30:36.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:36 smithi060 ceph-mon[134136]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-10-17T01:30:36.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:36 smithi181 ceph-mon[134520]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-10-17T01:30:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-17T01:30:38.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:38 smithi045 ceph-mon[98474]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 65 KiB/s wr, 19 op/s 2023-10-17T01:30:38.564 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-17T01:30:38.564 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:38.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 294, 2023-10-17T01:30:38.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 291, 2023-10-17T01:30:38.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 492992, 2023-10-17T01:30:38.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 166048, 2023-10-17T01:30:38.565 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-10-17T01:30:38.566 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2023-10-17T01:30:38.566 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.069975803, 2023-10-17T01:30:38.566 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.032459505, 2023-10-17T01:30:38.566 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.023779633, 2023-10-17T01:30:38.566 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.009601391, 2023-10-17T01:30:38.567 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:38.567 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 290, 2023-10-17T01:30:38.567 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.051401880, 2023-10-17T01:30:38.567 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000177247 2023-10-17T01:30:38.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.568 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:38.568 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:38.568 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:38.568 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:38.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.569 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 492992, 2023-10-17T01:30:38.569 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 166048 2023-10-17T01:30:38.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.569 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:38.569 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2023-10-17T01:30:38.570 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 222, 2023-10-17T01:30:38.570 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2023-10-17T01:30:38.570 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 191136, 2023-10-17T01:30:38.570 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2023-10-17T01:30:38.570 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2023-10-17T01:30:38.571 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.034565477, 2023-10-17T01:30:38.571 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.028957156, 2023-10-17T01:30:38.571 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002094388, 2023-10-17T01:30:38.571 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-17T01:30:38.571 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:38.571 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 220, 2023-10-17T01:30:38.572 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.040678339, 2023-10-17T01:30:38.572 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000184901 2023-10-17T01:30:38.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.572 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:38.572 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:38.573 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:38.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:38.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.573 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-17T01:30:38.573 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 191136 2023-10-17T01:30:38.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.574 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:38.574 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2023-10-17T01:30:38.574 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2023-10-17T01:30:38.574 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2944, 2023-10-17T01:30:38.575 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2023-10-17T01:30:38.575 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2023-10-17T01:30:38.575 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2023-10-17T01:30:38.575 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001298253, 2023-10-17T01:30:38.575 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000063185, 2023-10-17T01:30:38.576 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001179908, 2023-10-17T01:30:38.576 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-17T01:30:38.576 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:38.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2023-10-17T01:30:38.576 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001168474, 2023-10-17T01:30:38.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000389491 2023-10-17T01:30:38.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.577 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:38.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:38.577 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:38.578 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:38.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.578 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-17T01:30:38.578 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2023-10-17T01:30:38.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.579 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2023-10-17T01:30:38.579 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2023-10-17T01:30:38.579 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2023-10-17T01:30:38.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.579 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2023-10-17T01:30:38.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:38.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:38.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:38.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:38.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:38.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:38.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:38.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:38.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:38.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:38.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:38.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:38.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:38.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:38.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:38.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:38.583 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:38.583 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:38.583 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:38.583 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:38.583 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:38.583 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:38.584 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:38.584 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:38.584 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:38.584 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:38.584 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:38.585 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:38.585 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:38.585 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:38.585 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:38.585 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2023-10-17T01:30:38.586 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:38.586 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:38.586 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:38.586 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:38.586 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148744, 2023-10-17T01:30:38.586 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2023-10-17T01:30:38.587 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:38.587 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:38.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2023-10-17T01:30:38.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2023-10-17T01:30:38.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:38.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:38.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:38.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:38.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2023-10-17T01:30:38.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2023-10-17T01:30:38.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:38.589 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:38.589 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:38.589 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2023-10-17T01:30:38.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2023-10-17T01:30:38.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2023-10-17T01:30:38.590 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:38.590 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2023-10-17T01:30:38.590 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:38.590 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2023-10-17T01:30:38.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-17T01:30:38.591 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:38.591 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2023-10-17T01:30:38.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2023-10-17T01:30:38.591 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2023-10-17T01:30:38.591 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 296, 2023-10-17T01:30:38.592 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 451468, 2023-10-17T01:30:38.592 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:38.592 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 296, 2023-10-17T01:30:38.592 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2023-10-17T01:30:38.592 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2023-10-17T01:30:38.592 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 296, 2023-10-17T01:30:38.593 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 451468, 2023-10-17T01:30:38.593 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2023-10-17T01:30:38.593 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2023-10-17T01:30:38.593 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2023-10-17T01:30:38.593 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2023-10-17T01:30:38.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:38.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-17T01:30:38.594 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-17T01:30:38.595 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:38 smithi060 ceph-mon[134136]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 65 KiB/s wr, 19 op/s 2023-10-17T01:30:38.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:38 smithi181 ceph-mon[134520]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 65 KiB/s wr, 19 op/s 2023-10-17T01:30:39.100 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-17T01:30:39.100 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-17T01:30:39.100 DEBUG:tasks.check_counter:Getting stats from b 2023-10-17T01:30:39.101 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-17T01:30:40.458 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:30:40.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:40 smithi045 ceph-mon[98474]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 17 op/s 2023-10-17T01:30:40.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:40 smithi060 ceph-mon[134136]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 17 op/s 2023-10-17T01:30:40.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:40 smithi181 ceph-mon[134520]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 17 op/s 2023-10-17T01:30:41.835 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-10-17T01:30:41.835 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:41.835 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 3954, 2023-10-17T01:30:41.836 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 4015, 2023-10-17T01:30:41.836 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 1647737, 2023-10-17T01:30:41.836 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 9556223, 2023-10-17T01:30:41.836 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 13, 2023-10-17T01:30:41.836 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 5, 2023-10-17T01:30:41.837 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.331832461, 2023-10-17T01:30:41.837 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.095271889, 2023-10-17T01:30:41.837 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.112197125, 2023-10-17T01:30:41.837 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.112862370, 2023-10-17T01:30:41.837 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:41.838 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4015, 2023-10-17T01:30:41.838 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 5.529813294, 2023-10-17T01:30:41.838 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001377288 2023-10-17T01:30:41.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.838 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:41.839 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3577, 2023-10-17T01:30:41.839 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000867175, 2023-10-17T01:30:41.839 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000242 2023-10-17T01:30:41.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.839 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:30:41.840 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:30:41.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.840 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:41.840 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2389395, 2023-10-17T01:30:41.840 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2390081, 2023-10-17T01:30:41.841 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 378429725, 2023-10-17T01:30:41.841 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 401237262, 2023-10-17T01:30:41.841 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 10, 2023-10-17T01:30:41.841 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 5, 2023-10-17T01:30:41.841 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 100.486401228, 2023-10-17T01:30:41.842 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 37.348015114, 2023-10-17T01:30:41.842 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 72.764273383, 2023-10-17T01:30:41.842 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.099257806, 2023-10-17T01:30:41.842 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:41.842 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2390082, 2023-10-17T01:30:41.843 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 67.976334474, 2023-10-17T01:30:41.843 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000028441 2023-10-17T01:30:41.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.843 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:41.843 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4776009, 2023-10-17T01:30:41.844 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.698579771, 2023-10-17T01:30:41.844 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000146 2023-10-17T01:30:41.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.844 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-17T01:30:41.844 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 462640 2023-10-17T01:30:41.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.845 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:41.845 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2392993, 2023-10-17T01:30:41.845 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2395327, 2023-10-17T01:30:41.845 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 380041673, 2023-10-17T01:30:41.845 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 393753995, 2023-10-17T01:30:41.846 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 220, 2023-10-17T01:30:41.846 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 10, 2023-10-17T01:30:41.846 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 123.371710496, 2023-10-17T01:30:41.846 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 53.936877694, 2023-10-17T01:30:41.846 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 84.687283081, 2023-10-17T01:30:41.847 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.107744667, 2023-10-17T01:30:41.847 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:41.847 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2395322, 2023-10-17T01:30:41.847 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 47.228272022, 2023-10-17T01:30:41.847 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000019716 2023-10-17T01:30:41.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.848 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:41.848 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4783696, 2023-10-17T01:30:41.848 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.641244973, 2023-10-17T01:30:41.848 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000134 2023-10-17T01:30:41.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.849 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 714112, 2023-10-17T01:30:41.849 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 203616 2023-10-17T01:30:41.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.849 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-10-17T01:30:41.850 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-10-17T01:30:41.850 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-10-17T01:30:41.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.850 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-10-17T01:30:41.850 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:41.851 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:41.851 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3339, 2023-10-17T01:30:41.851 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 3.010812947, 2023-10-17T01:30:41.851 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000901710 2023-10-17T01:30:41.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.853 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:41.853 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:41.853 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:41.854 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 136, 2023-10-17T01:30:41.854 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.016886255, 2023-10-17T01:30:41.854 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000124163 2023-10-17T01:30:41.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.855 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-10-17T01:30:41.855 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 2329, 2023-10-17T01:30:41.855 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 1894, 2023-10-17T01:30:41.855 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-10-17T01:30:41.855 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1894, 2023-10-17T01:30:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 73.287887211, 2023-10-17T01:30:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.038694766 2023-10-17T01:30:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-10-17T01:30:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 435, 2023-10-17T01:30:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 5, 2023-10-17T01:30:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 2, 2023-10-17T01:30:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 661, 2023-10-17T01:30:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 3, 2023-10-17T01:30:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-10-17T01:30:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 318, 2023-10-17T01:30:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 35, 2023-10-17T01:30:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 265, 2023-10-17T01:30:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 18, 2023-10-17T01:30:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 268, 2023-10-17T01:30:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 1098, 2023-10-17T01:30:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 250, 2023-10-17T01:30:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 250, 2023-10-17T01:30:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 19, 2023-10-17T01:30:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 3059, 2023-10-17T01:30:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 1912, 2023-10-17T01:30:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 432, 2023-10-17T01:30:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 1, 2023-10-17T01:30:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 139, 2023-10-17T01:30:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 300, 2023-10-17T01:30:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-10-17T01:30:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 22, 2023-10-17T01:30:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 1127, 2023-10-17T01:30:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 9, 2023-10-17T01:30:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 335, 2023-10-17T01:30:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 412, 2023-10-17T01:30:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 1041, 2023-10-17T01:30:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 1711951064, 2023-10-17T01:30:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-10-17T01:30:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 412, 2023-10-17T01:30:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 39, 2023-10-17T01:30:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 2, 2023-10-17T01:30:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 37, 2023-10-17T01:30:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 373, 2023-10-17T01:30:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 9, 2023-10-17T01:30:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 187, 2023-10-17T01:30:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 1360, 2023-10-17T01:30:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 1365, 2023-10-17T01:30:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 1102, 2023-10-17T01:30:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 21, 2023-10-17T01:30:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 175 2023-10-17T01:30:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-10-17T01:30:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-10-17T01:30:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-10-17T01:30:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 2, 2023-10-17T01:30:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 39, 2023-10-17T01:30:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-10-17T01:30:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 49, 2023-10-17T01:30:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 32, 2023-10-17T01:30:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 68, 2023-10-17T01:30:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-10-17T01:30:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-10-17T01:30:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-10-17T01:30:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 2, 2023-10-17T01:30:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 22, 2023-10-17T01:30:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-10-17T01:30:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 3, 2023-10-17T01:30:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-10-17T01:30:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-10-17T01:30:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 3543, 2023-10-17T01:30:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 3524, 2023-10-17T01:30:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 3524, 2023-10-17T01:30:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 19, 2023-10-17T01:30:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-10-17T01:30:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 2, 2023-10-17T01:30:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 73, 2023-10-17T01:30:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 72, 2023-10-17T01:30:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 72, 2023-10-17T01:30:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 2, 2023-10-17T01:30:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-10-17T01:30:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 1, 2023-10-17T01:30:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 14613047, 2023-10-17T01:30:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 14805115, 2023-10-17T01:30:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 0, 2023-10-17T01:30:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-10-17T01:30:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1191, 2023-10-17T01:30:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.845367175, 2023-10-17T01:30:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.005747579 2023-10-17T01:30:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 0 2023-10-17T01:30:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-10-17T01:30:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 322, 2023-10-17T01:30:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 1419, 2023-10-17T01:30:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 1097, 2023-10-17T01:30:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 46, 2023-10-17T01:30:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 76, 2023-10-17T01:30:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 30, 2023-10-17T01:30:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 318, 2023-10-17T01:30:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 1775, 2023-10-17T01:30:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 1457, 2023-10-17T01:30:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 250, 2023-10-17T01:30:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 1410, 2023-10-17T01:30:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 1160, 2023-10-17T01:30:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 68792, 2023-10-17T01:30:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 207156 2023-10-17T01:30:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-10-17T01:30:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 2718, 2023-10-17T01:30:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 89, 2023-10-17T01:30:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 2329, 2023-10-17T01:30:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 482, 2023-10-17T01:30:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 180, 2023-10-17T01:30:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-10-17T01:30:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1027, 2023-10-17T01:30:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 69.532809133, 2023-10-17T01:30:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.067704780 2023-10-17T01:30:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-10-17T01:30:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 38, 2023-10-17T01:30:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.118102312, 2023-10-17T01:30:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003107955 2023-10-17T01:30:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-10-17T01:30:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 17, 2023-10-17T01:30:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.043835202, 2023-10-17T01:30:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002578541 2023-10-17T01:30:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-10-17T01:30:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 76, 2023-10-17T01:30:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.199228478, 2023-10-17T01:30:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002621427 2023-10-17T01:30:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-10-17T01:30:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2, 2023-10-17T01:30:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000376775, 2023-10-17T01:30:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000188387 2023-10-17T01:30:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-10-17T01:30:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-10-17T01:30:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-10-17T01:30:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 35, 2023-10-17T01:30:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.299019849, 2023-10-17T01:30:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.008543424 2023-10-17T01:30:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-10-17T01:30:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-10-17T01:30:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-10-17T01:30:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 521, 2023-10-17T01:30:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.644214832, 2023-10-17T01:30:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001236496 2023-10-17T01:30:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-10-17T01:30:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 43, 2023-10-17T01:30:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.156021704, 2023-10-17T01:30:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003628411 2023-10-17T01:30:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-10-17T01:30:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 91, 2023-10-17T01:30:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 2.179608112, 2023-10-17T01:30:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.023951737 2023-10-17T01:30:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-10-17T01:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-10-17T01:30:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2023-10-17T01:30:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.001698719, 2023-10-17T01:30:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001698719 2023-10-17T01:30:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-10-17T01:30:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-10-17T01:30:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 6, 2023-10-17T01:30:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.020668929, 2023-10-17T01:30:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003444821 2023-10-17T01:30:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-10-17T01:30:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-10-17T01:30:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 37, 2023-10-17T01:30:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.092303166, 2023-10-17T01:30:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002494680 2023-10-17T01:30:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-10-17T01:30:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-10-17T01:30:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 2, 2023-10-17T01:30:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 1, 2023-10-17T01:30:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2023-10-17T01:30:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-10-17T01:30:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 2, 2023-10-17T01:30:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 2, 2023-10-17T01:30:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 958 2023-10-17T01:30:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-10-17T01:30:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-10-17T01:30:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 334664, 2023-10-17T01:30:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 414, 2023-10-17T01:30:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-10-17T01:30:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-10-17T01:30:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 156, 2023-10-17T01:30:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-10-17T01:30:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 1104612, 2023-10-17T01:30:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 10371, 2023-10-17T01:30:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-10-17T01:30:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-10-17T01:30:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-10-17T01:30:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-10-17T01:30:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-10-17T01:30:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 4281, 2023-10-17T01:30:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 12781664, 2023-10-17T01:30:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 37, 2023-10-17T01:30:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 4281, 2023-10-17T01:30:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-10-17T01:30:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4281, 2023-10-17T01:30:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 104.649988058, 2023-10-17T01:30:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.024445220 2023-10-17T01:30:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-10-17T01:30:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-10-17T01:30:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4281, 2023-10-17T01:30:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 7683 2023-10-17T01:30:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 4281, 2023-10-17T01:30:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 437, 2023-10-17T01:30:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 3844, 2023-10-17T01:30:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-10-17T01:30:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-10-17T01:30:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 590, 2023-10-17T01:30:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 1053, 2023-10-17T01:30:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 16, 2023-10-17T01:30:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 1205, 2023-10-17T01:30:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 237, 2023-10-17T01:30:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-10-17T01:30:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-10-17T01:30:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-10-17T01:30:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 3, 2023-10-17T01:30:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 563, 2023-10-17T01:30:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-10-17T01:30:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 421, 2023-10-17T01:30:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 2106, 2023-10-17T01:30:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-10-17T01:30:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-10-17T01:30:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-10-17T01:30:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-10-17T01:30:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2023-10-17T01:30:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-10-17T01:30:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-10-17T01:30:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-10-17T01:30:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-10-17T01:30:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-10-17T01:30:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-10-17T01:30:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-10-17T01:30:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-10-17T01:30:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-10-17T01:30:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-10-17T01:30:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-10-17T01:30:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-10-17T01:30:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-10-17T01:30:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-10-17T01:30:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-10-17T01:30:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-10-17T01:30:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 85, 2023-10-17T01:30:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-10-17T01:30:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 3, 2023-10-17T01:30:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 12, 2023-10-17T01:30:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 12, 2023-10-17T01:30:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2023-10-17T01:30:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-10-17T01:30:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 1426, 2023-10-17T01:30:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 14, 2023-10-17T01:30:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 49 2023-10-17T01:30:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-10-17T01:30:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 1, 2023-10-17T01:30:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-10-17T01:30:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-10-17T01:30:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-10-17T01:30:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 2453 2023-10-17T01:30:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-10-17T01:30:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 19, 2023-10-17T01:30:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-10-17T01:30:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 6, 2023-10-17T01:30:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 32, 2023-10-17T01:30:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 4786338, 2023-10-17T01:30:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 329321176, 2023-10-17T01:30:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 4786338, 2023-10-17T01:30:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4786338, 2023-10-17T01:30:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 329321176, 2023-10-17T01:30:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 4281, 2023-10-17T01:30:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 12784808, 2023-10-17T01:30:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4281, 2023-10-17T01:30:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 12784808, 2023-10-17T01:30:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-10-17T01:30:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 4281, 2023-10-17T01:30:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 4281, 2023-10-17T01:30:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4281, 2023-10-17T01:30:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 4281, 2023-10-17T01:30:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:41.950 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:41.950 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 32, 2023-10-17T01:30:41.950 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 3232, 2023-10-17T01:30:41.950 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:41.950 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 32, 2023-10-17T01:30:41.951 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:41.951 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:41.951 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 14, 2023-10-17T01:30:41.951 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 3232, 2023-10-17T01:30:41.951 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:41.952 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.952 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.952 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:41.952 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x55adcccdc0a0": { 2023-10-17T01:30:41.953 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:41.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:41.953 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:41.953 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 3543, 2023-10-17T01:30:41.953 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 10610811, 2023-10-17T01:30:41.954 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:41.954 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 3543, 2023-10-17T01:30:41.954 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:41.954 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:41.954 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 1191, 2023-10-17T01:30:41.955 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 10610811, 2023-10-17T01:30:41.955 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:41.955 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:41.955 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:41.955 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:41.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:41.956 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-10-17T01:30:42.223 INFO:tasks.check_counter:Daemon mds.b mds.exported=22 2023-10-17T01:30:42.224 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2023-10-17T01:30:42.224 DEBUG:tasks.check_counter:Getting stats from e 2023-10-17T01:30:42.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-17T01:30:42.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:42 smithi045 ceph-mon[98474]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 17 op/s 2023-10-17T01:30:42.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:42 smithi060 ceph-mon[134136]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 17 op/s 2023-10-17T01:30:42.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:42 smithi181 ceph-mon[134520]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 17 op/s 2023-10-17T01:30:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:30:44.023 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-10-17T01:30:44.023 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:44.024 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1174, 2023-10-17T01:30:44.024 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 1176, 2023-10-17T01:30:44.024 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 4579275, 2023-10-17T01:30:44.024 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 365124, 2023-10-17T01:30:44.024 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2023-10-17T01:30:44.024 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2023-10-17T01:30:44.025 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.121827254, 2023-10-17T01:30:44.025 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.036855952, 2023-10-17T01:30:44.025 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.031272704, 2023-10-17T01:30:44.025 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.042981024, 2023-10-17T01:30:44.025 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:44.026 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1174, 2023-10-17T01:30:44.026 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.403672738, 2023-10-17T01:30:44.026 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000343843 2023-10-17T01:30:44.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.026 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:44.027 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.027 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.027 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.027 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:30:44.027 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:30:44.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.028 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:44.028 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1694, 2023-10-17T01:30:44.028 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 1914, 2023-10-17T01:30:44.028 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 2648707, 2023-10-17T01:30:44.028 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 1058918, 2023-10-17T01:30:44.029 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2023-10-17T01:30:44.029 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2023-10-17T01:30:44.029 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.146289389, 2023-10-17T01:30:44.029 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.035753898, 2023-10-17T01:30:44.029 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.037792249, 2023-10-17T01:30:44.030 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.061550557, 2023-10-17T01:30:44.030 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:44.030 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1912, 2023-10-17T01:30:44.030 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.635303641, 2023-10-17T01:30:44.030 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000332271 2023-10-17T01:30:44.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.031 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:44.031 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.031 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.031 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.032 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-17T01:30:44.032 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 532640 2023-10-17T01:30:44.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.032 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:44.033 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 934, 2023-10-17T01:30:44.033 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 909, 2023-10-17T01:30:44.033 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 4982039, 2023-10-17T01:30:44.033 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 356913, 2023-10-17T01:30:44.033 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2023-10-17T01:30:44.033 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2023-10-17T01:30:44.034 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.095634253, 2023-10-17T01:30:44.034 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.025888193, 2023-10-17T01:30:44.034 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.035424976, 2023-10-17T01:30:44.034 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.030491570, 2023-10-17T01:30:44.034 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:44.035 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 909, 2023-10-17T01:30:44.035 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.165773249, 2023-10-17T01:30:44.035 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000182368 2023-10-17T01:30:44.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.035 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:44.035 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.036 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.036 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.036 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 632480, 2023-10-17T01:30:44.036 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 167824 2023-10-17T01:30:44.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.037 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-10-17T01:30:44.037 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-10-17T01:30:44.037 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-10-17T01:30:44.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.038 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-10-17T01:30:44.038 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:44.038 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:44.038 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 5781, 2023-10-17T01:30:44.038 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.184773377, 2023-10-17T01:30:44.039 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000031962 2023-10-17T01:30:44.039 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.039 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:44.039 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:44.039 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:44.040 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.040 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.040 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.041 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-10-17T01:30:44.041 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 0, 2023-10-17T01:30:44.041 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 0, 2023-10-17T01:30:44.041 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-10-17T01:30:44.041 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.042 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.042 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.042 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-10-17T01:30:44.042 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2023-10-17T01:30:44.043 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:44.043 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:44.043 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 0, 2023-10-17T01:30:44.043 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2023-10-17T01:30:44.043 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-10-17T01:30:44.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 315, 2023-10-17T01:30:44.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 209, 2023-10-17T01:30:44.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 100, 2023-10-17T01:30:44.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 6, 2023-10-17T01:30:44.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 16, 2023-10-17T01:30:44.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 4, 2023-10-17T01:30:44.045 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 0, 2023-10-17T01:30:44.045 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 0, 2023-10-17T01:30:44.045 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 10, 2023-10-17T01:30:44.045 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 0, 2023-10-17T01:30:44.045 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 0, 2023-10-17T01:30:44.046 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2023-10-17T01:30:44.046 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2023-10-17T01:30:44.046 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:44.046 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:44.046 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 0, 2023-10-17T01:30:44.047 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2023-10-17T01:30:44.047 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-10-17T01:30:44.047 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2023-10-17T01:30:44.047 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2023-10-17T01:30:44.047 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2023-10-17T01:30:44.048 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2023-10-17T01:30:44.048 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:44.048 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2023-10-17T01:30:44.048 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:44.048 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 1038, 2023-10-17T01:30:44.048 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 1711951064, 2023-10-17T01:30:44.049 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-10-17T01:30:44.049 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2023-10-17T01:30:44.049 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:44.049 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:44.049 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:44.050 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:30:44.050 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:44.050 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:30:44.050 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2023-10-17T01:30:44.050 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:30:44.051 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 0, 2023-10-17T01:30:44.051 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 0, 2023-10-17T01:30:44.051 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:44.051 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:44.051 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:44.052 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 0, 2023-10-17T01:30:44.052 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 0, 2023-10-17T01:30:44.052 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 0, 2023-10-17T01:30:44.052 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 0 2023-10-17T01:30:44.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-10-17T01:30:44.053 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-10-17T01:30:44.053 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:44.053 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-10-17T01:30:44.053 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2023-10-17T01:30:44.054 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2023-10-17T01:30:44.054 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-10-17T01:30:44.054 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:44.054 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:44.054 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 0, 2023-10-17T01:30:44.055 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 0, 2023-10-17T01:30:44.055 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:44.055 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-10-17T01:30:44.055 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:44.055 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:44.055 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:44.056 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-10-17T01:30:44.056 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-10-17T01:30:44.056 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:44.056 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2023-10-17T01:30:44.056 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-10-17T01:30:44.056 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:44.057 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:44.057 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-10-17T01:30:44.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.057 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-10-17T01:30:44.057 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 0, 2023-10-17T01:30:44.058 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2023-10-17T01:30:44.058 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2023-10-17T01:30:44.058 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 3543, 2023-10-17T01:30:44.058 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-10-17T01:30:44.058 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2023-10-17T01:30:44.059 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 0, 2023-10-17T01:30:44.059 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2023-10-17T01:30:44.059 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2023-10-17T01:30:44.059 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 2, 2023-10-17T01:30:44.059 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-10-17T01:30:44.060 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2023-10-17T01:30:44.060 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 14613047, 2023-10-17T01:30:44.060 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 14805115, 2023-10-17T01:30:44.060 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 14799896, 2023-10-17T01:30:44.060 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-10-17T01:30:44.061 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.061 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.061 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.061 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 3543 2023-10-17T01:30:44.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.062 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-10-17T01:30:44.062 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 271, 2023-10-17T01:30:44.062 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 1410, 2023-10-17T01:30:44.062 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 1139, 2023-10-17T01:30:44.063 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 24, 2023-10-17T01:30:44.063 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 76, 2023-10-17T01:30:44.063 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 52, 2023-10-17T01:30:44.063 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 315, 2023-10-17T01:30:44.063 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 1448, 2023-10-17T01:30:44.064 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 1133, 2023-10-17T01:30:44.064 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 0, 2023-10-17T01:30:44.064 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 0, 2023-10-17T01:30:44.064 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 0, 2023-10-17T01:30:44.064 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 54112, 2023-10-17T01:30:44.065 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 207156 2023-10-17T01:30:44.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.065 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-10-17T01:30:44.065 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 0, 2023-10-17T01:30:44.065 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2023-10-17T01:30:44.066 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 0, 2023-10-17T01:30:44.066 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 0, 2023-10-17T01:30:44.066 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2023-10-17T01:30:44.066 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-10-17T01:30:44.066 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.067 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.067 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.067 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-10-17T01:30:44.067 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.068 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.068 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.068 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:44.068 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.069 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.069 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.069 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-10-17T01:30:44.069 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.070 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.070 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.070 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-10-17T01:30:44.070 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.071 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.071 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.071 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:44.071 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.072 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.072 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.072 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-10-17T01:30:44.072 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.073 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.073 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.073 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-10-17T01:30:44.073 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.073 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.074 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.074 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:44.074 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.074 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.075 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.075 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:44.075 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.075 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.076 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.076 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-10-17T01:30:44.076 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.076 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.077 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.077 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-10-17T01:30:44.077 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.077 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.078 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-10-17T01:30:44.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.078 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.079 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-10-17T01:30:44.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.079 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.080 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-10-17T01:30:44.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.080 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.081 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-10-17T01:30:44.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.081 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.082 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-10-17T01:30:44.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.082 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.083 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:44.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.083 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.084 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-10-17T01:30:44.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.084 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.085 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:44.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.085 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.086 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:44.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.086 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.087 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-10-17T01:30:44.087 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.087 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.087 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.088 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:44.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.088 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.089 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:44.089 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.089 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.089 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.089 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-10-17T01:30:44.090 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.090 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.090 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.090 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-10-17T01:30:44.091 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.091 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.091 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.091 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-10-17T01:30:44.092 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.092 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.092 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.092 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-10-17T01:30:44.092 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.093 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.093 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:44.093 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:44.093 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:44.094 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.094 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.094 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.094 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.095 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-10-17T01:30:44.095 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-10-17T01:30:44.095 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 2, 2023-10-17T01:30:44.095 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 1, 2023-10-17T01:30:44.095 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2023-10-17T01:30:44.096 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-10-17T01:30:44.096 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 0, 2023-10-17T01:30:44.096 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 0, 2023-10-17T01:30:44.096 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 2813 2023-10-17T01:30:44.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.097 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-10-17T01:30:44.097 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:44.097 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:44.097 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:44.097 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:44.097 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:44.098 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:44.098 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:44.098 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:44.098 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:44.098 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:44.099 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:44.099 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:44.099 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:44.099 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:44.099 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:44.101 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:44.101 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:44.101 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:44.102 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:44.102 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:44.102 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:44.102 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:44.102 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-10-17T01:30:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:44.105 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:44.105 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:44.105 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:44.105 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 200299, 2023-10-17T01:30:44.105 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 306, 2023-10-17T01:30:44.105 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:44.106 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:44.106 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-10-17T01:30:44.106 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-10-17T01:30:44.106 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:44.106 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:44.107 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:44.107 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:44.107 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:44.107 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 156, 2023-10-17T01:30:44.107 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:44.108 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:44.108 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:44.108 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-10-17T01:30:44.108 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 845926, 2023-10-17T01:30:44.108 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 6364, 2023-10-17T01:30:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-10-17T01:30:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-10-17T01:30:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-10-17T01:30:44.110 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-10-17T01:30:44.110 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-10-17T01:30:44.110 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 3474, 2023-10-17T01:30:44.110 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 26, 2023-10-17T01:30:44.110 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 4, 2023-10-17T01:30:44.111 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 3474, 2023-10-17T01:30:44.111 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-10-17T01:30:44.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3474, 2023-10-17T01:30:44.111 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 14.347870474, 2023-10-17T01:30:44.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.004130072 2023-10-17T01:30:44.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.112 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-10-17T01:30:44.112 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-10-17T01:30:44.112 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3474, 2023-10-17T01:30:44.112 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 3475 2023-10-17T01:30:44.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.113 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 3474, 2023-10-17T01:30:44.113 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 3474, 2023-10-17T01:30:44.113 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 0, 2023-10-17T01:30:44.113 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-10-17T01:30:44.113 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-10-17T01:30:44.114 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 2065, 2023-10-17T01:30:44.114 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 0, 2023-10-17T01:30:44.114 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 1406, 2023-10-17T01:30:44.114 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 0, 2023-10-17T01:30:44.114 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 0, 2023-10-17T01:30:44.115 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-10-17T01:30:44.115 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-10-17T01:30:44.115 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-10-17T01:30:44.115 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2023-10-17T01:30:44.115 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 0, 2023-10-17T01:30:44.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-10-17T01:30:44.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:44.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:44.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 2, 2023-10-17T01:30:44.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 0, 2023-10-17T01:30:44.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:44.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:44.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:44.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-10-17T01:30:44.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-10-17T01:30:44.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-10-17T01:30:44.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:44.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-10-17T01:30:44.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:44.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2023-10-17T01:30:44.118 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-10-17T01:30:44.119 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-10-17T01:30:44.119 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-10-17T01:30:44.119 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-10-17T01:30:44.119 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-10-17T01:30:44.119 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-10-17T01:30:44.119 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-10-17T01:30:44.120 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-10-17T01:30:44.120 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-10-17T01:30:44.120 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-10-17T01:30:44.120 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-10-17T01:30:44.120 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-10-17T01:30:44.121 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-10-17T01:30:44.121 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-10-17T01:30:44.121 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-10-17T01:30:44.121 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-10-17T01:30:44.121 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 85, 2023-10-17T01:30:44.121 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-10-17T01:30:44.122 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 2, 2023-10-17T01:30:44.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 5, 2023-10-17T01:30:44.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 6, 2023-10-17T01:30:44.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 1, 2023-10-17T01:30:44.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-10-17T01:30:44.123 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 0, 2023-10-17T01:30:44.123 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 2, 2023-10-17T01:30:44.123 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 0 2023-10-17T01:30:44.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-10-17T01:30:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 0, 2023-10-17T01:30:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-10-17T01:30:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-10-17T01:30:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.125 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-10-17T01:30:44.125 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2023-10-17T01:30:44.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.125 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-10-17T01:30:44.125 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:44.126 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 0, 2023-10-17T01:30:44.126 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-10-17T01:30:44.126 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 0, 2023-10-17T01:30:44.126 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 0, 2023-10-17T01:30:44.126 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:44.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.127 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:44.127 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:44.127 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:44.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:44.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 3798, 2023-10-17T01:30:44.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 11848767, 2023-10-17T01:30:44.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:44.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 3798, 2023-10-17T01:30:44.128 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:44.128 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:44.129 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3798, 2023-10-17T01:30:44.129 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 11848767, 2023-10-17T01:30:44.129 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:44.129 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.129 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.129 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 3474, 2023-10-17T01:30:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 10610821, 2023-10-17T01:30:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3474, 2023-10-17T01:30:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 10610821, 2023-10-17T01:30:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:44.134 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:44.134 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-10-17T01:30:44.134 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:44.134 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:44.134 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 3474, 2023-10-17T01:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 3474, 2023-10-17T01:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3474, 2023-10-17T01:30:44.136 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 3474, 2023-10-17T01:30:44.136 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:44.136 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.136 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.136 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.137 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.137 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:44.137 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:44.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:44.138 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:44.138 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:44.138 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:44.138 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:44.138 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:44.139 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:44.139 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:44.139 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-10-17T01:30:44.139 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-10-17T01:30:44.139 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:44.140 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.140 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.140 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:44.141 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x55e73c510640": { 2023-10-17T01:30:44.141 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:44.141 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:44.141 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:44.141 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:44.142 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:44.142 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:44.142 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:44.142 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:44.142 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:44.143 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-10-17T01:30:44.143 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-10-17T01:30:44.143 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:44.143 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:44.143 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:44.144 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:44.144 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.144 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:44.144 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-10-17T01:30:44.450 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:44 smithi060 ceph-mon[134136]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-10-17T01:30:44.452 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-17T01:30:44.452 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-17T01:30:44.452 DEBUG:tasks.check_counter:Getting stats from h 2023-10-17T01:30:44.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-17T01:30:44.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:44 smithi045 ceph-mon[98474]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-10-17T01:30:44.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:44 smithi181 ceph-mon[134520]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-10-17T01:30:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:30:46.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:46 smithi045 ceph-mon[98474]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-17T01:30:46.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:46 smithi060 ceph-mon[134136]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-17T01:30:46.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:46 smithi181 ceph-mon[134520]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-17T01:30:47.325 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:47 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:30:47.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:47 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:30:47.584 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-10-17T01:30:47.585 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:47.585 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 3177708, 2023-10-17T01:30:47.585 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 3177204, 2023-10-17T01:30:47.586 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 515518060, 2023-10-17T01:30:47.586 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 503704282, 2023-10-17T01:30:47.586 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 10, 2023-10-17T01:30:47.586 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:47.586 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 152.066105982, 2023-10-17T01:30:47.587 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 61.079131524, 2023-10-17T01:30:47.587 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 111.548816690, 2023-10-17T01:30:47.587 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.018267985, 2023-10-17T01:30:47.587 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:47.587 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3177206, 2023-10-17T01:30:47.588 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 52.052635567, 2023-10-17T01:30:47.588 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000016383 2023-10-17T01:30:47.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.588 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:47.588 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 6353375, 2023-10-17T01:30:47.589 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.828816144, 2023-10-17T01:30:47.589 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000130 2023-10-17T01:30:47.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.589 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:30:47.589 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:30:47.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.590 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:47.590 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 535, 2023-10-17T01:30:47.590 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 506, 2023-10-17T01:30:47.590 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 802538, 2023-10-17T01:30:47.591 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 358494, 2023-10-17T01:30:47.591 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 11, 2023-10-17T01:30:47.591 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 8, 2023-10-17T01:30:47.591 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.055426953, 2023-10-17T01:30:47.591 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.018040018, 2023-10-17T01:30:47.592 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.030668633, 2023-10-17T01:30:47.592 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.009749913, 2023-10-17T01:30:47.592 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:47.592 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 506, 2023-10-17T01:30:47.592 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.223942207, 2023-10-17T01:30:47.593 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000442573 2023-10-17T01:30:47.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.593 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:47.593 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 160, 2023-10-17T01:30:47.593 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000054737, 2023-10-17T01:30:47.593 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000342 2023-10-17T01:30:47.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 654464, 2023-10-17T01:30:47.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 173312 2023-10-17T01:30:47.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.595 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:47.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1814, 2023-10-17T01:30:47.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2339, 2023-10-17T01:30:47.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 637182, 2023-10-17T01:30:47.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 1628166, 2023-10-17T01:30:47.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 16, 2023-10-17T01:30:47.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 8, 2023-10-17T01:30:47.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.135568153, 2023-10-17T01:30:47.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.052153694, 2023-10-17T01:30:47.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.098560722, 2023-10-17T01:30:47.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000777699, 2023-10-17T01:30:47.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:47.597 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2333, 2023-10-17T01:30:47.597 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.248583418, 2023-10-17T01:30:47.598 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000106550 2023-10-17T01:30:47.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.598 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:47.598 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3565, 2023-10-17T01:30:47.598 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000779949, 2023-10-17T01:30:47.599 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000218 2023-10-17T01:30:47.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.599 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-17T01:30:47.599 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 541440 2023-10-17T01:30:47.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.600 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-10-17T01:30:47.600 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-10-17T01:30:47.600 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-10-17T01:30:47.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.601 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-10-17T01:30:47.601 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:47.601 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:47.601 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 864, 2023-10-17T01:30:47.601 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.206566462, 2023-10-17T01:30:47.602 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000239081 2023-10-17T01:30:47.602 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.602 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:47.603 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:47.603 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:47.603 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2, 2023-10-17T01:30:47.603 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000012730, 2023-10-17T01:30:47.603 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000006365 2023-10-17T01:30:47.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.604 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-10-17T01:30:47.604 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 1170, 2023-10-17T01:30:47.604 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 464, 2023-10-17T01:30:47.605 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-10-17T01:30:47.605 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 464, 2023-10-17T01:30:47.605 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.318327224, 2023-10-17T01:30:47.605 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000686050 2023-10-17T01:30:47.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.606 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-10-17T01:30:47.606 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 706, 2023-10-17T01:30:47.606 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:47.606 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:47.607 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 11, 2023-10-17T01:30:47.607 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2023-10-17T01:30:47.607 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-10-17T01:30:47.607 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 258, 2023-10-17T01:30:47.608 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2023-10-17T01:30:47.608 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 258, 2023-10-17T01:30:47.608 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 0, 2023-10-17T01:30:47.608 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 257, 2023-10-17T01:30:47.608 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 0, 2023-10-17T01:30:47.608 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 239, 2023-10-17T01:30:47.609 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 239, 2023-10-17T01:30:47.609 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 8, 2023-10-17T01:30:47.609 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 1575, 2023-10-17T01:30:47.609 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 858, 2023-10-17T01:30:47.609 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 706, 2023-10-17T01:30:47.610 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 7, 2023-10-17T01:30:47.610 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:47.610 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:47.610 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 0, 2023-10-17T01:30:47.610 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 3781, 2023-10-17T01:30:47.610 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-10-17T01:30:47.611 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 1, 2023-10-17T01:30:47.611 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2023-10-17T01:30:47.611 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 6, 2023-10-17T01:30:47.611 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 241, 2023-10-17T01:30:47.611 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:47.612 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 88, 2023-10-17T01:30:47.612 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:47.612 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 1041, 2023-10-17T01:30:47.612 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 1692970308, 2023-10-17T01:30:47.612 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-10-17T01:30:47.612 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 88, 2023-10-17T01:30:47.613 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:47.613 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:47.613 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:47.613 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:30:47.614 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:47.614 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:30:47.614 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 88, 2023-10-17T01:30:47.614 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 2, 2023-10-17T01:30:47.614 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 5, 2023-10-17T01:30:47.615 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 186, 2023-10-17T01:30:47.615 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 2, 2023-10-17T01:30:47.615 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:47.615 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:47.615 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 345, 2023-10-17T01:30:47.616 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 244, 2023-10-17T01:30:47.616 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 9, 2023-10-17T01:30:47.616 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 7 2023-10-17T01:30:47.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.616 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-10-17T01:30:47.617 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-10-17T01:30:47.617 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:47.617 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-10-17T01:30:47.617 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 8, 2023-10-17T01:30:47.617 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 4, 2023-10-17T01:30:47.617 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-10-17T01:30:47.618 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:47.618 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:47.618 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 0, 2023-10-17T01:30:47.618 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 0, 2023-10-17T01:30:47.618 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:47.619 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-10-17T01:30:47.619 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:47.619 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:47.619 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:47.619 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-10-17T01:30:47.620 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-10-17T01:30:47.620 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:47.620 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 1, 2023-10-17T01:30:47.620 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-10-17T01:30:47.620 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:47.620 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:47.621 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-10-17T01:30:47.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.621 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-10-17T01:30:47.621 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 859, 2023-10-17T01:30:47.621 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2023-10-17T01:30:47.622 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2023-10-17T01:30:47.622 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 859, 2023-10-17T01:30:47.622 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-10-17T01:30:47.622 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2023-10-17T01:30:47.622 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 1, 2023-10-17T01:30:47.623 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2023-10-17T01:30:47.623 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2023-10-17T01:30:47.623 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 1, 2023-10-17T01:30:47.623 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-10-17T01:30:47.623 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2023-10-17T01:30:47.623 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2023-10-17T01:30:47.624 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 5357311, 2023-10-17T01:30:47.624 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 0, 2023-10-17T01:30:47.624 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-10-17T01:30:47.624 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 359, 2023-10-17T01:30:47.624 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.379686532, 2023-10-17T01:30:47.625 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.017770714 2023-10-17T01:30:47.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.625 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 0 2023-10-17T01:30:47.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-10-17T01:30:47.626 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 261, 2023-10-17T01:30:47.626 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 261, 2023-10-17T01:30:47.626 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 0, 2023-10-17T01:30:47.626 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 26, 2023-10-17T01:30:47.626 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 27, 2023-10-17T01:30:47.627 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 1, 2023-10-17T01:30:47.627 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 258, 2023-10-17T01:30:47.627 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 266, 2023-10-17T01:30:47.627 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 8, 2023-10-17T01:30:47.627 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 239, 2023-10-17T01:30:47.627 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 478, 2023-10-17T01:30:47.628 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 239, 2023-10-17T01:30:47.628 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 57944, 2023-10-17T01:30:47.628 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 198964 2023-10-17T01:30:47.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.628 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-10-17T01:30:47.629 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 1175, 2023-10-17T01:30:47.629 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 1, 2023-10-17T01:30:47.629 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 1170, 2023-10-17T01:30:47.629 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 13, 2023-10-17T01:30:47.629 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 3, 2023-10-17T01:30:47.630 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-10-17T01:30:47.630 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2023-10-17T01:30:47.630 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002955276, 2023-10-17T01:30:47.630 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002955276 2023-10-17T01:30:47.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.630 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-10-17T01:30:47.631 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.631 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.631 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.631 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:47.632 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.632 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.632 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.632 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-10-17T01:30:47.633 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.633 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.633 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.633 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-10-17T01:30:47.633 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 397, 2023-10-17T01:30:47.634 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.243999345, 2023-10-17T01:30:47.634 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000614607 2023-10-17T01:30:47.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.634 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:47.634 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.635 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.635 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.635 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-10-17T01:30:47.635 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.635 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.636 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.636 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-10-17T01:30:47.636 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.636 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.637 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.637 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:47.637 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.637 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.637 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.638 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:47.638 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.638 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.638 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.639 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-10-17T01:30:47.639 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.639 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.639 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.640 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-10-17T01:30:47.640 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.640 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.640 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.641 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-10-17T01:30:47.641 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.641 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.641 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.642 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-10-17T01:30:47.642 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.642 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.642 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.643 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-10-17T01:30:47.643 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 59, 2023-10-17T01:30:47.643 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.056094136, 2023-10-17T01:30:47.643 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000950748 2023-10-17T01:30:47.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.644 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-10-17T01:30:47.644 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 5, 2023-10-17T01:30:47.644 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007758945, 2023-10-17T01:30:47.644 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001551789 2023-10-17T01:30:47.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.644 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-10-17T01:30:47.645 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.645 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.645 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.645 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:47.646 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.646 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.646 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.646 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-10-17T01:30:47.647 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.647 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.647 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.647 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:47.647 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.648 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.648 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.648 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:47.648 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.648 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.649 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.649 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-10-17T01:30:47.649 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2, 2023-10-17T01:30:47.649 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007519522, 2023-10-17T01:30:47.649 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003759761 2023-10-17T01:30:47.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.650 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:47.650 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.650 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.650 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.651 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:47.651 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.651 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.651 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.652 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-10-17T01:30:47.652 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.652 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.652 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.653 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-10-17T01:30:47.653 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.653 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.653 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.654 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-10-17T01:30:47.654 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.654 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.654 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.655 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-10-17T01:30:47.655 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.655 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.655 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.656 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:47.656 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:47.656 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:47.656 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.656 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.656 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.657 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-10-17T01:30:47.657 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-10-17T01:30:47.657 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 1, 2023-10-17T01:30:47.658 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 0, 2023-10-17T01:30:47.658 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2023-10-17T01:30:47.658 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-10-17T01:30:47.658 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 51, 2023-10-17T01:30:47.658 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 51, 2023-10-17T01:30:47.658 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 2815 2023-10-17T01:30:47.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-10-17T01:30:47.659 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:47.659 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:47.659 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:47.660 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:47.660 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:47.660 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:47.660 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:47.660 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:47.660 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:47.661 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:47.661 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:47.661 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:47.661 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:47.661 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:47.662 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:47.662 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:47.662 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:47.662 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:47.662 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:47.663 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:47.663 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:47.663 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:47.663 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:47.663 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:47.663 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:47.664 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:47.664 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:47.664 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:47.664 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:47.664 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:47.664 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:47.665 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-10-17T01:30:47.665 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:47.665 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:47.665 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:47.665 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:47.666 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 299237, 2023-10-17T01:30:47.666 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 363, 2023-10-17T01:30:47.666 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:47.666 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:47.666 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-10-17T01:30:47.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-10-17T01:30:47.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:47.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:47.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:47.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:47.668 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:47.668 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 156, 2023-10-17T01:30:47.668 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:47.668 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:47.668 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:47.668 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-10-17T01:30:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 903187, 2023-10-17T01:30:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 9452, 2023-10-17T01:30:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-10-17T01:30:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:47.670 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-10-17T01:30:47.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.670 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-10-17T01:30:47.670 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-10-17T01:30:47.670 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-10-17T01:30:47.670 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 559, 2023-10-17T01:30:47.671 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 1199897, 2023-10-17T01:30:47.671 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 33, 2023-10-17T01:30:47.671 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 559, 2023-10-17T01:30:47.671 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-10-17T01:30:47.671 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 559, 2023-10-17T01:30:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 7.525931621, 2023-10-17T01:30:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.013463205 2023-10-17T01:30:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-10-17T01:30:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-10-17T01:30:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 559, 2023-10-17T01:30:47.673 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 604 2023-10-17T01:30:47.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.673 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 559, 2023-10-17T01:30:47.673 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 88, 2023-10-17T01:30:47.673 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 471, 2023-10-17T01:30:47.674 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-10-17T01:30:47.674 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-10-17T01:30:47.674 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 0, 2023-10-17T01:30:47.674 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 10, 2023-10-17T01:30:47.674 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 0, 2023-10-17T01:30:47.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 359, 2023-10-17T01:30:47.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 60, 2023-10-17T01:30:47.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-10-17T01:30:47.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-10-17T01:30:47.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-10-17T01:30:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2023-10-17T01:30:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 6, 2023-10-17T01:30:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-10-17T01:30:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 88, 2023-10-17T01:30:47.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 20, 2023-10-17T01:30:47.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:47.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:47.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:47.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-10-17T01:30:47.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-10-17T01:30:47.678 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-10-17T01:30:47.678 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:47.678 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-10-17T01:30:47.678 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:47.678 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 3, 2023-10-17T01:30:47.679 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-10-17T01:30:47.679 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-10-17T01:30:47.679 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-10-17T01:30:47.679 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-10-17T01:30:47.679 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-10-17T01:30:47.680 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-10-17T01:30:47.680 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-10-17T01:30:47.680 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-10-17T01:30:47.680 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-10-17T01:30:47.680 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-10-17T01:30:47.680 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-10-17T01:30:47.681 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-10-17T01:30:47.681 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-10-17T01:30:47.681 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-10-17T01:30:47.681 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-10-17T01:30:47.681 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-10-17T01:30:47.681 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 85, 2023-10-17T01:30:47.682 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-10-17T01:30:47.682 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 2, 2023-10-17T01:30:47.682 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 12, 2023-10-17T01:30:47.682 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 12, 2023-10-17T01:30:47.682 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2023-10-17T01:30:47.683 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-10-17T01:30:47.683 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 46, 2023-10-17T01:30:47.683 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 0, 2023-10-17T01:30:47.683 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 12 2023-10-17T01:30:47.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.684 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-10-17T01:30:47.684 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 1, 2023-10-17T01:30:47.684 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:47.684 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-10-17T01:30:47.684 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-10-17T01:30:47.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.685 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-10-17T01:30:47.685 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 1172 2023-10-17T01:30:47.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.685 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-10-17T01:30:47.686 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:47.686 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 0, 2023-10-17T01:30:47.686 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-10-17T01:30:47.686 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 0, 2023-10-17T01:30:47.686 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 0, 2023-10-17T01:30:47.687 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:47.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.687 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:47.687 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:47.687 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:47.688 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:47.688 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 3180055, 2023-10-17T01:30:47.688 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 230732062, 2023-10-17T01:30:47.688 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:47.688 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 3180055, 2023-10-17T01:30:47.689 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:47.689 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:47.689 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3180055, 2023-10-17T01:30:47.689 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 230732062, 2023-10-17T01:30:47.689 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:47.690 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.690 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.690 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.691 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:47.691 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:47.691 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:47.691 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:47.691 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:47.691 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:47.692 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:47.692 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:47.692 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 559, 2023-10-17T01:30:47.692 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 1199897, 2023-10-17T01:30:47.692 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 559, 2023-10-17T01:30:47.693 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 1199897, 2023-10-17T01:30:47.693 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:47.693 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.693 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.693 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.694 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.694 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:47.694 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:47.694 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-10-17T01:30:47.695 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:47.695 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:47.695 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:47.695 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:47.695 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:47.696 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 559, 2023-10-17T01:30:47.696 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 559, 2023-10-17T01:30:47.696 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 559, 2023-10-17T01:30:47.696 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 559, 2023-10-17T01:30:47.696 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:47.697 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.697 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.697 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.697 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:47.698 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:47.698 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:47.698 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:47.698 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:47.698 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:47.699 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:47.699 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:47.699 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:47.699 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:47.699 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-10-17T01:30:47.700 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-10-17T01:30:47.700 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:47.700 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.700 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.700 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:47.701 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x562ea25040a0": { 2023-10-17T01:30:47.701 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:47.701 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:47.701 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:47.702 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 859, 2023-10-17T01:30:47.702 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 1163007, 2023-10-17T01:30:47.702 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:47.702 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 859, 2023-10-17T01:30:47.702 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:47.703 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:47.703 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 359, 2023-10-17T01:30:47.703 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 1163007, 2023-10-17T01:30:47.703 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:47.703 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:47.704 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:47.704 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:47.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:47.704 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-10-17T01:30:47.705 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:47 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-17T01:30:48.060 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2023-10-17T01:30:48.060 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2023-10-17T01:30:48.060 DEBUG:tasks.check_counter:Getting stats from k 2023-10-17T01:30:48.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-17T01:30:48.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:48 smithi045 ceph-mon[98474]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-10-17T01:30:48.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:30:48.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:30:48.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:48 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:30:48.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:48 smithi060 ceph-mon[134136]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-10-17T01:30:48.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:30:48.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:30:48.579 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:48 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:30:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-17T01:30:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:48 smithi181 ceph-mon[134520]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-10-17T01:30:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-17T01:30:48.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-17T01:30:48.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:48 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' 2023-10-17T01:30:49.875 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-10-17T01:30:49.875 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:49.875 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1375, 2023-10-17T01:30:49.876 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 1352, 2023-10-17T01:30:49.876 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 983264, 2023-10-17T01:30:49.876 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 496134, 2023-10-17T01:30:49.876 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 5, 2023-10-17T01:30:49.876 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2023-10-17T01:30:49.877 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.147996815, 2023-10-17T01:30:49.877 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.046866831, 2023-10-17T01:30:49.877 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.044673425, 2023-10-17T01:30:49.877 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.047339470, 2023-10-17T01:30:49.877 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:49.878 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1351, 2023-10-17T01:30:49.878 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.529988131, 2023-10-17T01:30:49.878 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000392293 2023-10-17T01:30:49.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.878 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:49.879 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.879 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.879 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.880 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 630464, 2023-10-17T01:30:49.880 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 169584 2023-10-17T01:30:49.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.880 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:49.880 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 4, 2023-10-17T01:30:49.880 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 227, 2023-10-17T01:30:49.881 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 876, 2023-10-17T01:30:49.881 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 536924, 2023-10-17T01:30:49.881 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2023-10-17T01:30:49.881 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:49.881 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.012223193, 2023-10-17T01:30:49.882 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.007993563, 2023-10-17T01:30:49.882 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.002163061, 2023-10-17T01:30:49.882 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000079358, 2023-10-17T01:30:49.882 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:49.883 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 225, 2023-10-17T01:30:49.883 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.034510502, 2023-10-17T01:30:49.883 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000153380 2023-10-17T01:30:49.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.884 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:49.884 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.884 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.884 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.885 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-17T01:30:49.885 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 536240 2023-10-17T01:30:49.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.885 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:49.885 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2300, 2023-10-17T01:30:49.886 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2300, 2023-10-17T01:30:49.886 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 7170566, 2023-10-17T01:30:49.886 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 714511, 2023-10-17T01:30:49.886 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 5, 2023-10-17T01:30:49.886 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:49.887 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.163767349, 2023-10-17T01:30:49.887 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.033055733, 2023-10-17T01:30:49.887 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.048712603, 2023-10-17T01:30:49.887 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.072052266, 2023-10-17T01:30:49.887 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:49.888 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2299, 2023-10-17T01:30:49.888 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.493737147, 2023-10-17T01:30:49.888 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000214761 2023-10-17T01:30:49.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.889 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:49.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.889 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.890 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.890 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-17T01:30:49.891 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 448 2023-10-17T01:30:49.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.891 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-10-17T01:30:49.892 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-10-17T01:30:49.892 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-10-17T01:30:49.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.893 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-10-17T01:30:49.893 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:49.893 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:49.894 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 5539, 2023-10-17T01:30:49.894 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.145588448, 2023-10-17T01:30:49.894 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000026284 2023-10-17T01:30:49.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.895 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:49.895 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-10-17T01:30:49.896 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-10-17T01:30:49.896 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.896 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.896 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-10-17T01:30:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 0, 2023-10-17T01:30:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 0, 2023-10-17T01:30:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-10-17T01:30:49.898 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.898 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.898 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.899 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-10-17T01:30:49.899 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2023-10-17T01:30:49.899 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:49.899 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:49.899 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 0, 2023-10-17T01:30:49.899 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2023-10-17T01:30:49.900 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-10-17T01:30:49.900 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 1023, 2023-10-17T01:30:49.900 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 118, 2023-10-17T01:30:49.900 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 853, 2023-10-17T01:30:49.900 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 52, 2023-10-17T01:30:49.901 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 853, 2023-10-17T01:30:49.901 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 1, 2023-10-17T01:30:49.901 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 0, 2023-10-17T01:30:49.901 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 0, 2023-10-17T01:30:49.901 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 27, 2023-10-17T01:30:49.902 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 0, 2023-10-17T01:30:49.902 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 0, 2023-10-17T01:30:49.902 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2023-10-17T01:30:49.902 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2023-10-17T01:30:49.902 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:49.903 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:49.903 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 0, 2023-10-17T01:30:49.903 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2023-10-17T01:30:49.903 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-10-17T01:30:49.904 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2023-10-17T01:30:49.904 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2023-10-17T01:30:49.904 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2023-10-17T01:30:49.904 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2023-10-17T01:30:49.904 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:49.904 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2023-10-17T01:30:49.905 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:49.905 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2023-10-17T01:30:49.905 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2023-10-17T01:30:49.905 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-10-17T01:30:49.905 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2023-10-17T01:30:49.906 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:49.906 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:49.906 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:49.906 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:30:49.906 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:49.907 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:30:49.907 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2023-10-17T01:30:49.907 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:30:49.907 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 0, 2023-10-17T01:30:49.907 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 0, 2023-10-17T01:30:49.908 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:49.908 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:49.908 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:49.908 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 0, 2023-10-17T01:30:49.908 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 0, 2023-10-17T01:30:49.908 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 0, 2023-10-17T01:30:49.909 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 0 2023-10-17T01:30:49.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-10-17T01:30:49.909 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-10-17T01:30:49.910 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:49.910 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-10-17T01:30:49.910 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2023-10-17T01:30:49.910 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2023-10-17T01:30:49.910 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-10-17T01:30:49.911 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:49.911 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:49.911 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 0, 2023-10-17T01:30:49.911 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 0, 2023-10-17T01:30:49.911 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:49.912 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-10-17T01:30:49.912 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:49.912 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:49.912 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:49.912 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-10-17T01:30:49.913 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-10-17T01:30:49.913 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:49.913 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2023-10-17T01:30:49.913 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-10-17T01:30:49.913 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:49.914 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:49.914 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-10-17T01:30:49.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-10-17T01:30:49.914 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 0, 2023-10-17T01:30:49.915 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2023-10-17T01:30:49.915 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2023-10-17T01:30:49.915 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 3145, 2023-10-17T01:30:49.915 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-10-17T01:30:49.915 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2023-10-17T01:30:49.915 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 0, 2023-10-17T01:30:49.916 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2023-10-17T01:30:49.916 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2023-10-17T01:30:49.916 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 4, 2023-10-17T01:30:49.916 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-10-17T01:30:49.916 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2023-10-17T01:30:49.917 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2023-10-17T01:30:49.917 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 10778296, 2023-10-17T01:30:49.917 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 10776835, 2023-10-17T01:30:49.917 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-10-17T01:30:49.917 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.918 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.918 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.918 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 3145 2023-10-17T01:30:49.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.919 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-10-17T01:30:49.919 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 265, 2023-10-17T01:30:49.919 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 662, 2023-10-17T01:30:49.919 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 397, 2023-10-17T01:30:49.920 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 58, 2023-10-17T01:30:49.920 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 62, 2023-10-17T01:30:49.920 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 4, 2023-10-17T01:30:49.920 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 1023, 2023-10-17T01:30:49.920 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 1425, 2023-10-17T01:30:49.921 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 402, 2023-10-17T01:30:49.921 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 0, 2023-10-17T01:30:49.921 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 0, 2023-10-17T01:30:49.921 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 0, 2023-10-17T01:30:49.921 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 53556, 2023-10-17T01:30:49.921 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 207156 2023-10-17T01:30:49.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.922 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-10-17T01:30:49.922 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 0, 2023-10-17T01:30:49.922 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2023-10-17T01:30:49.922 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 0, 2023-10-17T01:30:49.923 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 0, 2023-10-17T01:30:49.923 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2023-10-17T01:30:49.923 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-10-17T01:30:49.923 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.923 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.924 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.924 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-10-17T01:30:49.924 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.924 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.924 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.925 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:49.925 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.925 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.925 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.926 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-10-17T01:30:49.926 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.926 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.926 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.927 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-10-17T01:30:49.927 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.927 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.927 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.928 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:49.928 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.928 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.928 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.929 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-10-17T01:30:49.929 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.929 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.929 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.930 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-10-17T01:30:49.930 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.930 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.930 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.931 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:49.931 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.931 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.931 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.932 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:49.932 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.932 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.932 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.933 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-10-17T01:30:49.933 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.933 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.933 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.934 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-10-17T01:30:49.934 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.934 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.934 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.935 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-10-17T01:30:49.935 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.935 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.935 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.936 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-10-17T01:30:49.936 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.936 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.936 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.937 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-10-17T01:30:49.937 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.937 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.937 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.938 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-10-17T01:30:49.938 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.938 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.938 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.939 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-10-17T01:30:49.939 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.939 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.939 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.939 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:49.940 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.940 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.940 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.940 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-10-17T01:30:49.941 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.941 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.941 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.941 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:49.942 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.942 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.942 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.943 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:49.943 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.943 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.943 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-10-17T01:30:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.946 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:49.946 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.946 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.946 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.947 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-10-17T01:30:49.947 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.947 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.947 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.947 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-10-17T01:30:49.948 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.948 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.948 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.948 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-10-17T01:30:49.949 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.949 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.949 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.949 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-10-17T01:30:49.950 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.950 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.950 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.950 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:49.951 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:49.951 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:49.951 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.951 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.951 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.952 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-10-17T01:30:49.952 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-10-17T01:30:49.952 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 2, 2023-10-17T01:30:49.952 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 1, 2023-10-17T01:30:49.953 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2023-10-17T01:30:49.953 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-10-17T01:30:49.953 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 0, 2023-10-17T01:30:49.953 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 0, 2023-10-17T01:30:49.953 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 2819 2023-10-17T01:30:49.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.954 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-10-17T01:30:49.954 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:49.954 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:49.954 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:49.954 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-10-17T01:30:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 191485, 2023-10-17T01:30:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 302, 2023-10-17T01:30:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-10-17T01:30:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-10-17T01:30:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 156, 2023-10-17T01:30:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-10-17T01:30:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 1406089, 2023-10-17T01:30:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 29781, 2023-10-17T01:30:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-10-17T01:30:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-10-17T01:30:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-10-17T01:30:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-10-17T01:30:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-10-17T01:30:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 3348, 2023-10-17T01:30:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 21, 2023-10-17T01:30:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 3, 2023-10-17T01:30:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 3348, 2023-10-17T01:30:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-10-17T01:30:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3348, 2023-10-17T01:30:49.968 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 12.497887366, 2023-10-17T01:30:49.968 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003732941 2023-10-17T01:30:49.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.968 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-10-17T01:30:49.968 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-10-17T01:30:49.969 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3348, 2023-10-17T01:30:49.969 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 3349 2023-10-17T01:30:49.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.969 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 3348, 2023-10-17T01:30:49.969 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 3348, 2023-10-17T01:30:49.970 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 0, 2023-10-17T01:30:49.970 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-10-17T01:30:49.970 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-10-17T01:30:49.970 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 2016, 2023-10-17T01:30:49.970 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 0, 2023-10-17T01:30:49.970 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 1330, 2023-10-17T01:30:49.971 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 0, 2023-10-17T01:30:49.971 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 0, 2023-10-17T01:30:49.971 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-10-17T01:30:49.971 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-10-17T01:30:49.972 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-10-17T01:30:49.972 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2023-10-17T01:30:49.972 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 0, 2023-10-17T01:30:49.972 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-10-17T01:30:49.972 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:49.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:49.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 1, 2023-10-17T01:30:49.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 0, 2023-10-17T01:30:49.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:49.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:49.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:49.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-10-17T01:30:49.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-10-17T01:30:49.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-10-17T01:30:49.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:49.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-10-17T01:30:49.975 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:49.975 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2023-10-17T01:30:49.975 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-10-17T01:30:49.975 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-10-17T01:30:49.975 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-10-17T01:30:49.976 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-10-17T01:30:49.976 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-10-17T01:30:49.976 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-10-17T01:30:49.976 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-10-17T01:30:49.977 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-10-17T01:30:49.977 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-10-17T01:30:49.977 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-10-17T01:30:49.977 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-10-17T01:30:49.977 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-10-17T01:30:49.978 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-10-17T01:30:49.978 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-10-17T01:30:49.978 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-10-17T01:30:49.978 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-10-17T01:30:49.978 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 85, 2023-10-17T01:30:49.978 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-10-17T01:30:49.979 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 2, 2023-10-17T01:30:49.979 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 4, 2023-10-17T01:30:49.979 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 6, 2023-10-17T01:30:49.979 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 2, 2023-10-17T01:30:49.979 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-10-17T01:30:49.980 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 0, 2023-10-17T01:30:49.980 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 2, 2023-10-17T01:30:49.980 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 0 2023-10-17T01:30:49.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.981 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-10-17T01:30:49.981 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 0, 2023-10-17T01:30:49.981 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:49.981 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-10-17T01:30:49.981 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-10-17T01:30:49.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.982 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-10-17T01:30:49.982 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2023-10-17T01:30:49.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.982 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-10-17T01:30:49.982 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:49.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 0, 2023-10-17T01:30:49.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-10-17T01:30:49.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 0, 2023-10-17T01:30:49.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 0, 2023-10-17T01:30:49.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:49.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.984 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:49.984 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:49.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:49.984 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:49.984 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 3675, 2023-10-17T01:30:49.985 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 7804360, 2023-10-17T01:30:49.985 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:49.985 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 3675, 2023-10-17T01:30:49.985 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:49.985 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:49.986 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3675, 2023-10-17T01:30:49.986 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 7804360, 2023-10-17T01:30:49.986 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:49.986 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.986 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.987 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.987 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:49.988 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:49.988 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-10-17T01:30:49.988 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:49.988 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:49.988 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:49.989 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:49.989 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:49.989 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 3348, 2023-10-17T01:30:49.989 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 6585206, 2023-10-17T01:30:49.989 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3348, 2023-10-17T01:30:49.990 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 6585206, 2023-10-17T01:30:49.990 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:49.990 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.990 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.990 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.991 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:49.991 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:49.991 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-10-17T01:30:49.992 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:49.992 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:49.992 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:49.992 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:49.992 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:49.993 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 3348, 2023-10-17T01:30:49.993 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 3348, 2023-10-17T01:30:49.993 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 3348, 2023-10-17T01:30:49.993 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 3348, 2023-10-17T01:30:49.994 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:49.994 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.994 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.994 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.995 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:49.995 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:49.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:49.995 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:49.996 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:49.996 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:49.996 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:49.996 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:49.996 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:49.997 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:49.997 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-10-17T01:30:49.997 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-10-17T01:30:49.997 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:49.997 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:49.997 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:49.998 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:49.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:49.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-10-17T01:30:49.998 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x56158f11e500": { 2023-10-17T01:30:49.998 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-10-17T01:30:49.999 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-10-17T01:30:49.999 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-10-17T01:30:49.999 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-10-17T01:30:49.999 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-10-17T01:30:49.999 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:50.000 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:50.000 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-10-17T01:30:50.000 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-10-17T01:30:50.000 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-10-17T01:30:50.000 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-10-17T01:30:50.001 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-10-17T01:30:50.001 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-10-17T01:30:50.001 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-10-17T01:30:50.001 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-10-17T01:30:50.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:50.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-10-17T01:30:50.002 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-10-17T01:30:50.316 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-10-17T01:30:50.316 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-10-17T01:30:50.317 DEBUG:tasks.check_counter:Getting stats from c 2023-10-17T01:30:50.317 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-17T01:30:50.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:50 smithi045 ceph-mon[98474]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-10-17T01:30:50.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:50 smithi060 ceph-mon[134136]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-10-17T01:30:50.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:50 smithi181 ceph-mon[134520]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-10-17T01:30:50.988 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:30:52.371 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-17T01:30:52.371 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:52.372 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3209010, 2023-10-17T01:30:52.372 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 3208738, 2023-10-17T01:30:52.372 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 533337287, 2023-10-17T01:30:52.372 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 508647572, 2023-10-17T01:30:52.372 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 10, 2023-10-17T01:30:52.373 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2023-10-17T01:30:52.373 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 102.608978700, 2023-10-17T01:30:52.373 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 40.765841962, 2023-10-17T01:30:52.373 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 72.646599570, 2023-10-17T01:30:52.373 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.027449405, 2023-10-17T01:30:52.373 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:52.374 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3208740, 2023-10-17T01:30:52.374 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 55.724077558, 2023-10-17T01:30:52.374 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000017366 2023-10-17T01:30:52.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.375 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:52.375 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 6416234, 2023-10-17T01:30:52.375 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.589201348, 2023-10-17T01:30:52.375 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000091 2023-10-17T01:30:52.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.376 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:30:52.376 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:30:52.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.376 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:52.377 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 497, 2023-10-17T01:30:52.377 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 470, 2023-10-17T01:30:52.377 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 782141, 2023-10-17T01:30:52.377 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 321016, 2023-10-17T01:30:52.378 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2023-10-17T01:30:52.378 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 9, 2023-10-17T01:30:52.378 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.049744454, 2023-10-17T01:30:52.378 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.016986061, 2023-10-17T01:30:52.379 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.026897417, 2023-10-17T01:30:52.379 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.008184633, 2023-10-17T01:30:52.379 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:52.379 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 469, 2023-10-17T01:30:52.379 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.170530525, 2023-10-17T01:30:52.380 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000363604 2023-10-17T01:30:52.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.380 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:52.380 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 169, 2023-10-17T01:30:52.380 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000034616, 2023-10-17T01:30:52.380 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000204 2023-10-17T01:30:52.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.381 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 646464, 2023-10-17T01:30:52.381 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 172752 2023-10-17T01:30:52.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.381 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:52.382 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1038, 2023-10-17T01:30:52.382 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1484, 2023-10-17T01:30:52.382 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 422068, 2023-10-17T01:30:52.382 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1412893, 2023-10-17T01:30:52.383 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 16, 2023-10-17T01:30:52.383 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 8, 2023-10-17T01:30:52.383 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.077543311, 2023-10-17T01:30:52.383 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.033430467, 2023-10-17T01:30:52.383 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.051088176, 2023-10-17T01:30:52.384 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000682134, 2023-10-17T01:30:52.384 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:52.384 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1479, 2023-10-17T01:30:52.384 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.255430610, 2023-10-17T01:30:52.384 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000172704 2023-10-17T01:30:52.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.385 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:52.385 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1904, 2023-10-17T01:30:52.385 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000336881, 2023-10-17T01:30:52.385 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000176 2023-10-17T01:30:52.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.386 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-10-17T01:30:52.386 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 542064 2023-10-17T01:30:52.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.386 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-10-17T01:30:52.387 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-10-17T01:30:52.387 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-10-17T01:30:52.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.387 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-10-17T01:30:52.387 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:30:52.387 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:30:52.388 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 757, 2023-10-17T01:30:52.388 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.126685680, 2023-10-17T01:30:52.388 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000167352 2023-10-17T01:30:52.388 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.389 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:52.389 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:30:52.389 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:30:52.389 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17, 2023-10-17T01:30:52.389 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000392146, 2023-10-17T01:30:52.389 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000023067 2023-10-17T01:30:52.390 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.390 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-10-17T01:30:52.390 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 379, 2023-10-17T01:30:52.390 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 378, 2023-10-17T01:30:52.391 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-10-17T01:30:52.391 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 378, 2023-10-17T01:30:52.391 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.416644426, 2023-10-17T01:30:52.391 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001102233 2023-10-17T01:30:52.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.392 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-10-17T01:30:52.392 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 1, 2023-10-17T01:30:52.392 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:52.392 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:52.392 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2023-10-17T01:30:52.393 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2023-10-17T01:30:52.393 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2023-10-17T01:30:52.393 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 230, 2023-10-17T01:30:52.393 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2023-10-17T01:30:52.393 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 230, 2023-10-17T01:30:52.394 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2023-10-17T01:30:52.394 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 228, 2023-10-17T01:30:52.394 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2023-10-17T01:30:52.394 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 210, 2023-10-17T01:30:52.394 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 210, 2023-10-17T01:30:52.395 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 9, 2023-10-17T01:30:52.395 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 735, 2023-10-17T01:30:52.395 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 658, 2023-10-17T01:30:52.395 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2023-10-17T01:30:52.395 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 4, 2023-10-17T01:30:52.396 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:52.396 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:52.396 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2023-10-17T01:30:52.396 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 401, 2023-10-17T01:30:52.396 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2023-10-17T01:30:52.396 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2023-10-17T01:30:52.397 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2023-10-17T01:30:52.397 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 6, 2023-10-17T01:30:52.397 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 215, 2023-10-17T01:30:52.397 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:52.398 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 72, 2023-10-17T01:30:52.398 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2023-10-17T01:30:52.398 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 1041, 2023-10-17T01:30:52.398 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 1692970308, 2023-10-17T01:30:52.399 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2023-10-17T01:30:52.399 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 72, 2023-10-17T01:30:52.399 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:52.399 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:52.399 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:52.400 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:30:52.400 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:52.400 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:30:52.400 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 72, 2023-10-17T01:30:52.400 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:30:52.401 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 2, 2023-10-17T01:30:52.401 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 73, 2023-10-17T01:30:52.401 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:52.401 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:52.401 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:52.402 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 299, 2023-10-17T01:30:52.402 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 250, 2023-10-17T01:30:52.402 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 8, 2023-10-17T01:30:52.402 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 9 2023-10-17T01:30:52.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.403 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-10-17T01:30:52.403 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2023-10-17T01:30:52.403 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:52.403 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2023-10-17T01:30:52.403 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 4, 2023-10-17T01:30:52.403 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 6, 2023-10-17T01:30:52.404 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2023-10-17T01:30:52.404 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:52.404 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:52.404 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 2, 2023-10-17T01:30:52.405 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 2, 2023-10-17T01:30:52.405 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:52.405 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-10-17T01:30:52.405 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:52.405 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:52.406 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:52.406 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-10-17T01:30:52.406 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-10-17T01:30:52.406 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:52.406 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2023-10-17T01:30:52.407 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-10-17T01:30:52.407 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:52.407 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:52.407 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-10-17T01:30:52.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.408 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-10-17T01:30:52.408 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 930, 2023-10-17T01:30:52.408 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2023-10-17T01:30:52.408 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2023-10-17T01:30:52.409 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 930, 2023-10-17T01:30:52.409 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-10-17T01:30:52.409 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2023-10-17T01:30:52.409 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 1, 2023-10-17T01:30:52.409 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2023-10-17T01:30:52.410 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2023-10-17T01:30:52.410 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2023-10-17T01:30:52.410 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-10-17T01:30:52.410 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2023-10-17T01:30:52.410 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2023-10-17T01:30:52.411 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 5293441, 2023-10-17T01:30:52.411 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2023-10-17T01:30:52.411 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-10-17T01:30:52.412 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 317, 2023-10-17T01:30:52.412 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3.295116896, 2023-10-17T01:30:52.412 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.010394690 2023-10-17T01:30:52.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.413 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2023-10-17T01:30:52.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.413 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-10-17T01:30:52.413 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 229, 2023-10-17T01:30:52.413 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 231, 2023-10-17T01:30:52.414 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2023-10-17T01:30:52.414 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 25, 2023-10-17T01:30:52.414 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 26, 2023-10-17T01:30:52.414 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2023-10-17T01:30:52.414 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 230, 2023-10-17T01:30:52.415 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 237, 2023-10-17T01:30:52.415 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 7, 2023-10-17T01:30:52.415 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 210, 2023-10-17T01:30:52.416 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 407, 2023-10-17T01:30:52.416 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 197, 2023-10-17T01:30:52.416 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 54016, 2023-10-17T01:30:52.416 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 198964 2023-10-17T01:30:52.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.417 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-10-17T01:30:52.417 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 384, 2023-10-17T01:30:52.417 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2023-10-17T01:30:52.418 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 379, 2023-10-17T01:30:52.418 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 13, 2023-10-17T01:30:52.418 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2023-10-17T01:30:52.418 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-10-17T01:30:52.418 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.419 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.419 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-10-17T01:30:52.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2023-10-17T01:30:52.419 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001146683, 2023-10-17T01:30:52.420 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001146683 2023-10-17T01:30:52.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.420 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:52.420 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.420 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.421 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.421 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-10-17T01:30:52.421 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.422 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.422 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.422 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-10-17T01:30:52.422 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 351, 2023-10-17T01:30:52.423 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.377449162, 2023-10-17T01:30:52.423 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001075353 2023-10-17T01:30:52.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.423 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:52.423 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.424 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.424 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.424 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-10-17T01:30:52.424 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.425 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.425 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.425 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-10-17T01:30:52.426 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.426 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.426 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.427 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:52.427 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.427 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.427 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.428 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:52.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.428 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.428 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-10-17T01:30:52.429 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.429 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.429 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.429 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-10-17T01:30:52.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.430 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.430 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-10-17T01:30:52.431 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.431 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.431 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.431 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-10-17T01:30:52.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.432 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.432 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-10-17T01:30:52.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 18, 2023-10-17T01:30:52.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.029872719, 2023-10-17T01:30:52.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001659595 2023-10-17T01:30:52.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.434 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-10-17T01:30:52.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 6, 2023-10-17T01:30:52.434 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.004660603, 2023-10-17T01:30:52.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000776767 2023-10-17T01:30:52.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.435 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-10-17T01:30:52.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.436 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.436 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:52.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.437 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.438 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-10-17T01:30:52.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.438 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.439 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:52.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.439 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.440 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:52.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.440 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.441 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-10-17T01:30:52.441 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.441 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.442 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:52.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.443 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.443 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:52.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.444 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.444 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-10-17T01:30:52.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.444 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.445 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-10-17T01:30:52.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.446 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-10-17T01:30:52.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.447 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.447 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.447 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-10-17T01:30:52.448 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2023-10-17T01:30:52.448 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.003515259, 2023-10-17T01:30:52.448 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001757629 2023-10-17T01:30:52.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.448 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:52.448 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:52.449 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:52.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.449 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.449 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-10-17T01:30:52.450 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2023-10-17T01:30:52.450 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2023-10-17T01:30:52.450 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2023-10-17T01:30:52.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2023-10-17T01:30:52.451 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-10-17T01:30:52.451 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 42, 2023-10-17T01:30:52.451 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 42, 2023-10-17T01:30:52.451 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 965 2023-10-17T01:30:52.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-10-17T01:30:52.452 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:52.452 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:52.453 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:52.453 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:52.453 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:52.454 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:52.454 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:52.454 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:52.454 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:52.454 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:52.455 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:52.455 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:52.455 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:52.456 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:52.456 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:52.456 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:52.457 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:52.457 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:52.457 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:52.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:52.458 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:52.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:52.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:52.459 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:52.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:52.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:52.460 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:52.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:52.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:52.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:52.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:52.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-10-17T01:30:52.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:52.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:52.462 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:52.463 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:52.463 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 294717, 2023-10-17T01:30:52.463 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 331, 2023-10-17T01:30:52.464 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:30:52.464 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2023-10-17T01:30:52.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-10-17T01:30:52.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-10-17T01:30:52.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:52.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:52.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:52.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:52.465 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:52.466 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 156, 2023-10-17T01:30:52.466 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:52.466 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:52.466 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:52.466 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-10-17T01:30:52.467 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 798249, 2023-10-17T01:30:52.467 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 8277, 2023-10-17T01:30:52.467 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:52.467 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-10-17T01:30:52.468 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:52.468 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-10-17T01:30:52.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.468 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-10-17T01:30:52.469 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-10-17T01:30:52.469 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-10-17T01:30:52.469 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 512, 2023-10-17T01:30:52.469 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 1133140, 2023-10-17T01:30:52.470 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 27, 2023-10-17T01:30:52.470 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 512, 2023-10-17T01:30:52.470 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-10-17T01:30:52.470 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 512, 2023-10-17T01:30:52.470 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 4.398921129, 2023-10-17T01:30:52.471 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.008591642 2023-10-17T01:30:52.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.471 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-10-17T01:30:52.472 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-10-17T01:30:52.472 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 512, 2023-10-17T01:30:52.472 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 558 2023-10-17T01:30:52.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.473 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 512, 2023-10-17T01:30:52.473 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 73, 2023-10-17T01:30:52.473 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 439, 2023-10-17T01:30:52.473 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-10-17T01:30:52.474 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-10-17T01:30:52.474 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2023-10-17T01:30:52.474 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2023-10-17T01:30:52.475 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1, 2023-10-17T01:30:52.475 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 318, 2023-10-17T01:30:52.475 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 61, 2023-10-17T01:30:52.475 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-10-17T01:30:52.475 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-10-17T01:30:52.476 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-10-17T01:30:52.476 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2023-10-17T01:30:52.476 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 14, 2023-10-17T01:30:52.476 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-10-17T01:30:52.476 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:52.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:52.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 72, 2023-10-17T01:30:52.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2023-10-17T01:30:52.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:52.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:52.477 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:52.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-10-17T01:30:52.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-10-17T01:30:52.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-10-17T01:30:52.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:52.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-10-17T01:30:52.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:52.479 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-10-17T01:30:52.479 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-10-17T01:30:52.479 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-10-17T01:30:52.480 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-10-17T01:30:52.480 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-10-17T01:30:52.480 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-10-17T01:30:52.480 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-10-17T01:30:52.480 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-10-17T01:30:52.481 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-10-17T01:30:52.481 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-10-17T01:30:52.481 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-10-17T01:30:52.481 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-10-17T01:30:52.481 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-10-17T01:30:52.482 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-10-17T01:30:52.482 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-10-17T01:30:52.482 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-10-17T01:30:52.482 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-10-17T01:30:52.482 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 85, 2023-10-17T01:30:52.482 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-10-17T01:30:52.483 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2023-10-17T01:30:52.483 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2023-10-17T01:30:52.483 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2023-10-17T01:30:52.483 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2023-10-17T01:30:52.483 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-10-17T01:30:52.484 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 50, 2023-10-17T01:30:52.484 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2023-10-17T01:30:52.484 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 12 2023-10-17T01:30:52.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.484 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-10-17T01:30:52.485 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2023-10-17T01:30:52.485 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:52.485 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-10-17T01:30:52.485 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-10-17T01:30:52.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-10-17T01:30:52.486 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 379 2023-10-17T01:30:52.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.486 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-10-17T01:30:52.486 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:52.486 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 2, 2023-10-17T01:30:52.487 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-10-17T01:30:52.487 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 2, 2023-10-17T01:30:52.487 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 2, 2023-10-17T01:30:52.487 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:52.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.488 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:52.488 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:52.488 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:30:52.488 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:52.488 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 3210542, 2023-10-17T01:30:52.488 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 245572058, 2023-10-17T01:30:52.489 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:52.489 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 3210542, 2023-10-17T01:30:52.489 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:30:52.489 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:30:52.490 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3210542, 2023-10-17T01:30:52.490 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 245572058, 2023-10-17T01:30:52.490 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:52.490 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.491 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.491 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.492 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:52.493 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:52.493 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:30:52.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:52.493 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:30:52.494 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:30:52.494 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:52.494 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:52.495 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 512, 2023-10-17T01:30:52.495 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1133342, 2023-10-17T01:30:52.495 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 512, 2023-10-17T01:30:52.495 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1133342, 2023-10-17T01:30:52.496 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:52.496 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.496 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.497 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.497 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:52.497 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:52.497 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-10-17T01:30:52.498 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:52.498 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:30:52.498 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:30:52.499 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:52.499 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:52.499 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 512, 2023-10-17T01:30:52.499 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 512, 2023-10-17T01:30:52.500 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 512, 2023-10-17T01:30:52.500 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 512, 2023-10-17T01:30:52.500 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:52.500 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.501 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.501 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.502 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:52.502 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:52.502 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:30:52.503 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:52.503 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2, 2023-10-17T01:30:52.503 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 202, 2023-10-17T01:30:52.503 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:52.503 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2, 2023-10-17T01:30:52.504 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:30:52.504 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:30:52.504 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1, 2023-10-17T01:30:52.504 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 202, 2023-10-17T01:30:52.504 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:52.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.505 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.505 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:52.506 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55900c04a0a0": { 2023-10-17T01:30:52.506 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:52.506 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:30:52.506 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:52.507 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 930, 2023-10-17T01:30:52.507 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1099137, 2023-10-17T01:30:52.507 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:52.507 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 930, 2023-10-17T01:30:52.507 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:30:52.508 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:30:52.508 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 317, 2023-10-17T01:30:52.508 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1099137, 2023-10-17T01:30:52.508 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:52.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:52.509 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:52.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:52.509 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.509 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:52.509 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-17T01:30:52.510 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:52 smithi181 ceph-mon[134520]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-10-17T01:30:52.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:52 smithi045 ceph-mon[98474]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-10-17T01:30:52.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:52 smithi060 ceph-mon[134136]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-10-17T01:30:52.864 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-10-17T01:30:52.864 INFO:tasks.check_counter:Daemon mds.c mds.imported=6 2023-10-17T01:30:52.864 DEBUG:tasks.check_counter:Getting stats from f 2023-10-17T01:30:52.865 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-17T01:30:53.468 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:30:54.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:54 smithi045 ceph-mon[98474]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-10-17T01:30:54.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:54 smithi060 ceph-mon[134136]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-10-17T01:30:54.692 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:54 smithi181 ceph-mon[134520]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-10-17T01:30:56.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:56 smithi060 ceph-mon[134136]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-10-17T01:30:56.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:56 smithi181 ceph-mon[134520]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-10-17T01:30:56.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:56 smithi045 ceph-mon[98474]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-10-17T01:30:57.015 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-17T01:30:57.015 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:30:57.015 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2785, 2023-10-17T01:30:57.015 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2521, 2023-10-17T01:30:57.016 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1009212, 2023-10-17T01:30:57.016 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 3450038, 2023-10-17T01:30:57.016 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2023-10-17T01:30:57.016 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2023-10-17T01:30:57.016 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.174006664, 2023-10-17T01:30:57.017 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.060885079, 2023-10-17T01:30:57.017 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.096763574, 2023-10-17T01:30:57.017 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.025940306, 2023-10-17T01:30:57.017 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:57.017 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2519, 2023-10-17T01:30:57.018 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.459495646, 2023-10-17T01:30:57.018 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000182411 2023-10-17T01:30:57.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.018 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:57.018 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3569, 2023-10-17T01:30:57.019 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000679659, 2023-10-17T01:30:57.019 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000190 2023-10-17T01:30:57.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.019 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:30:57.019 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:30:57.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.020 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:30:57.020 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2503, 2023-10-17T01:30:57.020 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 3301, 2023-10-17T01:30:57.020 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1397995, 2023-10-17T01:30:57.021 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 1830544, 2023-10-17T01:30:57.021 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 14, 2023-10-17T01:30:57.021 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2023-10-17T01:30:57.021 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.175620157, 2023-10-17T01:30:57.021 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.068349772, 2023-10-17T01:30:57.022 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.114770300, 2023-10-17T01:30:57.022 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.010191673, 2023-10-17T01:30:57.022 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:57.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3300, 2023-10-17T01:30:57.022 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.387496542, 2023-10-17T01:30:57.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000117423 2023-10-17T01:30:57.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.023 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:57.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4249, 2023-10-17T01:30:57.024 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000745228, 2023-10-17T01:30:57.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000175 2023-10-17T01:30:57.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.025 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 641968, 2023-10-17T01:30:57.025 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 174800 2023-10-17T01:30:57.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.025 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:30:57.025 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 466, 2023-10-17T01:30:57.026 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 686, 2023-10-17T01:30:57.026 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 222629, 2023-10-17T01:30:57.026 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 852759, 2023-10-17T01:30:57.026 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 12, 2023-10-17T01:30:57.026 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2023-10-17T01:30:57.027 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.046229637, 2023-10-17T01:30:57.027 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.018801530, 2023-10-17T01:30:57.027 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.015243089, 2023-10-17T01:30:57.027 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.010013492, 2023-10-17T01:30:57.027 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:30:57.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 685, 2023-10-17T01:30:57.028 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.181356201, 2023-10-17T01:30:57.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000264753 2023-10-17T01:30:57.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.028 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:30:57.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 175, 2023-10-17T01:30:57.029 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000039925, 2023-10-17T01:30:57.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000228 2023-10-17T01:30:57.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.029 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-10-17T01:30:57.030 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 544016 2023-10-17T01:30:57.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.030 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-10-17T01:30:57.030 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-10-17T01:30:57.030 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-10-17T01:30:57.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.031 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-10-17T01:30:57.031 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:30:57.031 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:30:57.031 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 942, 2023-10-17T01:30:57.032 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.548335834, 2023-10-17T01:30:57.032 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000582097 2023-10-17T01:30:57.032 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.032 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-10-17T01:30:57.033 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:30:57.033 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:30:57.033 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 963, 2023-10-17T01:30:57.033 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.033763449, 2023-10-17T01:30:57.033 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000035060 2023-10-17T01:30:57.033 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.034 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-10-17T01:30:57.034 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 916, 2023-10-17T01:30:57.034 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 915, 2023-10-17T01:30:57.034 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-10-17T01:30:57.035 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 915, 2023-10-17T01:30:57.035 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 22.398702002, 2023-10-17T01:30:57.035 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.024479455 2023-10-17T01:30:57.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.035 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-10-17T01:30:57.036 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 1, 2023-10-17T01:30:57.036 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2023-10-17T01:30:57.036 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-10-17T01:30:57.036 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2023-10-17T01:30:57.036 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2023-10-17T01:30:57.036 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2023-10-17T01:30:57.037 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 167, 2023-10-17T01:30:57.037 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2023-10-17T01:30:57.037 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 0, 2023-10-17T01:30:57.037 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 167, 2023-10-17T01:30:57.037 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 167, 2023-10-17T01:30:57.038 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 6, 2023-10-17T01:30:57.038 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 126, 2023-10-17T01:30:57.038 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 126, 2023-10-17T01:30:57.038 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 9, 2023-10-17T01:30:57.038 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 1749, 2023-10-17T01:30:57.039 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 1367, 2023-10-17T01:30:57.039 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2023-10-17T01:30:57.039 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 7, 2023-10-17T01:30:57.039 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:30:57.039 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-10-17T01:30:57.039 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 92, 2023-10-17T01:30:57.040 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 323, 2023-10-17T01:30:57.040 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2023-10-17T01:30:57.040 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 1, 2023-10-17T01:30:57.040 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 6, 2023-10-17T01:30:57.040 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 7, 2023-10-17T01:30:57.040 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 127, 2023-10-17T01:30:57.041 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-10-17T01:30:57.041 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 76, 2023-10-17T01:30:57.041 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 1, 2023-10-17T01:30:57.041 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 1041, 2023-10-17T01:30:57.041 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 1692970308, 2023-10-17T01:30:57.042 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2023-10-17T01:30:57.042 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 76, 2023-10-17T01:30:57.042 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-10-17T01:30:57.042 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:30:57.042 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:30:57.043 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 2, 2023-10-17T01:30:57.043 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:30:57.043 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 2, 2023-10-17T01:30:57.043 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 74, 2023-10-17T01:30:57.043 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:30:57.043 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 33, 2023-10-17T01:30:57.044 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 731, 2023-10-17T01:30:57.044 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:30:57.044 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:30:57.044 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:30:57.044 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 501, 2023-10-17T01:30:57.044 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 258, 2023-10-17T01:30:57.045 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 160, 2023-10-17T01:30:57.045 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 1022 2023-10-17T01:30:57.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.045 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-10-17T01:30:57.045 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2023-10-17T01:30:57.046 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2023-10-17T01:30:57.046 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2023-10-17T01:30:57.046 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 7, 2023-10-17T01:30:57.046 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 8, 2023-10-17T01:30:57.046 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2023-10-17T01:30:57.047 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-10-17T01:30:57.047 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:30:57.047 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 246, 2023-10-17T01:30:57.047 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 246, 2023-10-17T01:30:57.047 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2023-10-17T01:30:57.047 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-10-17T01:30:57.048 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-10-17T01:30:57.048 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:30:57.048 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:30:57.048 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-10-17T01:30:57.048 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-10-17T01:30:57.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:30:57.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 1, 2023-10-17T01:30:57.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-10-17T01:30:57.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:30:57.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-10-17T01:30:57.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-10-17T01:30:57.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.050 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-10-17T01:30:57.050 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 1581, 2023-10-17T01:30:57.050 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2023-10-17T01:30:57.050 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2023-10-17T01:30:57.051 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 1581, 2023-10-17T01:30:57.051 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-10-17T01:30:57.051 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2023-10-17T01:30:57.051 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 2, 2023-10-17T01:30:57.051 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2023-10-17T01:30:57.052 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2023-10-17T01:30:57.052 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 2, 2023-10-17T01:30:57.052 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-10-17T01:30:57.052 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2023-10-17T01:30:57.052 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2023-10-17T01:30:57.053 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 6686758, 2023-10-17T01:30:57.053 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2023-10-17T01:30:57.053 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-10-17T01:30:57.053 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 339, 2023-10-17T01:30:57.053 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3.017144945, 2023-10-17T01:30:57.054 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.008900132 2023-10-17T01:30:57.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.054 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2023-10-17T01:30:57.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.054 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-10-17T01:30:57.054 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 146, 2023-10-17T01:30:57.055 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 392, 2023-10-17T01:30:57.055 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 246, 2023-10-17T01:30:57.055 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 28, 2023-10-17T01:30:57.055 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 29, 2023-10-17T01:30:57.055 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 1, 2023-10-17T01:30:57.056 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 167, 2023-10-17T01:30:57.056 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 419, 2023-10-17T01:30:57.056 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 252, 2023-10-17T01:30:57.056 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 126, 2023-10-17T01:30:57.056 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 458, 2023-10-17T01:30:57.056 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 332, 2023-10-17T01:30:57.057 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 53564, 2023-10-17T01:30:57.057 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 198964 2023-10-17T01:30:57.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.057 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-10-17T01:30:57.057 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 1266, 2023-10-17T01:30:57.058 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2023-10-17T01:30:57.058 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 916, 2023-10-17T01:30:57.058 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 50, 2023-10-17T01:30:57.058 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 82, 2023-10-17T01:30:57.058 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-10-17T01:30:57.059 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 243, 2023-10-17T01:30:57.059 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.502671221, 2023-10-17T01:30:57.059 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002068605 2023-10-17T01:30:57.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.059 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-10-17T01:30:57.059 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2023-10-17T01:30:57.060 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002033972, 2023-10-17T01:30:57.060 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002033972 2023-10-17T01:30:57.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.060 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-10-17T01:30:57.060 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.061 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.061 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.061 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.061 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-10-17T01:30:57.062 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.062 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.062 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.062 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-10-17T01:30:57.063 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 270, 2023-10-17T01:30:57.063 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 21.273938887, 2023-10-17T01:30:57.063 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.078792366 2023-10-17T01:30:57.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.063 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-10-17T01:30:57.064 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.064 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.064 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.064 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.064 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-10-17T01:30:57.064 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.065 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.065 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.065 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-10-17T01:30:57.065 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.066 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.066 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.066 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-10-17T01:30:57.066 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-10-17T01:30:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.067 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.068 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.068 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-10-17T01:30:57.068 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.068 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.069 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.069 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-10-17T01:30:57.069 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.069 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.069 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.070 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-10-17T01:30:57.070 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.070 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.070 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.071 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-10-17T01:30:57.071 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.071 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.071 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.072 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-10-17T01:30:57.072 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 133, 2023-10-17T01:30:57.072 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.058745906, 2023-10-17T01:30:57.072 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000441698 2023-10-17T01:30:57.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.073 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-10-17T01:30:57.073 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 22, 2023-10-17T01:30:57.073 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.013940507, 2023-10-17T01:30:57.073 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000633659 2023-10-17T01:30:57.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.074 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-10-17T01:30:57.074 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 221, 2023-10-17T01:30:57.074 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.439351175, 2023-10-17T01:30:57.074 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001988014 2023-10-17T01:30:57.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.075 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-10-17T01:30:57.075 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.075 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.075 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.075 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-10-17T01:30:57.076 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.076 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.076 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.077 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-10-17T01:30:57.077 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.077 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.077 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.078 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-10-17T01:30:57.078 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.078 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.078 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.079 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-10-17T01:30:57.079 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.079 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.079 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.080 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-10-17T01:30:57.080 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.080 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.080 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.080 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-10-17T01:30:57.081 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.081 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.081 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.081 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-10-17T01:30:57.082 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.082 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.082 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.082 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-10-17T01:30:57.083 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.083 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.083 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.083 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.083 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-10-17T01:30:57.084 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.084 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.084 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.084 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-10-17T01:30:57.084 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 25, 2023-10-17T01:30:57.085 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.108020334, 2023-10-17T01:30:57.085 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004320813 2023-10-17T01:30:57.085 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.085 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:30:57.085 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:30:57.085 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-10-17T01:30:57.086 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.086 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.086 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.086 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.086 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.087 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-10-17T01:30:57.087 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2023-10-17T01:30:57.087 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2023-10-17T01:30:57.087 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2023-10-17T01:30:57.087 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2023-10-17T01:30:57.088 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-10-17T01:30:57.088 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 101, 2023-10-17T01:30:57.088 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 101, 2023-10-17T01:30:57.088 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 964 2023-10-17T01:30:57.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.089 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-10-17T01:30:57.089 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:30:57.089 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-10-17T01:30:57.089 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:30:57.089 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:30:57.089 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:30:57.090 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:30:57.090 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:30:57.090 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:30:57.090 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:30:57.090 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:30:57.091 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:30:57.091 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:30:57.091 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:30:57.091 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:30:57.091 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:30:57.092 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-10-17T01:30:57.092 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:30:57.092 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-10-17T01:30:57.092 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:30:57.092 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:30:57.093 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:30:57.093 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:30:57.093 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:30:57.093 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:30:57.093 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:30:57.094 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-10-17T01:30:57.094 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:30:57.094 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:30:57.094 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:30:57.094 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-10-17T01:30:57.095 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-10-17T01:30:57.095 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-10-17T01:30:57.095 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:30:57.095 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:30:57.095 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:30:57.095 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:30:57.096 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 333500, 2023-10-17T01:30:57.096 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 266, 2023-10-17T01:30:57.096 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 528, 2023-10-17T01:30:57.096 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 6, 2023-10-17T01:30:57.096 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-10-17T01:30:57.097 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-10-17T01:30:57.097 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:30:57.097 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-10-17T01:30:57.097 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:30:57.097 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-10-17T01:30:57.097 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 7432, 2023-10-17T01:30:57.098 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 156, 2023-10-17T01:30:57.098 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:30:57.098 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:30:57.098 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-10-17T01:30:57.098 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-10-17T01:30:57.099 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 531209, 2023-10-17T01:30:57.099 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 4931, 2023-10-17T01:30:57.099 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-10-17T01:30:57.099 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-10-17T01:30:57.099 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-10-17T01:30:57.099 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-10-17T01:30:57.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.100 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-10-17T01:30:57.100 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-10-17T01:30:57.100 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-10-17T01:30:57.100 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 1240, 2023-10-17T01:30:57.101 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 2587236, 2023-10-17T01:30:57.101 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2023-10-17T01:30:57.101 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 1240, 2023-10-17T01:30:57.101 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-10-17T01:30:57.101 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1240, 2023-10-17T01:30:57.102 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 8.771843716, 2023-10-17T01:30:57.102 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.007074067 2023-10-17T01:30:57.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.102 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-10-17T01:30:57.102 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-10-17T01:30:57.103 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1240, 2023-10-17T01:30:57.103 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1439 2023-10-17T01:30:57.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.103 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 1240, 2023-10-17T01:30:57.103 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 194, 2023-10-17T01:30:57.103 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 1046, 2023-10-17T01:30:57.104 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-10-17T01:30:57.104 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-10-17T01:30:57.104 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2023-10-17T01:30:57.104 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2023-10-17T01:30:57.104 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 118, 2023-10-17T01:30:57.105 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 457, 2023-10-17T01:30:57.105 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 158, 2023-10-17T01:30:57.105 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-10-17T01:30:57.105 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-10-17T01:30:57.105 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-10-17T01:30:57.105 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2023-10-17T01:30:57.106 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 258, 2023-10-17T01:30:57.106 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-10-17T01:30:57.106 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-10-17T01:30:57.106 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-10-17T01:30:57.106 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 76, 2023-10-17T01:30:57.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2023-10-17T01:30:57.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:30:57.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-10-17T01:30:57.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:30:57.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-10-17T01:30:57.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-10-17T01:30:57.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-10-17T01:30:57.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:30:57.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-10-17T01:30:57.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:30:57.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-10-17T01:30:57.109 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-10-17T01:30:57.109 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-10-17T01:30:57.109 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-10-17T01:30:57.109 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-10-17T01:30:57.109 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-10-17T01:30:57.109 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-10-17T01:30:57.110 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-10-17T01:30:57.110 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-10-17T01:30:57.110 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-10-17T01:30:57.110 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-10-17T01:30:57.110 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-10-17T01:30:57.111 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-10-17T01:30:57.111 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-10-17T01:30:57.111 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-10-17T01:30:57.111 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-10-17T01:30:57.111 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-10-17T01:30:57.111 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 85, 2023-10-17T01:30:57.112 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-10-17T01:30:57.112 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2023-10-17T01:30:57.112 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2023-10-17T01:30:57.112 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2023-10-17T01:30:57.112 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2023-10-17T01:30:57.113 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-10-17T01:30:57.113 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 252, 2023-10-17T01:30:57.113 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2023-10-17T01:30:57.113 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 90 2023-10-17T01:30:57.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.113 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-10-17T01:30:57.114 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2023-10-17T01:30:57.114 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:30:57.114 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-10-17T01:30:57.114 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-10-17T01:30:57.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.115 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-10-17T01:30:57.115 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 917 2023-10-17T01:30:57.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.115 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-10-17T01:30:57.115 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-10-17T01:30:57.116 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 6, 2023-10-17T01:30:57.116 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-10-17T01:30:57.116 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 6, 2023-10-17T01:30:57.116 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 246, 2023-10-17T01:30:57.116 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-10-17T01:30:57.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.117 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:30:57.117 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:57.117 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:30:57.117 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:57.117 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 5750, 2023-10-17T01:30:57.118 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 2091278, 2023-10-17T01:30:57.118 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:57.118 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 5750, 2023-10-17T01:30:57.118 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:30:57.118 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:30:57.119 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 5750, 2023-10-17T01:30:57.119 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2091278, 2023-10-17T01:30:57.119 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:57.119 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.119 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.120 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.120 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.120 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-10-17T01:30:57.120 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:57.120 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:30:57.121 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:57.121 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:30:57.121 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:30:57.121 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:57.121 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:57.122 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 1240, 2023-10-17T01:30:57.122 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2612082, 2023-10-17T01:30:57.122 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1240, 2023-10-17T01:30:57.122 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2612082, 2023-10-17T01:30:57.122 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:57.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.123 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.123 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.123 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.123 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-10-17T01:30:57.124 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:57.124 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-10-17T01:30:57.124 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:57.124 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:30:57.124 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:30:57.125 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:57.125 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:30:57.125 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 1240, 2023-10-17T01:30:57.125 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1240, 2023-10-17T01:30:57.125 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1240, 2023-10-17T01:30:57.125 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1240, 2023-10-17T01:30:57.126 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:57.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.126 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.127 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:30:57.127 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:57.128 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:30:57.128 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:57.128 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 246, 2023-10-17T01:30:57.128 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 24846, 2023-10-17T01:30:57.129 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:57.129 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 246, 2023-10-17T01:30:57.129 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:30:57.129 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:30:57.129 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 118, 2023-10-17T01:30:57.130 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 24846, 2023-10-17T01:30:57.130 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:57.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.130 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.131 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:30:57.131 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55f5e11ba0a0": { 2023-10-17T01:30:57.131 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:30:57.131 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:30:57.132 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:30:57.132 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1581, 2023-10-17T01:30:57.132 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 2492454, 2023-10-17T01:30:57.132 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:30:57.132 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1581, 2023-10-17T01:30:57.133 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:30:57.133 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:30:57.133 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 339, 2023-10-17T01:30:57.133 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2492454, 2023-10-17T01:30:57.133 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:30:57.133 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:30:57.134 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:30:57.134 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:30:57.134 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.134 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:30:57.134 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-17T01:30:58.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:30:58 smithi060 ceph-mon[134136]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-17T01:30:58.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:30:58 smithi181 ceph-mon[134520]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-17T01:30:58.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:30:58 smithi045 ceph-mon[98474]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-17T01:31:00.352 INFO:tasks.check_counter:Daemon mds.f mds.exported=1 2023-10-17T01:31:00.352 INFO:tasks.check_counter:Daemon mds.f mds.imported=7 2023-10-17T01:31:00.352 DEBUG:tasks.check_counter:Getting stats from i 2023-10-17T01:31:00.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-17T01:31:00.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:00 smithi060 ceph-mon[134136]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-17T01:31:00.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:00 smithi181 ceph-mon[134520]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-17T01:31:00.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:00 smithi045 ceph-mon[98474]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-17T01:31:01.176 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:31:02.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:02 smithi060 ceph-mon[134136]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-17T01:31:02.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:02 smithi181 ceph-mon[134520]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-17T01:31:02.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:02 smithi045 ceph-mon[98474]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-17T01:31:02.827 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-17T01:31:02.827 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:31:02.828 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 893005, 2023-10-17T01:31:02.828 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 893026, 2023-10-17T01:31:02.828 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 142285570, 2023-10-17T01:31:02.828 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 156013731, 2023-10-17T01:31:02.828 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2023-10-17T01:31:02.829 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2023-10-17T01:31:02.829 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 32.576564952, 2023-10-17T01:31:02.829 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 14.359663733, 2023-10-17T01:31:02.829 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 22.137656532, 2023-10-17T01:31:02.830 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.066215121, 2023-10-17T01:31:02.830 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:31:02.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 893024, 2023-10-17T01:31:02.830 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 15.720461055, 2023-10-17T01:31:02.830 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000017603 2023-10-17T01:31:02.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.831 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:31:02.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1781155, 2023-10-17T01:31:02.831 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.162680950, 2023-10-17T01:31:02.831 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000091 2023-10-17T01:31:02.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:31:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:31:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:31:02.832 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 8477, 2023-10-17T01:31:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 10267, 2023-10-17T01:31:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3114042, 2023-10-17T01:31:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 5817749, 2023-10-17T01:31:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 13, 2023-10-17T01:31:02.833 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2023-10-17T01:31:02.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.445039116, 2023-10-17T01:31:02.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.183475578, 2023-10-17T01:31:02.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.329020851, 2023-10-17T01:31:02.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.009407561, 2023-10-17T01:31:02.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:31:02.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10266, 2023-10-17T01:31:02.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.910405315, 2023-10-17T01:31:02.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000088681 2023-10-17T01:31:02.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.835 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:31:02.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 14777, 2023-10-17T01:31:02.836 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002608890, 2023-10-17T01:31:02.836 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000176 2023-10-17T01:31:02.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 657920, 2023-10-17T01:31:02.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 177040 2023-10-17T01:31:02.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.837 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:31:02.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 889111, 2023-10-17T01:31:02.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 889382, 2023-10-17T01:31:02.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 140655075, 2023-10-17T01:31:02.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 144865274, 2023-10-17T01:31:02.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 13, 2023-10-17T01:31:02.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2023-10-17T01:31:02.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 26.700196707, 2023-10-17T01:31:02.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 10.263468782, 2023-10-17T01:31:02.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 19.100789257, 2023-10-17T01:31:02.839 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.006944840, 2023-10-17T01:31:02.839 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:31:02.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 889381, 2023-10-17T01:31:02.839 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 20.943363923, 2023-10-17T01:31:02.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000023548 2023-10-17T01:31:02.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.840 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:31:02.840 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1777698, 2023-10-17T01:31:02.840 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.178800100, 2023-10-17T01:31:02.840 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000100 2023-10-17T01:31:02.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.840 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-17T01:31:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 462016 2023-10-17T01:31:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-10-17T01:31:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-10-17T01:31:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-10-17T01:31:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.842 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-10-17T01:31:02.842 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:31:02.842 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:31:02.842 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4573, 2023-10-17T01:31:02.843 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.984492935, 2023-10-17T01:31:02.843 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000652633 2023-10-17T01:31:02.843 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.843 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-10-17T01:31:02.843 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:31:02.844 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:31:02.844 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 15, 2023-10-17T01:31:02.844 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000301007, 2023-10-17T01:31:02.844 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000020067 2023-10-17T01:31:02.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.845 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-10-17T01:31:02.845 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 5859, 2023-10-17T01:31:02.845 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 5834, 2023-10-17T01:31:02.845 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-10-17T01:31:02.845 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5834, 2023-10-17T01:31:02.846 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 19.855534337, 2023-10-17T01:31:02.846 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003403416 2023-10-17T01:31:02.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.846 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-10-17T01:31:02.846 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 25, 2023-10-17T01:31:02.847 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 1, 2023-10-17T01:31:02.847 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-10-17T01:31:02.847 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2023-10-17T01:31:02.847 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2023-10-17T01:31:02.847 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2023-10-17T01:31:02.847 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 1787, 2023-10-17T01:31:02.848 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 531, 2023-10-17T01:31:02.848 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 1221, 2023-10-17T01:31:02.848 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 35, 2023-10-17T01:31:02.848 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 1028, 2023-10-17T01:31:02.848 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 390, 2023-10-17T01:31:02.849 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 254, 2023-10-17T01:31:02.849 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 254, 2023-10-17T01:31:02.849 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 27, 2023-10-17T01:31:02.849 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 20010, 2023-10-17T01:31:02.849 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 14389, 2023-10-17T01:31:02.850 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 17, 2023-10-17T01:31:02.850 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 5, 2023-10-17T01:31:02.850 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:31:02.850 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-10-17T01:31:02.850 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 2931, 2023-10-17T01:31:02.851 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 3073, 2023-10-17T01:31:02.851 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 1, 2023-10-17T01:31:02.851 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 11, 2023-10-17T01:31:02.851 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 396, 2023-10-17T01:31:02.851 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 7, 2023-10-17T01:31:02.851 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 611, 2023-10-17T01:31:02.852 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-10-17T01:31:02.852 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 192, 2023-10-17T01:31:02.852 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2023-10-17T01:31:02.852 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 1041, 2023-10-17T01:31:02.852 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 1692970308, 2023-10-17T01:31:02.853 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2023-10-17T01:31:02.853 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 192, 2023-10-17T01:31:02.853 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-10-17T01:31:02.853 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:31:02.853 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:31:02.853 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 1, 2023-10-17T01:31:02.854 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:31:02.854 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 1, 2023-10-17T01:31:02.854 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 191, 2023-10-17T01:31:02.854 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 4, 2023-10-17T01:31:02.854 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 1002, 2023-10-17T01:31:02.855 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 1588, 2023-10-17T01:31:02.855 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:31:02.855 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:31:02.855 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:31:02.855 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 2175, 2023-10-17T01:31:02.855 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 729, 2023-10-17T01:31:02.856 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 68, 2023-10-17T01:31:02.856 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 2159 2023-10-17T01:31:02.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.856 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-10-17T01:31:02.856 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2023-10-17T01:31:02.857 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2023-10-17T01:31:02.857 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2023-10-17T01:31:02.857 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 39, 2023-10-17T01:31:02.857 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 3, 2023-10-17T01:31:02.857 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2023-10-17T01:31:02.858 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-10-17T01:31:02.858 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:31:02.858 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 522, 2023-10-17T01:31:02.858 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 1, 2023-10-17T01:31:02.858 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 2082, 2023-10-17T01:31:02.859 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-10-17T01:31:02.859 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-10-17T01:31:02.859 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:31:02.859 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:31:02.859 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-10-17T01:31:02.859 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-10-17T01:31:02.860 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:31:02.860 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 11, 2023-10-17T01:31:02.860 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-10-17T01:31:02.860 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:31:02.860 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-10-17T01:31:02.861 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-10-17T01:31:02.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.861 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-10-17T01:31:02.861 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 3151, 2023-10-17T01:31:02.861 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2023-10-17T01:31:02.861 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2023-10-17T01:31:02.862 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 3151, 2023-10-17T01:31:02.862 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-10-17T01:31:02.862 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2023-10-17T01:31:02.862 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 4, 2023-10-17T01:31:02.862 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2023-10-17T01:31:02.863 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2023-10-17T01:31:02.863 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 4, 2023-10-17T01:31:02.863 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-10-17T01:31:02.863 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2023-10-17T01:31:02.863 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2023-10-17T01:31:02.863 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 10790032, 2023-10-17T01:31:02.864 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2023-10-17T01:31:02.864 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-10-17T01:31:02.864 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2181, 2023-10-17T01:31:02.864 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 10.763784329, 2023-10-17T01:31:02.864 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.004935251 2023-10-17T01:31:02.864 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.865 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2023-10-17T01:31:02.865 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.865 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-10-17T01:31:02.865 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 280, 2023-10-17T01:31:02.865 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 666, 2023-10-17T01:31:02.865 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 386, 2023-10-17T01:31:02.866 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 64, 2023-10-17T01:31:02.866 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 66, 2023-10-17T01:31:02.866 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 2, 2023-10-17T01:31:02.866 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 1787, 2023-10-17T01:31:02.866 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 2179, 2023-10-17T01:31:02.867 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 392, 2023-10-17T01:31:02.867 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 254, 2023-10-17T01:31:02.867 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 894, 2023-10-17T01:31:02.867 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 640, 2023-10-17T01:31:02.867 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 58996, 2023-10-17T01:31:02.867 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 198964 2023-10-17T01:31:02.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.868 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-10-17T01:31:02.868 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 11800, 2023-10-17T01:31:02.868 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 19, 2023-10-17T01:31:02.868 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 5859, 2023-10-17T01:31:02.868 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 1043, 2023-10-17T01:31:02.869 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 62, 2023-10-17T01:31:02.869 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-10-17T01:31:02.869 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 49, 2023-10-17T01:31:02.869 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.040140737, 2023-10-17T01:31:02.869 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000819198 2023-10-17T01:31:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-10-17T01:31:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 13, 2023-10-17T01:31:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.071178679, 2023-10-17T01:31:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.005475283 2023-10-17T01:31:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.871 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-10-17T01:31:02.871 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.871 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.871 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.872 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-10-17T01:31:02.872 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 521, 2023-10-17T01:31:02.872 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.298635393, 2023-10-17T01:31:02.872 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002492582 2023-10-17T01:31:02.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.872 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-10-17T01:31:02.873 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2494, 2023-10-17T01:31:02.873 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2.268673157, 2023-10-17T01:31:02.873 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000909652 2023-10-17T01:31:02.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.873 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-10-17T01:31:02.873 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.874 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.874 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.874 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-10-17T01:31:02.874 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.875 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.875 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.875 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-10-17T01:31:02.875 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.875 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.876 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.876 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-10-17T01:31:02.876 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.876 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.876 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.877 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-10-17T01:31:02.877 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.877 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.877 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.878 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-10-17T01:31:02.878 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.878 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.878 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.879 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-10-17T01:31:02.879 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.879 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.879 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.879 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-10-17T01:31:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-10-17T01:31:02.881 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.881 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.881 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.881 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-10-17T01:31:02.882 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 111, 2023-10-17T01:31:02.882 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.128921204, 2023-10-17T01:31:02.882 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001161452 2023-10-17T01:31:02.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.882 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-10-17T01:31:02.883 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 39, 2023-10-17T01:31:02.883 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.054967105, 2023-10-17T01:31:02.883 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001409412 2023-10-17T01:31:02.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.883 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-10-17T01:31:02.884 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2085, 2023-10-17T01:31:02.884 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 15.164116734, 2023-10-17T01:31:02.884 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.007272957 2023-10-17T01:31:02.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.884 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-10-17T01:31:02.885 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.885 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.885 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.885 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-10-17T01:31:02.886 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.886 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.886 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.886 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-10-17T01:31:02.887 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.887 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.887 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.887 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-10-17T01:31:02.887 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.888 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.888 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.888 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-10-17T01:31:02.888 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.889 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.889 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.889 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-10-17T01:31:02.889 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.890 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.890 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.890 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-10-17T01:31:02.891 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.891 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.891 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.891 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-10-17T01:31:02.892 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.892 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.892 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.892 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-10-17T01:31:02.893 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.893 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.893 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.893 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-10-17T01:31:02.893 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.894 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.894 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.894 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-10-17T01:31:02.894 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 522, 2023-10-17T01:31:02.895 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.828901328, 2023-10-17T01:31:02.895 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001587933 2023-10-17T01:31:02.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.895 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:31:02.895 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:31:02.896 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-10-17T01:31:02.896 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.896 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.896 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.896 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.897 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-10-17T01:31:02.897 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2023-10-17T01:31:02.897 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 2, 2023-10-17T01:31:02.897 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 1, 2023-10-17T01:31:02.897 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2023-10-17T01:31:02.898 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-10-17T01:31:02.898 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 437, 2023-10-17T01:31:02.898 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 437, 2023-10-17T01:31:02.898 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2816 2023-10-17T01:31:02.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.899 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-10-17T01:31:02.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:31:02.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-10-17T01:31:02.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:31:02.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:31:02.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:31:02.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:31:02.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:31:02.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:31:02.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:31:02.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:31:02.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:31:02.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:31:02.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:31:02.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:31:02.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:31:02.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-10-17T01:31:02.902 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:31:02.902 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-10-17T01:31:02.902 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:31:02.902 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:31:02.902 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:31:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:31:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:31:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:31:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:31:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-10-17T01:31:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:31:02.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:31:02.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:31:02.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-10-17T01:31:02.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-10-17T01:31:02.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-10-17T01:31:02.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:31:02.905 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:31:02.905 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:31:02.905 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:31:02.905 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 327304, 2023-10-17T01:31:02.905 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 389, 2023-10-17T01:31:02.906 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 264, 2023-10-17T01:31:02.906 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 3, 2023-10-17T01:31:02.906 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-10-17T01:31:02.906 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-10-17T01:31:02.906 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:31:02.906 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-10-17T01:31:02.907 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:31:02.907 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-10-17T01:31:02.907 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 7432, 2023-10-17T01:31:02.907 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 156, 2023-10-17T01:31:02.907 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:31:02.908 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:31:02.908 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-10-17T01:31:02.908 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-10-17T01:31:02.908 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 2158926, 2023-10-17T01:31:02.908 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 63245, 2023-10-17T01:31:02.909 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-10-17T01:31:02.909 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-10-17T01:31:02.909 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-10-17T01:31:02.909 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-10-17T01:31:02.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.909 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-10-17T01:31:02.910 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-10-17T01:31:02.910 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-10-17T01:31:02.910 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 2642, 2023-10-17T01:31:02.910 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 6684315, 2023-10-17T01:31:02.910 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2023-10-17T01:31:02.911 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 2642, 2023-10-17T01:31:02.911 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-10-17T01:31:02.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2642, 2023-10-17T01:31:02.911 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 15.995713352, 2023-10-17T01:31:02.911 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006054395 2023-10-17T01:31:02.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.912 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-10-17T01:31:02.912 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-10-17T01:31:02.912 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2642, 2023-10-17T01:31:02.912 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2857 2023-10-17T01:31:02.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.913 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 2642, 2023-10-17T01:31:02.913 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 194, 2023-10-17T01:31:02.913 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 2448, 2023-10-17T01:31:02.913 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-10-17T01:31:02.913 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-10-17T01:31:02.913 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2023-10-17T01:31:02.914 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2023-10-17T01:31:02.914 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1, 2023-10-17T01:31:02.914 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 2182, 2023-10-17T01:31:02.914 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 69, 2023-10-17T01:31:02.914 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-10-17T01:31:02.915 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-10-17T01:31:02.915 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-10-17T01:31:02.915 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2023-10-17T01:31:02.915 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 14, 2023-10-17T01:31:02.915 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-10-17T01:31:02.916 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-10-17T01:31:02.916 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-10-17T01:31:02.916 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 192, 2023-10-17T01:31:02.916 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2023-10-17T01:31:02.916 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:31:02.917 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-10-17T01:31:02.917 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:31:02.917 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-10-17T01:31:02.917 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-10-17T01:31:02.917 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-10-17T01:31:02.917 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:31:02.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-10-17T01:31:02.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:31:02.918 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-10-17T01:31:02.918 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-10-17T01:31:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-10-17T01:31:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-10-17T01:31:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-10-17T01:31:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-10-17T01:31:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-10-17T01:31:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-10-17T01:31:02.920 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-10-17T01:31:02.920 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-10-17T01:31:02.920 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-10-17T01:31:02.920 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-10-17T01:31:02.920 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-10-17T01:31:02.921 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-10-17T01:31:02.921 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-10-17T01:31:02.921 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-10-17T01:31:02.921 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-10-17T01:31:02.921 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 85, 2023-10-17T01:31:02.922 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-10-17T01:31:02.922 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2023-10-17T01:31:02.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2023-10-17T01:31:02.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2023-10-17T01:31:02.922 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2023-10-17T01:31:02.923 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-10-17T01:31:02.923 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 219, 2023-10-17T01:31:02.923 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2023-10-17T01:31:02.923 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 148 2023-10-17T01:31:02.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.924 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-10-17T01:31:02.924 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2023-10-17T01:31:02.924 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:31:02.924 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-10-17T01:31:02.924 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-10-17T01:31:02.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.925 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-10-17T01:31:02.925 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 5889 2023-10-17T01:31:02.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.925 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-10-17T01:31:02.925 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-10-17T01:31:02.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 1, 2023-10-17T01:31:02.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-10-17T01:31:02.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 1, 2023-10-17T01:31:02.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 1, 2023-10-17T01:31:02.926 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-10-17T01:31:02.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.927 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:31:02.927 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:02.927 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:31:02.927 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:02.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1790593, 2023-10-17T01:31:02.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 124880317, 2023-10-17T01:31:02.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:02.928 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1790593, 2023-10-17T01:31:02.928 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:31:02.928 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:31:02.929 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1790593, 2023-10-17T01:31:02.929 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 124880317, 2023-10-17T01:31:02.929 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:02.929 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.929 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.930 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.930 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.930 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-10-17T01:31:02.930 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:02.931 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:31:02.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:02.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:31:02.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:31:02.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:02.931 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:31:02.932 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2642, 2023-10-17T01:31:02.932 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 6684400, 2023-10-17T01:31:02.932 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2642, 2023-10-17T01:31:02.932 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 6684400, 2023-10-17T01:31:02.932 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:02.933 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.933 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.933 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.933 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.934 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-10-17T01:31:02.934 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:02.934 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-10-17T01:31:02.934 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:02.934 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:31:02.935 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:31:02.935 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:02.935 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:31:02.935 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2642, 2023-10-17T01:31:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2642, 2023-10-17T01:31:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2642, 2023-10-17T01:31:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2642, 2023-10-17T01:31:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.937 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.937 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.937 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.937 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:31:02.938 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:02.938 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:31:02.938 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:02.938 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1, 2023-10-17T01:31:02.938 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 101, 2023-10-17T01:31:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1, 2023-10-17T01:31:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:31:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:31:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1, 2023-10-17T01:31:02.939 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 101, 2023-10-17T01:31:02.940 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:02.940 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.940 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.940 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:02.941 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x56032ba6c0a0": { 2023-10-17T01:31:02.941 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:02.941 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:31:02.941 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:02.942 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 3151, 2023-10-17T01:31:02.942 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 6595728, 2023-10-17T01:31:02.942 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:02.942 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 3151, 2023-10-17T01:31:02.942 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:31:02.943 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:31:02.943 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2181, 2023-10-17T01:31:02.943 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 6595728, 2023-10-17T01:31:02.943 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:02.943 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:02.943 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:02.944 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:02.944 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.944 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:02.944 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-17T01:31:03.173 INFO:tasks.check_counter:Daemon mds.i mds.exported=11 2023-10-17T01:31:03.174 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2023-10-17T01:31:03.174 DEBUG:tasks.check_counter:Getting stats from l 2023-10-17T01:31:03.174 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ab9fe43522bc41de59746200b95f65d4fbcccffe shell --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-17T01:31:03.809 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-17T01:31:04.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:04 smithi060 ceph-mon[134136]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2023-10-17T01:31:04.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:04 smithi181 ceph-mon[134520]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2023-10-17T01:31:04.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:04 smithi045 ceph-mon[98474]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2023-10-17T01:31:06.318 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-17T01:31:06.318 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2023-10-17T01:31:06.318 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1238, 2023-10-17T01:31:06.318 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1240, 2023-10-17T01:31:06.319 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1484994, 2023-10-17T01:31:06.319 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 385082, 2023-10-17T01:31:06.319 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2023-10-17T01:31:06.319 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2023-10-17T01:31:06.319 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.092114317, 2023-10-17T01:31:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.022993787, 2023-10-17T01:31:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.025301729, 2023-10-17T01:31:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.035146254, 2023-10-17T01:31:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:31:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1238, 2023-10-17T01:31:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.284358151, 2023-10-17T01:31:06.321 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000229691 2023-10-17T01:31:06.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.321 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:31:06.321 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.321 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.322 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.322 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-17T01:31:06.322 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2023-10-17T01:31:06.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2023-10-17T01:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 928, 2023-10-17T01:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1154, 2023-10-17T01:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 228224, 2023-10-17T01:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 743054, 2023-10-17T01:31:06.324 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2023-10-17T01:31:06.324 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2023-10-17T01:31:06.324 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.086257115, 2023-10-17T01:31:06.324 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.023839152, 2023-10-17T01:31:06.324 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.018253842, 2023-10-17T01:31:06.325 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.037021792, 2023-10-17T01:31:06.325 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:31:06.325 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1152, 2023-10-17T01:31:06.325 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.411109364, 2023-10-17T01:31:06.325 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000356865 2023-10-17T01:31:06.325 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.326 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:31:06.326 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.326 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.326 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.327 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-17T01:31:06.327 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 454752 2023-10-17T01:31:06.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.327 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2023-10-17T01:31:06.327 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1380, 2023-10-17T01:31:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1355, 2023-10-17T01:31:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 986288, 2023-10-17T01:31:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 498017, 2023-10-17T01:31:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 3, 2023-10-17T01:31:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2023-10-17T01:31:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.134430455, 2023-10-17T01:31:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.039206990, 2023-10-17T01:31:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.041690549, 2023-10-17T01:31:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.045723930, 2023-10-17T01:31:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2023-10-17T01:31:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1355, 2023-10-17T01:31:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.458439326, 2023-10-17T01:31:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000338331 2023-10-17T01:31:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2023-10-17T01:31:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 634496, 2023-10-17T01:31:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 172400 2023-10-17T01:31:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2023-10-17T01:31:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2023-10-17T01:31:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2023-10-17T01:31:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2023-10-17T01:31:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:31:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:31:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 5447, 2023-10-17T01:31:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.141529201, 2023-10-17T01:31:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000025982 2023-10-17T01:31:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2023-10-17T01:31:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2023-10-17T01:31:06.335 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2023-10-17T01:31:06.335 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.335 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.335 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.336 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.336 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2023-10-17T01:31:06.336 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2023-10-17T01:31:06.336 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2023-10-17T01:31:06.336 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2023-10-17T01:31:06.336 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.337 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.337 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.337 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.337 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2023-10-17T01:31:06.337 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2023-10-17T01:31:06.338 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2023-10-17T01:31:06.338 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2023-10-17T01:31:06.338 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2023-10-17T01:31:06.338 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2023-10-17T01:31:06.338 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2023-10-17T01:31:06.339 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 252, 2023-10-17T01:31:06.339 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 6, 2023-10-17T01:31:06.339 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 148, 2023-10-17T01:31:06.339 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 98, 2023-10-17T01:31:06.339 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 243, 2023-10-17T01:31:06.339 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2023-10-17T01:31:06.340 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2023-10-17T01:31:06.340 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2023-10-17T01:31:06.340 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 7, 2023-10-17T01:31:06.340 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2023-10-17T01:31:06.340 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2023-10-17T01:31:06.341 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2023-10-17T01:31:06.341 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2023-10-17T01:31:06.341 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2023-10-17T01:31:06.341 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2023-10-17T01:31:06.341 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2023-10-17T01:31:06.341 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2023-10-17T01:31:06.342 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2023-10-17T01:31:06.342 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2023-10-17T01:31:06.342 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2023-10-17T01:31:06.342 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2023-10-17T01:31:06.342 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2023-10-17T01:31:06.343 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2023-10-17T01:31:06.343 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2023-10-17T01:31:06.343 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2023-10-17T01:31:06.343 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2023-10-17T01:31:06.343 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2023-10-17T01:31:06.344 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2023-10-17T01:31:06.344 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2023-10-17T01:31:06.344 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2023-10-17T01:31:06.344 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2023-10-17T01:31:06.344 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2023-10-17T01:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2023-10-17T01:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2023-10-17T01:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2023-10-17T01:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2023-10-17T01:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2023-10-17T01:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2023-10-17T01:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2023-10-17T01:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2023-10-17T01:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-17T01:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-17T01:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2023-10-17T01:31:06.347 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2023-10-17T01:31:06.347 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2023-10-17T01:31:06.347 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2023-10-17T01:31:06.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.347 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2023-10-17T01:31:06.347 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2023-10-17T01:31:06.348 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2023-10-17T01:31:06.348 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2023-10-17T01:31:06.348 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2023-10-17T01:31:06.348 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2023-10-17T01:31:06.348 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2023-10-17T01:31:06.349 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2023-10-17T01:31:06.349 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2023-10-17T01:31:06.349 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2023-10-17T01:31:06.349 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2023-10-17T01:31:06.349 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2023-10-17T01:31:06.350 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2023-10-17T01:31:06.350 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2023-10-17T01:31:06.350 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2023-10-17T01:31:06.350 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2023-10-17T01:31:06.350 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2023-10-17T01:31:06.351 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2023-10-17T01:31:06.351 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2023-10-17T01:31:06.351 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2023-10-17T01:31:06.351 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2023-10-17T01:31:06.351 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2023-10-17T01:31:06.352 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2023-10-17T01:31:06.352 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2023-10-17T01:31:06.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.352 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2023-10-17T01:31:06.352 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2023-10-17T01:31:06.352 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2023-10-17T01:31:06.353 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2023-10-17T01:31:06.353 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 859, 2023-10-17T01:31:06.353 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2023-10-17T01:31:06.353 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2023-10-17T01:31:06.353 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2023-10-17T01:31:06.354 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2023-10-17T01:31:06.354 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2023-10-17T01:31:06.354 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2023-10-17T01:31:06.354 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2023-10-17T01:31:06.354 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2023-10-17T01:31:06.354 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2023-10-17T01:31:06.355 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 5357311, 2023-10-17T01:31:06.355 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 5356737, 2023-10-17T01:31:06.355 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2023-10-17T01:31:06.355 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.355 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.356 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.356 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.356 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 859 2023-10-17T01:31:06.356 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.356 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2023-10-17T01:31:06.357 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 249, 2023-10-17T01:31:06.357 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 251, 2023-10-17T01:31:06.357 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2023-10-17T01:31:06.357 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 14, 2023-10-17T01:31:06.357 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 16, 2023-10-17T01:31:06.357 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 2, 2023-10-17T01:31:06.358 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 252, 2023-10-17T01:31:06.358 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 254, 2023-10-17T01:31:06.358 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 2, 2023-10-17T01:31:06.358 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2023-10-17T01:31:06.358 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2023-10-17T01:31:06.359 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2023-10-17T01:31:06.359 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 48820, 2023-10-17T01:31:06.359 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 207156 2023-10-17T01:31:06.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.359 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2023-10-17T01:31:06.360 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2023-10-17T01:31:06.360 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2023-10-17T01:31:06.360 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2023-10-17T01:31:06.360 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2023-10-17T01:31:06.360 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2023-10-17T01:31:06.361 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2023-10-17T01:31:06.361 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.361 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.361 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.362 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2023-10-17T01:31:06.362 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.362 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.362 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.363 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2023-10-17T01:31:06.363 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.363 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.363 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.363 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2023-10-17T01:31:06.364 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.364 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.364 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.364 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2023-10-17T01:31:06.365 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.365 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.365 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.365 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.365 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2023-10-17T01:31:06.366 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.366 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.366 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.366 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2023-10-17T01:31:06.367 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.367 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.367 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.367 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2023-10-17T01:31:06.367 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.368 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.368 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.368 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2023-10-17T01:31:06.368 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.369 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.369 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.369 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2023-10-17T01:31:06.369 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.369 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.370 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.370 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2023-10-17T01:31:06.370 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.370 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.371 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.371 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2023-10-17T01:31:06.371 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.371 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.372 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.372 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2023-10-17T01:31:06.372 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.372 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.373 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.373 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.373 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2023-10-17T01:31:06.373 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.373 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.373 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.374 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2023-10-17T01:31:06.374 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.374 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.374 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.375 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2023-10-17T01:31:06.375 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.375 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.375 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.376 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2023-10-17T01:31:06.376 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.376 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.376 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.377 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2023-10-17T01:31:06.377 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.377 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.377 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.378 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2023-10-17T01:31:06.378 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.378 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.378 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2023-10-17T01:31:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2023-10-17T01:31:06.380 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.380 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.380 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.380 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2023-10-17T01:31:06.381 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.381 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.381 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.381 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2023-10-17T01:31:06.382 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.382 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.382 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.382 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2023-10-17T01:31:06.383 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.383 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.383 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.383 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2023-10-17T01:31:06.384 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.384 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.384 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.384 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2023-10-17T01:31:06.385 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.385 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.385 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.385 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2023-10-17T01:31:06.385 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.386 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.386 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.386 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2023-10-17T01:31:06.386 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.387 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.387 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.387 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2023-10-17T01:31:06.387 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2023-10-17T01:31:06.388 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2023-10-17T01:31:06.388 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.388 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.388 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.388 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.389 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2023-10-17T01:31:06.389 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2023-10-17T01:31:06.389 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2023-10-17T01:31:06.389 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2023-10-17T01:31:06.390 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2023-10-17T01:31:06.390 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2023-10-17T01:31:06.390 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2023-10-17T01:31:06.390 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2023-10-17T01:31:06.390 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 2816 2023-10-17T01:31:06.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.391 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2023-10-17T01:31:06.391 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2023-10-17T01:31:06.391 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2023-10-17T01:31:06.391 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2023-10-17T01:31:06.391 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2023-10-17T01:31:06.392 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2023-10-17T01:31:06.392 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2023-10-17T01:31:06.392 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-17T01:31:06.392 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2023-10-17T01:31:06.392 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-17T01:31:06.393 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2023-10-17T01:31:06.393 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2023-10-17T01:31:06.393 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2023-10-17T01:31:06.393 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-17T01:31:06.393 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2023-10-17T01:31:06.393 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2023-10-17T01:31:06.394 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2023-10-17T01:31:06.394 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2023-10-17T01:31:06.394 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2023-10-17T01:31:06.394 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-17T01:31:06.394 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2023-10-17T01:31:06.395 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-17T01:31:06.395 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2023-10-17T01:31:06.395 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2023-10-17T01:31:06.395 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2023-10-17T01:31:06.395 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2023-10-17T01:31:06.395 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2023-10-17T01:31:06.396 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-17T01:31:06.396 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2023-10-17T01:31:06.396 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2023-10-17T01:31:06.396 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2023-10-17T01:31:06.396 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2023-10-17T01:31:06.397 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2023-10-17T01:31:06.397 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2023-10-17T01:31:06.397 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2023-10-17T01:31:06.397 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2023-10-17T01:31:06.397 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2023-10-17T01:31:06.397 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 199161, 2023-10-17T01:31:06.398 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 295, 2023-10-17T01:31:06.398 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2023-10-17T01:31:06.398 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2023-10-17T01:31:06.398 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2023-10-17T01:31:06.398 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2023-10-17T01:31:06.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2023-10-17T01:31:06.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2023-10-17T01:31:06.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2023-10-17T01:31:06.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2023-10-17T01:31:06.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 7432, 2023-10-17T01:31:06.400 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 156, 2023-10-17T01:31:06.400 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2023-10-17T01:31:06.400 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2023-10-17T01:31:06.400 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2023-10-17T01:31:06.400 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2023-10-17T01:31:06.400 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 748409, 2023-10-17T01:31:06.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 6542, 2023-10-17T01:31:06.401 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2023-10-17T01:31:06.401 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2023-10-17T01:31:06.401 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2023-10-17T01:31:06.401 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2023-10-17T01:31:06.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.402 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2023-10-17T01:31:06.402 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2023-10-17T01:31:06.402 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2023-10-17T01:31:06.402 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 3212, 2023-10-17T01:31:06.403 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2023-10-17T01:31:06.403 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2023-10-17T01:31:06.403 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 3212, 2023-10-17T01:31:06.403 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2023-10-17T01:31:06.403 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3212, 2023-10-17T01:31:06.403 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 38.487312931, 2023-10-17T01:31:06.404 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.011982351 2023-10-17T01:31:06.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.404 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2023-10-17T01:31:06.404 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2023-10-17T01:31:06.404 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3212, 2023-10-17T01:31:06.405 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 3213 2023-10-17T01:31:06.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.405 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 3212, 2023-10-17T01:31:06.405 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 3212, 2023-10-17T01:31:06.405 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2023-10-17T01:31:06.405 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2023-10-17T01:31:06.406 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2023-10-17T01:31:06.406 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 1972, 2023-10-17T01:31:06.406 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2023-10-17T01:31:06.406 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 1238, 2023-10-17T01:31:06.406 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2023-10-17T01:31:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2023-10-17T01:31:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2023-10-17T01:31:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2023-10-17T01:31:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2023-10-17T01:31:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2023-10-17T01:31:06.408 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2023-10-17T01:31:06.408 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2023-10-17T01:31:06.408 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2023-10-17T01:31:06.408 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2023-10-17T01:31:06.408 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2023-10-17T01:31:06.408 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2023-10-17T01:31:06.409 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2023-10-17T01:31:06.409 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2023-10-17T01:31:06.409 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2023-10-17T01:31:06.409 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2023-10-17T01:31:06.409 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2023-10-17T01:31:06.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2023-10-17T01:31:06.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2023-10-17T01:31:06.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2023-10-17T01:31:06.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2023-10-17T01:31:06.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2023-10-17T01:31:06.411 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2023-10-17T01:31:06.411 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2023-10-17T01:31:06.411 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2023-10-17T01:31:06.411 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2023-10-17T01:31:06.411 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2023-10-17T01:31:06.411 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2023-10-17T01:31:06.412 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2023-10-17T01:31:06.412 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2023-10-17T01:31:06.412 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2023-10-17T01:31:06.412 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2023-10-17T01:31:06.412 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2023-10-17T01:31:06.413 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2023-10-17T01:31:06.413 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2023-10-17T01:31:06.413 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2023-10-17T01:31:06.413 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2023-10-17T01:31:06.413 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2023-10-17T01:31:06.413 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 85, 2023-10-17T01:31:06.414 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2023-10-17T01:31:06.414 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 2, 2023-10-17T01:31:06.414 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 5, 2023-10-17T01:31:06.414 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 7, 2023-10-17T01:31:06.414 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 2, 2023-10-17T01:31:06.415 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2023-10-17T01:31:06.415 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2023-10-17T01:31:06.415 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2023-10-17T01:31:06.415 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2023-10-17T01:31:06.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.416 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2023-10-17T01:31:06.416 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2023-10-17T01:31:06.416 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2023-10-17T01:31:06.416 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2023-10-17T01:31:06.416 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2023-10-17T01:31:06.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.417 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2023-10-17T01:31:06.417 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2023-10-17T01:31:06.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.417 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2023-10-17T01:31:06.418 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2023-10-17T01:31:06.418 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2023-10-17T01:31:06.418 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2023-10-17T01:31:06.418 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2023-10-17T01:31:06.418 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2023-10-17T01:31:06.419 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2023-10-17T01:31:06.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.419 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-17T01:31:06.419 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:06.419 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:31:06.419 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:06.420 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 3542, 2023-10-17T01:31:06.420 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 2360965, 2023-10-17T01:31:06.420 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:06.420 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 3542, 2023-10-17T01:31:06.420 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:31:06.421 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:31:06.421 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3542, 2023-10-17T01:31:06.421 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2360965, 2023-10-17T01:31:06.421 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:06.421 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.421 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.422 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.422 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.422 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2023-10-17T01:31:06.422 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:06.423 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2023-10-17T01:31:06.423 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:06.423 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:31:06.423 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:31:06.423 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:06.424 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:31:06.424 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 3212, 2023-10-17T01:31:06.424 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1163012, 2023-10-17T01:31:06.424 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3212, 2023-10-17T01:31:06.424 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1163012, 2023-10-17T01:31:06.424 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:06.425 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.425 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.425 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.425 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2023-10-17T01:31:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2023-10-17T01:31:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:31:06.426 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:31:06.427 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:06.427 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:31:06.427 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 3212, 2023-10-17T01:31:06.427 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 3212, 2023-10-17T01:31:06.427 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 3212, 2023-10-17T01:31:06.428 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3212, 2023-10-17T01:31:06.428 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:06.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.428 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.428 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.429 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2023-10-17T01:31:06.429 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:06.429 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:31:06.429 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:31:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:31:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:31:06.430 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:31:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:31:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2023-10-17T01:31:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2023-10-17T01:31:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:06.431 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.432 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-17T01:31:06.432 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x5640e468a500": { 2023-10-17T01:31:06.433 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2023-10-17T01:31:06.433 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2023-10-17T01:31:06.433 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2023-10-17T01:31:06.433 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2023-10-17T01:31:06.433 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2023-10-17T01:31:06.434 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2023-10-17T01:31:06.434 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2023-10-17T01:31:06.434 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2023-10-17T01:31:06.434 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2023-10-17T01:31:06.434 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2023-10-17T01:31:06.435 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2023-10-17T01:31:06.435 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2023-10-17T01:31:06.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2023-10-17T01:31:06.435 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2023-10-17T01:31:06.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2023-10-17T01:31:06.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-17T01:31:06.436 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-17T01:31:06.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:06 smithi060 ceph-mon[134136]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-10-17T01:31:06.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:06 smithi181 ceph-mon[134520]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-10-17T01:31:06.787 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-10-17T01:31:06.787 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-10-17T01:31:06.788 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-17T01:31:06.799 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-17T01:31:06.800 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:31:06.800 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-17T01:31:06.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:06 smithi045 ceph-mon[98474]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-10-17T01:31:06.852 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:31:06.852 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-17T01:31:06.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:31:07.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 -- 172.21.15.45:0/2784934068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5400737b0 msgr2=0x7fe540073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 --2- 172.21.15.45:0/2784934068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5400737b0 0x7fe540073c30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe530009990 tx=0x7fe530009ca0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 -- 172.21.15.45:0/2784934068 shutdown_connections 2023-10-17T01:31:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 --2- 172.21.15.45:0/2784934068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe540108c10 0x7fe54010b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 --2- 172.21.15.45:0/2784934068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe540074d90 0x7fe5400731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 --2- 172.21.15.45:0/2784934068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5400737b0 0x7fe540073c30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 -- 172.21.15.45:0/2784934068 >> 172.21.15.45:0/2784934068 conn(0x7fe5400fc280 msgr2=0x7fe5400fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 -- 172.21.15.45:0/2784934068 shutdown_connections 2023-10-17T01:31:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.149+0000 7fe5479ff700 1 -- 172.21.15.45:0/2784934068 wait complete. 2023-10-17T01:31:07.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.150+0000 7fe5479ff700 1 Processor -- start 2023-10-17T01:31:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.150+0000 7fe5479ff700 1 -- start start 2023-10-17T01:31:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe5479ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 0x7fe540198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe5479ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe540074d90 0x7fe5401991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe5479ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe540108c10 0x7fe5401a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe5479ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5401a07c0 con 0x7fe540074d90 2023-10-17T01:31:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe5479ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5401a0930 con 0x7fe540108c10 2023-10-17T01:31:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe5479ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5401a0af0 con 0x7fe5400737b0 2023-10-17T01:31:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 0x7fe540198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 0x7fe540198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:47070/0 (socket says 172.21.15.45:47070) 2023-10-17T01:31:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 -- 172.21.15.45:0/1022736111 learned_addr learned my addr 172.21.15.45:0/1022736111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe544f9a700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe540074d90 0x7fe5401991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe545f9c700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe540108c10 0x7fe5401a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 -- 172.21.15.45:0/1022736111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe540108c10 msgr2=0x7fe5401a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe540108c10 0x7fe5401a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 -- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe540074d90 msgr2=0x7fe5401991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe540074d90 0x7fe5401991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.151+0000 7fe54579b700 1 -- 172.21.15.45:0/1022736111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300095f0 con 0x7fe5400737b0 2023-10-17T01:31:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.152+0000 7fe54579b700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 0x7fe540198c80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00e9a0 tx=0x7fe53c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.152+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c009790 con 0x7fe5400737b0 2023-10-17T01:31:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.152+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe53c010430 con 0x7fe5400737b0 2023-10-17T01:31:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.152+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5401a0dd0 con 0x7fe5400737b0 2023-10-17T01:31:07.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.152+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c018680 con 0x7fe5400737b0 2023-10-17T01:31:07.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.152+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5401a1200 con 0x7fe5400737b0 2023-10-17T01:31:07.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.153+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5401072c0 con 0x7fe5400737b0 2023-10-17T01:31:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.155+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fe53c0105a0 con 0x7fe5400737b0 2023-10-17T01:31:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.157+0000 7fe5367fc700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe52c0a0bf0 0x7fe52c0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.157+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe53c014070 con 0x7fe5400737b0 2023-10-17T01:31:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.157+0000 7fe544f9a700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe52c0a0bf0 0x7fe52c0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.157+0000 7fe544f9a700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe52c0a0bf0 0x7fe52c0a30b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe53000b2e0 tx=0x7fe530005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.159+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fe53c095e90 con 0x7fe5400737b0 2023-10-17T01:31:07.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.372+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe54002ce00 con 0x7fe5400737b0 2023-10-17T01:31:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.374+0000 7fe5367fc700 1 -- 172.21.15.45:0/1022736111 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7fe53c026020 con 0x7fe5400737b0 2023-10-17T01:31:07.375 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:31:07.376 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:31:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe52c0a0bf0 msgr2=0x7fe52c0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe52c0a0bf0 0x7fe52c0a30b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe53000b2e0 tx=0x7fe530005fb0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 msgr2=0x7fe540198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 0x7fe540198c80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00e9a0 tx=0x7fe53c00ecb0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 shutdown_connections 2023-10-17T01:31:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fe52c0a0bf0 0x7fe52c0a30b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe540108c10 0x7fe5401a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5400737b0 0x7fe540198c80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 --2- 172.21.15.45:0/1022736111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe540074d90 0x7fe5401991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 >> 172.21.15.45:0/1022736111 conn(0x7fe5400fc280 msgr2=0x7fe5400fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 shutdown_connections 2023-10-17T01:31:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.376+0000 7fe5479ff700 1 -- 172.21.15.45:0/1022736111 wait complete. 2023-10-17T01:31:07.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-17T01:08:48.955476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-17T01:12:59.332496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-17T01:13:00.462889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-17T01:31:07.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-17T01:31:07.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 -- 172.21.15.45:0/1586630286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50101db0 msgr2=0x7efc50102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 --2- 172.21.15.45:0/1586630286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50101db0 0x7efc50102210 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efc400099c0 tx=0x7efc40009cd0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 -- 172.21.15.45:0/1586630286 shutdown_connections 2023-10-17T01:31:07.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 --2- 172.21.15.45:0/1586630286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50102750 0x7efc50109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 --2- 172.21.15.45:0/1586630286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50100bc0 0x7efc50100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 --2- 172.21.15.45:0/1586630286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50101db0 0x7efc50102210 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 -- 172.21.15.45:0/1586630286 >> 172.21.15.45:0/1586630286 conn(0x7efc500fc130 msgr2=0x7efc500fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:07.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 -- 172.21.15.45:0/1586630286 shutdown_connections 2023-10-17T01:31:07.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.604+0000 7efc5679d700 1 -- 172.21.15.45:0/1586630286 wait complete. 2023-10-17T01:31:07.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.605+0000 7efc5679d700 1 Processor -- start 2023-10-17T01:31:07.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.605+0000 7efc5679d700 1 -- start start 2023-10-17T01:31:07.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5679d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 0x7efc501967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5679d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50101db0 0x7efc50196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5679d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50102750 0x7efc5019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5679d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5019e350 con 0x7efc50102750 2023-10-17T01:31:07.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5679d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5019e490 con 0x7efc50100bc0 2023-10-17T01:31:07.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5679d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5019e620 con 0x7efc50101db0 2023-10-17T01:31:07.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 0x7efc501967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 0x7efc501967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:55760/0 (socket says 172.21.15.45:55760) 2023-10-17T01:31:07.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 -- 172.21.15.45:0/2508669927 learned_addr learned my addr 172.21.15.45:0/2508669927 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:07.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc54f9a700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50101db0 0x7efc50196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc55f9c700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50102750 0x7efc5019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 -- 172.21.15.45:0/2508669927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50101db0 msgr2=0x7efc50196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50101db0 0x7efc50196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 -- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50102750 msgr2=0x7efc5019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50102750 0x7efc5019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.606+0000 7efc5579b700 1 -- 172.21.15.45:0/2508669927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400095f0 con 0x7efc50100bc0 2023-10-17T01:31:07.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.607+0000 7efc5579b700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 0x7efc501967d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e970 tx=0x7efc4c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:07.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.607+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c009790 con 0x7efc50100bc0 2023-10-17T01:31:07.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.607+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4c00c7e0 con 0x7efc50100bc0 2023-10-17T01:31:07.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.607+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5019e900 con 0x7efc50100bc0 2023-10-17T01:31:07.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.607+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c003c70 con 0x7efc50100bc0 2023-10-17T01:31:07.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.607+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5019ee60 con 0x7efc50100bc0 2023-10-17T01:31:07.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.608+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50108110 con 0x7efc50100bc0 2023-10-17T01:31:07.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.610+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7efc4c00cc50 con 0x7efc50100bc0 2023-10-17T01:31:07.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.611+0000 7efc467fc700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc3c0a0c40 0x7efc3c0a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:07.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.611+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efc4c014070 con 0x7efc50100bc0 2023-10-17T01:31:07.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.611+0000 7efc54f9a700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc3c0a0c40 0x7efc3c0a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:07.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.612+0000 7efc54f9a700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc3c0a0c40 0x7efc3c0a3100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc40009fd0 tx=0x7efc4000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:07.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.613+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7efc4c0960e0 con 0x7efc50100bc0 2023-10-17T01:31:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.826+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc5002cde0 con 0x7efc50100bc0 2023-10-17T01:31:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.827+0000 7efc467fc700 1 -- 172.21.15.45:0/2508669927 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc4c099700 con 0x7efc50100bc0 2023-10-17T01:31:07.829 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-17T01:31:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc3c0a0c40 msgr2=0x7efc3c0a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc3c0a0c40 0x7efc3c0a3100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc40009fd0 tx=0x7efc4000b540 comp rx=0 tx=0).stop 2023-10-17T01:31:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 msgr2=0x7efc501967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 0x7efc501967d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e970 tx=0x7efc4c00ec80 comp rx=0 tx=0).stop 2023-10-17T01:31:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 shutdown_connections 2023-10-17T01:31:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc3c0a0c40 0x7efc3c0a3100 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50102750 0x7efc5019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc50100bc0 0x7efc501967d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 --2- 172.21.15.45:0/2508669927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50101db0 0x7efc50196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 >> 172.21.15.45:0/2508669927 conn(0x7efc500fc130 msgr2=0x7efc50105250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 shutdown_connections 2023-10-17T01:31:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:07.829+0000 7efc5679d700 1 -- 172.21.15.45:0/2508669927 wait complete. 2023-10-17T01:31:07.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-17T01:31:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 -- 172.21.15.45:0/3648296410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8500101f30 msgr2=0x7f8500102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 --2- 172.21.15.45:0/3648296410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8500101f30 0x7f8500102390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f84f00099c0 tx=0x7f84f0009cd0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 -- 172.21.15.45:0/3648296410 shutdown_connections 2023-10-17T01:31:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 --2- 172.21.15.45:0/3648296410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85001028d0 0x7f8500109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 --2- 172.21.15.45:0/3648296410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 0x7f8500101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 --2- 172.21.15.45:0/3648296410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8500101f30 0x7f8500102390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 -- 172.21.15.45:0/3648296410 >> 172.21.15.45:0/3648296410 conn(0x7f85000fc290 msgr2=0x7f85000fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 -- 172.21.15.45:0/3648296410 shutdown_connections 2023-10-17T01:31:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.057+0000 7f8507bff700 1 -- 172.21.15.45:0/3648296410 wait complete. 2023-10-17T01:31:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.058+0000 7f8507bff700 1 Processor -- start 2023-10-17T01:31:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.058+0000 7f8507bff700 1 -- start start 2023-10-17T01:31:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f8507bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 0x7f8500198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f8507bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8500101f30 0x7f8500199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f8507bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85001028d0 0x7f85001a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f8507bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001a0880 con 0x7f8500100d40 2023-10-17T01:31:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f8507bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001a09f0 con 0x7f85001028d0 2023-10-17T01:31:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f8507bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85001a0bb0 con 0x7f8500101f30 2023-10-17T01:31:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850599b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 0x7f8500198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850519a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8500101f30 0x7f8500199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85001028d0 0x7f85001a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850519a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8500101f30 0x7f8500199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34884/0 (socket says 172.21.15.45:34884) 2023-10-17T01:31:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850519a700 1 -- 172.21.15.45:0/684213537 learned_addr learned my addr 172.21.15.45:0/684213537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850619c700 1 -- 172.21.15.45:0/684213537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8500101f30 msgr2=0x7f8500199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850619c700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8500101f30 0x7f8500199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850619c700 1 -- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 msgr2=0x7f8500198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850619c700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 0x7f8500198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850619c700 1 -- 172.21.15.45:0/684213537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00095f0 con 0x7f85001028d0 2023-10-17T01:31:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.059+0000 7f850599b700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 0x7f8500198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.060+0000 7f850619c700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85001028d0 0x7f85001a02b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f84f800efc0 tx=0x7f84f800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.060+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8018070 con 0x7f85001028d0 2023-10-17T01:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.060+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f800f040 con 0x7f85001028d0 2023-10-17T01:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.060+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85001a0e30 con 0x7f85001028d0 2023-10-17T01:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.060+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800cd40 con 0x7f85001028d0 2023-10-17T01:31:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.060+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85001a1250 con 0x7f85001028d0 2023-10-17T01:31:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.061+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8500193390 con 0x7f85001028d0 2023-10-17T01:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.063+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f84f8003a00 con 0x7f85001028d0 2023-10-17T01:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.064+0000 7f84f6ffd700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84ec0a0b20 0x7f84ec0a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.064+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f84f80cbe40 con 0x7f85001028d0 2023-10-17T01:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.064+0000 7f850599b700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84ec0a0b20 0x7f84ec0a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.065+0000 7f850599b700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84ec0a0b20 0x7f84ec0a2fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f850019a210 tx=0x7f84fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.067+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f84f8095830 con 0x7f85001028d0 2023-10-17T01:31:08.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.279+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f850002ce80 con 0x7f85001028d0 2023-10-17T01:31:08.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.280+0000 7f84f6ffd700 1 -- 172.21.15.45:0/684213537 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84f8098e50 con 0x7f85001028d0 2023-10-17T01:31:08.282 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-17T01:31:08.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84ec0a0b20 msgr2=0x7f84ec0a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84ec0a0b20 0x7f84ec0a2fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f850019a210 tx=0x7f84fc009260 comp rx=0 tx=0).stop 2023-10-17T01:31:08.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85001028d0 msgr2=0x7f85001a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85001028d0 0x7f85001a02b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f84f800efc0 tx=0x7f84f800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 shutdown_connections 2023-10-17T01:31:08.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f84ec0a0b20 0x7f84ec0a2fe0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85001028d0 0x7f85001a02b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8500100d40 0x7f8500198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 --2- 172.21.15.45:0/684213537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8500101f30 0x7f8500199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.282+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 >> 172.21.15.45:0/684213537 conn(0x7f85000fc290 msgr2=0x7f8500107910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:08.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.283+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 shutdown_connections 2023-10-17T01:31:08.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.283+0000 7f8507bff700 1 -- 172.21.15.45:0/684213537 wait complete. 2023-10-17T01:31:08.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-17T01:31:08.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.511+0000 7f28a7bff700 1 -- 172.21.15.45:0/3425410406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0101f30 msgr2=0x7f28a0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.511+0000 7f28a7bff700 1 --2- 172.21.15.45:0/3425410406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0101f30 0x7f28a0102390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2890009990 tx=0x7f2890009ca0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 -- 172.21.15.45:0/3425410406 shutdown_connections 2023-10-17T01:31:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 --2- 172.21.15.45:0/3425410406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 0x7f28a0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 --2- 172.21.15.45:0/3425410406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a0100d40 0x7f28a0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 --2- 172.21.15.45:0/3425410406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0101f30 0x7f28a0102390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 -- 172.21.15.45:0/3425410406 >> 172.21.15.45:0/3425410406 conn(0x7f28a00fc290 msgr2=0x7f28a00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:08.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 -- 172.21.15.45:0/3425410406 shutdown_connections 2023-10-17T01:31:08.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.512+0000 7f28a7bff700 1 -- 172.21.15.45:0/3425410406 wait complete. 2023-10-17T01:31:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.513+0000 7f28a7bff700 1 Processor -- start 2023-10-17T01:31:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.513+0000 7f28a7bff700 1 -- start start 2023-10-17T01:31:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a7bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 0x7f28a0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a7bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a0101f30 0x7f28a0196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a7bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 0x7f28a019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a7bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a019e5b0 con 0x7f28a0101f30 2023-10-17T01:31:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a7bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a019e6f0 con 0x7f28a01028d0 2023-10-17T01:31:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a7bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a019e880 con 0x7f28a0100d40 2023-10-17T01:31:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 0x7f28a0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a599b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 0x7f28a0196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34900/0 (socket says 172.21.15.45:34900) 2023-10-17T01:31:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a599b700 1 -- 172.21.15.45:0/2596379923 learned_addr learned my addr 172.21.15.45:0/2596379923 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a619c700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 0x7f28a019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.514+0000 7f28a519a700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a0101f30 0x7f28a0196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a619c700 1 -- 172.21.15.45:0/2596379923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 msgr2=0x7f28a0196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a619c700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 0x7f28a0196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a619c700 1 -- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a0101f30 msgr2=0x7f28a0196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a619c700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a0101f30 0x7f28a0196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a619c700 1 -- 172.21.15.45:0/2596379923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28900095f0 con 0x7f28a01028d0 2023-10-17T01:31:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a599b700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 0x7f28a0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:31:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a619c700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 0x7f28a019dfe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f289800efc0 tx=0x7f289800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898018070 con 0x7f28a01028d0 2023-10-17T01:31:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289800f040 con 0x7f28a01028d0 2023-10-17T01:31:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a019eb00 con 0x7f28a01028d0 2023-10-17T01:31:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800cd40 con 0x7f28a01028d0 2023-10-17T01:31:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.515+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a019f060 con 0x7f28a01028d0 2023-10-17T01:31:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.516+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a010c550 con 0x7f28a01028d0 2023-10-17T01:31:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.518+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f2898003a00 con 0x7f28a01028d0 2023-10-17T01:31:08.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.519+0000 7f2896ffd700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f288c0a0ba0 0x7f288c0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.520+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f28980cbdf0 con 0x7f28a01028d0 2023-10-17T01:31:08.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.520+0000 7f28a599b700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f288c0a0ba0 0x7f288c0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.520+0000 7f28a599b700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f288c0a0ba0 0x7f288c0a3060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f289c005950 tx=0x7f289c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:08.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.522+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f28980957e0 con 0x7f28a01028d0 2023-10-17T01:31:08.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:08 smithi045 ceph-mon[98474]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-10-17T01:31:08.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:08 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1022736111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:31:08.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:08 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2508669927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:31:08.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:08 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/684213537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:31:08.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:08 smithi060 ceph-mon[134136]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-10-17T01:31:08.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:08 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1022736111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:31:08.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:08 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2508669927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:31:08.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:08 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/684213537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:31:08.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:08 smithi181 ceph-mon[134520]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-10-17T01:31:08.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:08 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1022736111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:31:08.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:08 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2508669927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-17T01:31:08.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:08 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/684213537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-17T01:31:08.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.735+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28a0107c60 con 0x7f28a01028d0 2023-10-17T01:31:08.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.735+0000 7f2896ffd700 1 -- 172.21.15.45:0/2596379923 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2898098e00 con 0x7f28a01028d0 2023-10-17T01:31:08.737 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-17T01:31:08.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f288c0a0ba0 msgr2=0x7f288c0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f288c0a0ba0 0x7f288c0a3060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f289c005950 tx=0x7f289c009260 comp rx=0 tx=0).stop 2023-10-17T01:31:08.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 msgr2=0x7f28a019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 0x7f28a019dfe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f289800efc0 tx=0x7f289800c3a0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 shutdown_connections 2023-10-17T01:31:08.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f288c0a0ba0 0x7f288c0a3060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28a01028d0 0x7f28a019dfe0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28a0100d40 0x7f28a0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 --2- 172.21.15.45:0/2596379923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28a0101f30 0x7f28a0196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 >> 172.21.15.45:0/2596379923 conn(0x7f28a00fc290 msgr2=0x7f28a00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 shutdown_connections 2023-10-17T01:31:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.738+0000 7f28a7bff700 1 -- 172.21.15.45:0/2596379923 wait complete. 2023-10-17T01:31:08.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-17T01:31:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 -- 172.21.15.45:0/3849643261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100102940 msgr2=0x7f51001091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 --2- 172.21.15.45:0/3849643261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100102940 0x7f51001091f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b0d0 tx=0x7f50f800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 -- 172.21.15.45:0/3849643261 shutdown_connections 2023-10-17T01:31:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 --2- 172.21.15.45:0/3849643261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100102940 0x7f51001091f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 --2- 172.21.15.45:0/3849643261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5100100db0 0x7f51001011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 --2- 172.21.15.45:0/3849643261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5100101fa0 0x7f5100102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.968+0000 7f5106fff700 1 -- 172.21.15.45:0/3849643261 >> 172.21.15.45:0/3849643261 conn(0x7f51000fc320 msgr2=0x7f51000fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.969+0000 7f5106fff700 1 -- 172.21.15.45:0/3849643261 shutdown_connections 2023-10-17T01:31:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.969+0000 7f5106fff700 1 -- 172.21.15.45:0/3849643261 wait complete. 2023-10-17T01:31:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 Processor -- start 2023-10-17T01:31:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 -- start start 2023-10-17T01:31:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 0x7f5100198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5100101fa0 0x7f51001992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5100102940 0x7f51001a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51001a08e0 con 0x7f5100101fa0 2023-10-17T01:31:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51001a0a20 con 0x7f5100100db0 2023-10-17T01:31:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5106fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51001a0bb0 con 0x7f5100102940 2023-10-17T01:31:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5104d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 0x7f5100198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5104d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 0x7f5100198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40146/0 (socket says 172.21.15.45:40146) 2023-10-17T01:31:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f5104d9b700 1 -- 172.21.15.45:0/493413489 learned_addr learned my addr 172.21.15.45:0/493413489 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f510559c700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5100102940 0x7f51001a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.970+0000 7f50f7fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5100101fa0 0x7f51001992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5104d9b700 1 -- 172.21.15.45:0/493413489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5100102940 msgr2=0x7f51001a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5104d9b700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5100102940 0x7f51001a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5104d9b700 1 -- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5100101fa0 msgr2=0x7f51001992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5104d9b700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5100101fa0 0x7f51001992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5104d9b700 1 -- 172.21.15.45:0/493413489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f8009d00 con 0x7f5100100db0 2023-10-17T01:31:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5104d9b700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 0x7f5100198d60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00e970 tx=0x7f50fc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc009790 con 0x7f5100100db0 2023-10-17T01:31:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50fc00eec0 con 0x7f5100100db0 2023-10-17T01:31:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003cf0 con 0x7f5100100db0 2023-10-17T01:31:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51001a0e90 con 0x7f5100100db0 2023-10-17T01:31:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.971+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51001a1200 con 0x7f5100100db0 2023-10-17T01:31:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.972+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510010c5b0 con 0x7f5100100db0 2023-10-17T01:31:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.974+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f50fc00ccd0 con 0x7f5100100db0 2023-10-17T01:31:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.975+0000 7f50f5ffb700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50e80a0b20 0x7f50e80a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.976+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f50fc014070 con 0x7f5100100db0 2023-10-17T01:31:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.976+0000 7f50f7fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50e80a0b20 0x7f50e80a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.976+0000 7f50f7fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50e80a0b20 0x7f50e80a2fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009bf0 tx=0x7f50f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:08.979+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f50fc095d60 con 0x7f5100100db0 2023-10-17T01:31:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.208+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51001a1b50 con 0x7f5100100db0 2023-10-17T01:31:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.208+0000 7f50f5ffb700 1 -- 172.21.15.45:0/493413489 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+590 (secure 0 0 0) 0x7f50fc099380 con 0x7f5100100db0 2023-10-17T01:31:09.210 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/518390758 2023-10-18T01:06:32.348953+0000 2023-10-17T01:31:09.210 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1775908551 2023-10-18T01:06:32.348953+0000 2023-10-17T01:31:09.211 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1775908551 2023-10-18T01:06:32.348953+0000 2023-10-17T01:31:09.211 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3224370568 2023-10-18T01:06:05.897832+0000 2023-10-17T01:31:09.211 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3128471816 2023-10-18T01:06:05.897832+0000 2023-10-17T01:31:09.211 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2130990792 2023-10-18T01:06:05.897832+0000 2023-10-17T01:31:09.211 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3224370568 2023-10-18T01:06:05.897832+0000 2023-10-17T01:31:09.212 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/887015676 2023-10-18T01:06:32.348953+0000 2023-10-17T01:31:09.212 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/4289245093 2023-10-18T01:06:05.897832+0000 2023-10-17T01:31:09.212 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2694295985 2023-10-18T01:06:32.348953+0000 2023-10-17T01:31:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50e80a0b20 msgr2=0x7f50e80a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50e80a0b20 0x7f50e80a2fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009bf0 tx=0x7f50f0009260 comp rx=0 tx=0).stop 2023-10-17T01:31:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 msgr2=0x7f5100198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:09.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 0x7f5100198d60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00e970 tx=0x7f50fc00ec80 comp rx=0 tx=0).stop 2023-10-17T01:31:09.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 shutdown_connections 2023-10-17T01:31:09.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f50e80a0b20 0x7f50e80a2fe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:09.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5100102940 0x7f51001a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:09.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5100100db0 0x7f5100198d60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 --2- 172.21.15.45:0/493413489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5100101fa0 0x7f51001992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 >> 172.21.15.45:0/493413489 conn(0x7f51000fc320 msgr2=0x7f51001077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 shutdown_connections 2023-10-17T01:31:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:09.211+0000 7f5106fff700 1 -- 172.21.15.45:0/493413489 wait complete. 2023-10-17T01:31:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2023-10-17T01:31:09.231 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-17T01:31:09.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:09.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /tmp/cephfs 2023-10-17T01:31:09.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:09 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2596379923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:31:09.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:09 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/493413489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-17T01:31:09.581 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:09 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2596379923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:31:09.581 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:09 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/493413489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-17T01:31:09.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:09 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2596379923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-17T01:31:09.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:09 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/493413489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-17T01:31:09.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:31:09.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:09.681 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-10-17T01:31:09.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:09.783 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-17T01:31:09.870 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-10-17T01:31:09.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:09.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-10-17T01:31:09.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:09.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-10-17T01:31:10.182 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-17T01:31:10.182 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-17T01:31:10.183 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-17T01:31:10.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351424k,nr_inodes=4087856,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/e76c99e3c4242bff56e7dfc5965238765ba8c63b0eb177016ca1aac746583b6c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/e76c99e3c4242bff56e7dfc5965238765ba8c63b0eb177016ca1aac746583b6c/diff,workdir=/var/lib/containers/storage/overlay/e76c99e3c4242bff56e7dfc5965238765ba8c63b0eb177016ca1aac746583b6c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/af87d567b4885f1915b4749662057cb6dddf4315412dafce437fba28735d0f81/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/af87d567b4885f1915b4749662057cb6dddf4315412dafce437fba28735d0f81/diff,workdir=/var/lib/containers/storage/overlay/af87d567b4885f1915b4749662057cb6dddf4315412dafce437fba28735d0f81/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/2b56039fcdd0cc5ccd54005ea6b3614f9296526c9bb55bdeaaba2d411c42bc69/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/2b56039fcdd0cc5ccd54005ea6b3614f9296526c9bb55bdeaaba2d411c42bc69/diff,workdir=/var/lib/containers/storage/overlay/2b56039fcdd0cc5ccd54005ea6b3614f9296526c9bb55bdeaaba2d411c42bc69/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6f3bb425743526589ff3c4d53e7fc50803e091031040e13850bc5a9a5dfc63dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/6f3bb425743526589ff3c4d53e7fc50803e091031040e13850bc5a9a5dfc63dd/diff,workdir=/var/lib/containers/storage/overlay/6f3bb425743526589ff3c4d53e7fc50803e091031040e13850bc5a9a5dfc63dd/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7ecef7a5389fd6b4b2f2060720d4d7195e10d58b56e908c516be8d6d76e86550/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/7ecef7a5389fd6b4b2f2060720d4d7195e10d58b56e908c516be8d6d76e86550/diff,workdir=/var/lib/containers/storage/overlay/7ecef7a5389fd6b4b2f2060720d4d7195e10d58b56e908c516be8d6d76e86550/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b95cd454b0358ad4542ffbe6e9e8dc4b1c29ae613d9dce7751956e2c485381a6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/b95cd454b0358ad4542ffbe6e9e8dc4b1c29ae613d9dce7751956e2c485381a6/diff,workdir=/var/lib/containers/storage/overlay/b95cd454b0358ad4542ffbe6e9e8dc4b1c29ae613d9dce7751956e2c485381a6/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d53fe23933fc9300eb922a88c2c89651f24db02f2e084a7c671f23f5fa09c50e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/d53fe23933fc9300eb922a88c2c89651f24db02f2e084a7c671f23f5fa09c50e/diff,workdir=/var/lib/containers/storage/overlay/d53fe23933fc9300eb922a88c2c89651f24db02f2e084a7c671f23f5fa09c50e/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fbf05fb6b7506a707f26569790e97ae6c495ac6ba920c8c9b0bb8f49bcf130ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/fbf05fb6b7506a707f26569790e97ae6c495ac6ba920c8c9b0bb8f49bcf130ca/diff,workdir=/var/lib/containers/storage/overlay/fbf05fb6b7506a707f26569790e97ae6c495ac6ba920c8c9b0bb8f49bcf130ca/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/eeb83488f32d9a9b8c0c80f34a3b4e1abc288f8639afefccd1827ef6b6af85eb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/eeb83488f32d9a9b8c0c80f34a3b4e1abc288f8639afefccd1827ef6b6af85eb/diff,workdir=/var/lib/containers/storage/overlay/eeb83488f32d9a9b8c0c80f34a3b4e1abc288f8639afefccd1827ef6b6af85eb/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a0cd40f51416407e158161990b31bfd880898bea136ca944d0dacc95bb6f9226/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/CJOCI5SP4OYVQ3OCXOMUBBW65Y:/var/lib/containers/storage/overlay/l/ZXNBGF77RQBTP3X2URKQ73H6OA,upperdir=/var/lib/containers/storage/overlay/a0cd40f51416407e158161990b31bfd880898bea136ca944d0dacc95bb6f9226/diff,workdir=/var/lib/containers/storage/overlay/a0cd40f51416407e158161990b31bfd880898bea136ca944d0dacc95bb6f9226/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-17T01:31:10.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:10.279 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-17T01:31:10.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-17T01:31:10.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-10-17T01:31:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2023-10-17T01:31:10.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:31:10.455 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-17T01:31:10.464 INFO:tasks.cephadm:Teardown begin 2023-10-17T01:31:10.465 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-17T01:31:10.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:31:10.578 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:10 smithi060 ceph-mon[134136]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-10-17T01:31:10.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:10 smithi181 ceph-mon[134520]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-10-17T01:31:10.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.670+0000 7f1ad2fff700 1 -- 172.21.15.45:0/135704841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 msgr2=0x7f1acc1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.670+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/135704841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc1023a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80099c0 tx=0x7f1ab8009cd0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 -- 172.21.15.45:0/135704841 shutdown_connections 2023-10-17T01:31:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/135704841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc1028e0 0x7f1acc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/135704841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1acc100d50 0x7f1acc101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/135704841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc1023a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 -- 172.21.15.45:0/135704841 >> 172.21.15.45:0/135704841 conn(0x7f1acc0fc280 msgr2=0x7f1acc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 -- 172.21.15.45:0/135704841 shutdown_connections 2023-10-17T01:31:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.671+0000 7f1ad2fff700 1 -- 172.21.15.45:0/135704841 wait complete. 2023-10-17T01:31:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 Processor -- start 2023-10-17T01:31:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 -- start start 2023-10-17T01:31:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100d50 0x7f1acc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1acc1028e0 0x7f1acc1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1a0770 con 0x7f1acc1028e0 2023-10-17T01:31:10.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1a08b0 con 0x7f1acc101f40 2023-10-17T01:31:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad2fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1a0a40 con 0x7f1acc100d50 2023-10-17T01:31:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad0d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100d50 0x7f1acc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1acbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad0d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100d50 0x7f1acc198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34924/0 (socket says 172.21.15.45:34924) 2023-10-17T01:31:10.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1acbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40158/0 (socket says 172.21.15.45:40158) 2023-10-17T01:31:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad0d9b700 1 -- 172.21.15.45:0/2806294815 learned_addr learned my addr 172.21.15.45:0/2806294815 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.672+0000 7f1ad159c700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1acc1028e0 0x7f1acc1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1acbfff700 1 -- 172.21.15.45:0/2806294815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100d50 msgr2=0x7f1acc198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1acbfff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100d50 0x7f1acc198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1acbfff700 1 -- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1acc1028e0 msgr2=0x7f1acc1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1acbfff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1acc1028e0 0x7f1acc1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1acbfff700 1 -- 172.21.15.45:0/2806294815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80095f0 con 0x7f1acc101f40 2023-10-17T01:31:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1acbfff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc199130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009fd0 tx=0x7f1ab8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab801d070 con 0x7f1acc101f40 2023-10-17T01:31:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab8005020 con 0x7f1acc101f40 2023-10-17T01:31:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8004a90 con 0x7f1acc101f40 2023-10-17T01:31:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc1a0cc0 con 0x7f1acc101f40 2023-10-17T01:31:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.673+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc1a1080 con 0x7f1acc101f40 2023-10-17T01:31:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.674+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc193350 con 0x7f1acc101f40 2023-10-17T01:31:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.676+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f1ab804f080 con 0x7f1acc101f40 2023-10-17T01:31:10.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.678+0000 7f1ac9ffb700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1abc0a0bf0 0x7f1abc0a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:10.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.678+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1ab80ccdc0 con 0x7f1acc101f40 2023-10-17T01:31:10.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.678+0000 7f1ad0d9b700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1abc0a0bf0 0x7f1abc0a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:10.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.679+0000 7f1ad0d9b700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1abc0a0bf0 0x7f1abc0a30b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005950 tx=0x7f1ac0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:10.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.680+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f1ab80967b0 con 0x7f1acc101f40 2023-10-17T01:31:10.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:10 smithi045 ceph-mon[98474]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-10-17T01:31:10.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.936+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1acc02d050 con 0x7f1acc101f40 2023-10-17T01:31:10.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.937+0000 7f1ac9ffb700 1 -- 172.21.15.45:0/2806294815 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f1ab8099dd0 con 0x7f1acc101f40 2023-10-17T01:31:10.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:31:10.940 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:30:25.960655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:31:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.939+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1abc0a0bf0 msgr2=0x7f1abc0a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.939+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1abc0a0bf0 0x7f1abc0a30b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005950 tx=0x7f1ac0009260 comp rx=0 tx=0).stop 2023-10-17T01:31:10.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.939+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 msgr2=0x7f1acc199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.939+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc199130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009fd0 tx=0x7f1ab8009580 comp rx=0 tx=0).stop 2023-10-17T01:31:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 shutdown_connections 2023-10-17T01:31:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f1abc0a0bf0 0x7f1abc0a30b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1acc1028e0 0x7f1acc1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1acc100d50 0x7f1acc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 --2- 172.21.15.45:0/2806294815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1acc101f40 0x7f1acc199130 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:10.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 >> 172.21.15.45:0/2806294815 conn(0x7f1acc0fc280 msgr2=0x7f1acc107840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 shutdown_connections 2023-10-17T01:31:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:10.940+0000 7f1ad2fff700 1 -- 172.21.15.45:0/2806294815 wait complete. 2023-10-17T01:31:10.945 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2023-10-17T01:31:10.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-17T01:31:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.167+0000 7fb2591ff700 1 -- 172.21.15.45:0/3777593539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2540737b0 msgr2=0x7fb254073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.167+0000 7fb2591ff700 1 --2- 172.21.15.45:0/3777593539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2540737b0 0x7fb254073c30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2480098f0 tx=0x7fb248009c00 comp rx=0 tx=0).stop 2023-10-17T01:31:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 -- 172.21.15.45:0/3777593539 shutdown_connections 2023-10-17T01:31:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 --2- 172.21.15.45:0/3777593539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb254108c10 0x7fb25410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 --2- 172.21.15.45:0/3777593539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb254074d90 0x7fb2540731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 --2- 172.21.15.45:0/3777593539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2540737b0 0x7fb254073c30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 -- 172.21.15.45:0/3777593539 >> 172.21.15.45:0/3777593539 conn(0x7fb2540fc280 msgr2=0x7fb2540fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 -- 172.21.15.45:0/3777593539 shutdown_connections 2023-10-17T01:31:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.168+0000 7fb2591ff700 1 -- 172.21.15.45:0/3777593539 wait complete. 2023-10-17T01:31:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.169+0000 7fb2591ff700 1 Processor -- start 2023-10-17T01:31:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.169+0000 7fb2591ff700 1 -- start start 2023-10-17T01:31:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb2591ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 0x7fb254198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb2591ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb254074d90 0x7fb254199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb2591ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb254108c10 0x7fb2541a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb2591ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2541a0790 con 0x7fb254108c10 2023-10-17T01:31:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb2591ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2541a0900 con 0x7fb2540737b0 2023-10-17T01:31:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb2591ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2541a0ac0 con 0x7fb254074d90 2023-10-17T01:31:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 0x7fb254198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 0x7fb254198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40176/0 (socket says 172.21.15.45:40176) 2023-10-17T01:31:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 -- 172.21.15.45:0/194791702 learned_addr learned my addr 172.21.15.45:0/194791702 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb25259c700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb254074d90 0x7fb254199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb25359e700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb254108c10 0x7fb2541a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 -- 172.21.15.45:0/194791702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb254074d90 msgr2=0x7fb254199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb254074d90 0x7fb254199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 -- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb254108c10 msgr2=0x7fb2541a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb254108c10 0x7fb2541a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.170+0000 7fb252d9d700 1 -- 172.21.15.45:0/194791702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248009520 con 0x7fb2540737b0 2023-10-17T01:31:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.171+0000 7fb252d9d700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 0x7fb254198c50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb24400e9a0 tx=0x7fb24400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.171+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244009790 con 0x7fb2540737b0 2023-10-17T01:31:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.171+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2541a0da0 con 0x7fb2540737b0 2023-10-17T01:31:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.171+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb24400cbd0 con 0x7fb2540737b0 2023-10-17T01:31:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.171+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244018680 con 0x7fb2540737b0 2023-10-17T01:31:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.171+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2541a1210 con 0x7fb2540737b0 2023-10-17T01:31:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.172+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2541072c0 con 0x7fb2540737b0 2023-10-17T01:31:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.174+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fb24400cd40 con 0x7fb2540737b0 2023-10-17T01:31:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.175+0000 7fb23bfff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb23c0a0b70 0x7fb23c0a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.176+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb244014070 con 0x7fb2540737b0 2023-10-17T01:31:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.176+0000 7fb25259c700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb23c0a0b70 0x7fb23c0a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.176+0000 7fb25259c700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb23c0a0b70 0x7fb23c0a3030 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb2480098c0 tx=0x7fb248019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:11.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.179+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fb244096070 con 0x7fb2540737b0 2023-10-17T01:31:11.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.441+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb25402cc30 con 0x7fb2540737b0 2023-10-17T01:31:11.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.443+0000 7fb23bfff700 1 -- 172.21.15.45:0/194791702 <== mon.2 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fb244099690 con 0x7fb2540737b0 2023-10-17T01:31:11.444 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:31:11.444 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1708283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1708283883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1708283883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1507187876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1507187876},{"type":"v1","addr":"172.21.15.45:6837","nonce":1507187876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-17T01:13:01.038089+0000","modified":"2023-10-17T01:30:25.960655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24587,"mds_1":24550,"mds_2":24539,"mds_3":24526,"mds_4":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14544":{"gid":14544,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/931419854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":931419854},{"type":"v1","addr":"172.21.15.45:6839","nonce":931419854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14556":{"gid":14556,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/1667366220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":1667366220},{"type":"v1","addr":"172.21.15.45:6841","nonce":1667366220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6833/2648148860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2648148860},{"type":"v1","addr":"172.21.15.181:6833","nonce":2648148860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6833/3345979209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3345979209},{"type":"v1","addr":"172.21.15.60:6833","nonce":3345979209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/1725566254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1725566254},{"type":"v1","addr":"172.21.15.60:6835","nonce":1725566254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6835/1888494346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1888494346},{"type":"v1","addr":"172.21.15.181:6835","nonce":1888494346}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.181:6837/1396793294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1396793294},{"type":"v1","addr":"172.21.15.181:6837","nonce":1396793294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6837/1246290891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1246290891},{"type":"v1","addr":"172.21.15.60:6837","nonce":1246290891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6839/1273587217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":1273587217},{"type":"v1","addr":"172.21.15.181:6839","nonce":1273587217}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6839/165785544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":165785544},{"type":"v1","addr":"172.21.15.60:6839","nonce":165785544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-17T01:31:11.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb23c0a0b70 msgr2=0x7fb23c0a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb23c0a0b70 0x7fb23c0a3030 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb2480098c0 tx=0x7fb248019040 comp rx=0 tx=0).stop 2023-10-17T01:31:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 msgr2=0x7fb254198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 0x7fb254198c50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb24400e9a0 tx=0x7fb24400ecb0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 shutdown_connections 2023-10-17T01:31:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fb23c0a0b70 0x7fb23c0a3030 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb254108c10 0x7fb2541a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2540737b0 0x7fb254198c50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 --2- 172.21.15.45:0/194791702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb254074d90 0x7fb254199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 >> 172.21.15.45:0/194791702 conn(0x7fb2540fc280 msgr2=0x7fb2540fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 shutdown_connections 2023-10-17T01:31:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.445+0000 7fb2591ff700 1 -- 172.21.15.45:0/194791702 wait complete. 2023-10-17T01:31:11.449 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2023-10-17T01:31:11.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-17T01:31:11.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:11 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2806294815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:31:11.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 -- 172.21.15.45:0/2233765244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b501028e0 msgr2=0x7f0b50109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 --2- 172.21.15.45:0/2233765244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b501028e0 0x7f0b50109190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800b0d0 tx=0x7f0b4800b3e0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 -- 172.21.15.45:0/2233765244 shutdown_connections 2023-10-17T01:31:11.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 --2- 172.21.15.45:0/2233765244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b501028e0 0x7f0b50109190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 --2- 172.21.15.45:0/2233765244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b50100d50 0x7f0b50101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 --2- 172.21.15.45:0/2233765244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 0x7f0b501023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.669+0000 7f0b55bff700 1 -- 172.21.15.45:0/2233765244 >> 172.21.15.45:0/2233765244 conn(0x7f0b500fc280 msgr2=0x7f0b500fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:11.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.670+0000 7f0b55bff700 1 -- 172.21.15.45:0/2233765244 shutdown_connections 2023-10-17T01:31:11.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.670+0000 7f0b55bff700 1 -- 172.21.15.45:0/2233765244 wait complete. 2023-10-17T01:31:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.671+0000 7f0b55bff700 1 Processor -- start 2023-10-17T01:31:11.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.671+0000 7f0b55bff700 1 -- start start 2023-10-17T01:31:11.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b55bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 0x7f0b50196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b55bff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 0x7f0b50197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b55bff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b501028e0 0x7f0b5019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b55bff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019e680 con 0x7f0b50101f40 2023-10-17T01:31:11.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b55bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019e7c0 con 0x7f0b50100d50 2023-10-17T01:31:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b55bff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019e950 con 0x7f0b501028e0 2023-10-17T01:31:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 0x7f0b50196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b501028e0 0x7f0b5019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 0x7f0b50196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.45:40188/0 (socket says 172.21.15.45:40188) 2023-10-17T01:31:11.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4f7fe700 1 -- 172.21.15.45:0/1313078360 learned_addr learned my addr 172.21.15.45:0/1313078360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4effd700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 0x7f0b50197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4ffff700 1 -- 172.21.15.45:0/1313078360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 msgr2=0x7f0b50196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4ffff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 0x7f0b50196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4ffff700 1 -- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 msgr2=0x7f0b50197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4ffff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 0x7f0b50197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4ffff700 1 -- 172.21.15.45:0/1313078360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48009d00 con 0x7f0b501028e0 2023-10-17T01:31:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4f7fe700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 0x7f0b50196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:31:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.672+0000 7f0b4effd700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 0x7f0b50197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:31:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.673+0000 7f0b4ffff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b501028e0 0x7f0b5019e0b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007a30 tx=0x7f0b48008eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:11.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.673+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800e040 con 0x7f0b501028e0 2023-10-17T01:31:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.673+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5019ebd0 con 0x7f0b501028e0 2023-10-17T01:31:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.673+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5019f050 con 0x7f0b501028e0 2023-10-17T01:31:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.673+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4800bd30 con 0x7f0b501028e0 2023-10-17T01:31:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.673+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4801d470 con 0x7f0b501028e0 2023-10-17T01:31:11.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.674+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50108240 con 0x7f0b501028e0 2023-10-17T01:31:11.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.676+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f0b48005000 con 0x7f0b501028e0 2023-10-17T01:31:11.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.677+0000 7f0b4cff9700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0b380a0bf0 0x7f0b380a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:11.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.677+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0b480cd7a0 con 0x7f0b501028e0 2023-10-17T01:31:11.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.678+0000 7f0b4f7fe700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0b380a0bf0 0x7f0b380a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:11.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.678+0000 7f0b4f7fe700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0b380a0bf0 0x7f0b380a30b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005950 tx=0x7f0b40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:11.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.680+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f0b48097190 con 0x7f0b501028e0 2023-10-17T01:31:11.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:11 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2806294815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:31:11.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:11 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2806294815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:31:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.895+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b5002cd90 con 0x7f0b501028e0 2023-10-17T01:31:11.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.899+0000 7f0b4cff9700 1 -- 172.21.15.45:0/1313078360 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22390 (secure 0 0 0) 0x7f0b48025e90 con 0x7f0b501028e0 2023-10-17T01:31:11.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-17T01:31:11.900 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"2804d2ac-6c89-11ee-8db6-212e2dc638e7","created":"2023-10-17T01:05:32.108363+0000","modified":"2023-10-17T01:13:33.505285+0000","last_up_change":"2023-10-17T01:12:12.428287+0000","last_in_change":"2023-10-17T01:11:54.991971+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-17T01:08:48.955476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-17T01:12:59.332496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-17T01:13:00.462889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84831425-d402-4501-8e75-de13106f9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6803","nonce":563780419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6805","nonce":563780419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6809","nonce":563780419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":563780419},{"type":"v1","addr":"172.21.15.45:6807","nonce":563780419}]},"public_addr":"172.21.15.45:6803/563780419","cluster_addr":"172.21.15.45:6805/563780419","heartbeat_back_addr":"172.21.15.45:6809/563780419","heartbeat_front_addr":"172.21.15.45:6807/563780419","state":["exists","up"]},{"osd":1,"uuid":"a91b91e7-247a-46b5-b229-bdadf8126588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6801","nonce":1446433682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6803","nonce":1446433682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6807","nonce":1446433682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1446433682},{"type":"v1","addr":"172.21.15.60:6805","nonce":1446433682}]},"public_addr":"172.21.15.60:6801/1446433682","cluster_addr":"172.21.15.60:6803/1446433682","heartbeat_back_addr":"172.21.15.60:6807/1446433682","heartbeat_front_addr":"172.21.15.60:6805/1446433682","state":["exists","up"]},{"osd":2,"uuid":"e2b9dd7f-cf88-49c9-bd44-36a439272919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6801","nonce":2332429544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6803","nonce":2332429544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6807","nonce":2332429544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2332429544},{"type":"v1","addr":"172.21.15.181:6805","nonce":2332429544}]},"public_addr":"172.21.15.181:6801/2332429544","cluster_addr":"172.21.15.181:6803/2332429544","heartbeat_back_addr":"172.21.15.181:6807/2332429544","heartbeat_front_addr":"172.21.15.181:6805/2332429544","state":["exists","up"]},{"osd":3,"uuid":"806e62ab-4fa6-4c3f-80b7-eb25f498baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6811","nonce":570572613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6813","nonce":570572613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6817","nonce":570572613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":570572613},{"type":"v1","addr":"172.21.15.45:6815","nonce":570572613}]},"public_addr":"172.21.15.45:6811/570572613","cluster_addr":"172.21.15.45:6813/570572613","heartbeat_back_addr":"172.21.15.45:6817/570572613","heartbeat_front_addr":"172.21.15.45:6815/570572613","state":["exists","up"]},{"osd":4,"uuid":"3232fc8e-ae84-448d-9665-adc7f64f6c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6809","nonce":2865514976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6811","nonce":2865514976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6815","nonce":2865514976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2865514976},{"type":"v1","addr":"172.21.15.60:6813","nonce":2865514976}]},"public_addr":"172.21.15.60:6809/2865514976","cluster_addr":"172.21.15.60:6811/2865514976","heartbeat_back_addr":"172.21.15.60:6815/2865514976","heartbeat_front_addr":"172.21.15.60:6813/2865514976","state":["exists","up"]},{"osd":5,"uuid":"28b10aa9-ecab-4539-9fa5-e753609c3bb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6809","nonce":547299005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6811","nonce":547299005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6815","nonce":547299005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":547299005},{"type":"v1","addr":"172.21.15.181:6813","nonce":547299005}]},"public_addr":"172.21.15.181:6809/547299005","cluster_addr":"172.21.15.181:6811/547299005","heartbeat_back_addr":"172.21.15.181:6815/547299005","heartbeat_front_addr":"172.21.15.181:6813/547299005","state":["exists","up"]},{"osd":6,"uuid":"fb616844-fabd-44f0-9347-9868616fa738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6819","nonce":2740142104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6821","nonce":2740142104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6825","nonce":2740142104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":2740142104},{"type":"v1","addr":"172.21.15.45:6823","nonce":2740142104}]},"public_addr":"172.21.15.45:6819/2740142104","cluster_addr":"172.21.15.45:6821/2740142104","heartbeat_back_addr":"172.21.15.45:6825/2740142104","heartbeat_front_addr":"172.21.15.45:6823/2740142104","state":["exists","up"]},{"osd":7,"uuid":"d58f8b41-8d53-4e6c-b2d2-7997cb77e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6817","nonce":4084805853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6819","nonce":4084805853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6823","nonce":4084805853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":4084805853},{"type":"v1","addr":"172.21.15.60:6821","nonce":4084805853}]},"public_addr":"172.21.15.60:6817/4084805853","cluster_addr":"172.21.15.60:6819/4084805853","heartbeat_back_addr":"172.21.15.60:6823/4084805853","heartbeat_front_addr":"172.21.15.60:6821/4084805853","state":["exists","up"]},{"osd":8,"uuid":"58034e5c-8cae-4694-8a22-bfec322b43b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6817","nonce":3559316201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6819","nonce":3559316201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6823","nonce":3559316201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3559316201},{"type":"v1","addr":"172.21.15.181:6821","nonce":3559316201}]},"public_addr":"172.21.15.181:6817/3559316201","cluster_addr":"172.21.15.181:6819/3559316201","heartbeat_back_addr":"172.21.15.181:6823/3559316201","heartbeat_front_addr":"172.21.15.181:6821/3559316201","state":["exists","up"]},{"osd":9,"uuid":"f6c8b883-1f5b-4e01-8453-7ecae38051e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6827","nonce":1619950386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6829","nonce":1619950386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6833","nonce":1619950386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1619950386},{"type":"v1","addr":"172.21.15.45:6831","nonce":1619950386}]},"public_addr":"172.21.15.45:6827/1619950386","cluster_addr":"172.21.15.45:6829/1619950386","heartbeat_back_addr":"172.21.15.45:6833/1619950386","heartbeat_front_addr":"172.21.15.45:6831/1619950386","state":["exists","up"]},{"osd":10,"uuid":"50b41c1b-61b5-4b1b-9c52-93d9adb7e080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6825","nonce":1158474064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6827","nonce":1158474064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6831","nonce":1158474064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1158474064},{"type":"v1","addr":"172.21.15.60:6829","nonce":1158474064}]},"public_addr":"172.21.15.60:6825/1158474064","cluster_addr":"172.21.15.60:6827/1158474064","heartbeat_back_addr":"172.21.15.60:6831/1158474064","heartbeat_front_addr":"172.21.15.60:6829/1158474064","state":["exists","up"]},{"osd":11,"uuid":"6a70d229-f101-42fe-8231-5e3cc447d528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6825","nonce":429412155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6827","nonce":429412155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6831","nonce":429412155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":429412155},{"type":"v1","addr":"172.21.15.181:6829","nonce":429412155}]},"public_addr":"172.21.15.181:6825/429412155","cluster_addr":"172.21.15.181:6827/429412155","heartbeat_back_addr":"172.21.15.181:6831/429412155","heartbeat_front_addr":"172.21.15.181:6829/429412155","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:04.227928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:25.984528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:08:46.325477+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:11.867888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:33.459701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:09:53.688814+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:19.590265+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:10:42.299566+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:02.847603+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:28.664301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:11:50.084677+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-17T01:12:10.808258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/518390758":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6801/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/1775908551":"2023-10-18T01:06:32.348953+0000","172.21.15.45:6800/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/3128471816":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2130990792":"2023-10-18T01:06:05.897832+0000","172.21.15.45:6801/3224370568":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/887015676":"2023-10-18T01:06:32.348953+0000","172.21.15.45:0/4289245093":"2023-10-18T01:06:05.897832+0000","172.21.15.45:0/2694295985":"2023-10-18T01:06:32.348953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-17T01:31:11.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0b380a0bf0 msgr2=0x7f0b380a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0b380a0bf0 0x7f0b380a30b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005950 tx=0x7f0b40009260 comp rx=0 tx=0).stop 2023-10-17T01:31:11.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b501028e0 msgr2=0x7f0b5019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b501028e0 0x7f0b5019e0b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007a30 tx=0x7f0b48008eb0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 shutdown_connections 2023-10-17T01:31:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f0b380a0bf0 0x7f0b380a30b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b501028e0 0x7f0b5019e0b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b50100d50 0x7f0b50196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 --2- 172.21.15.45:0/1313078360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b50101f40 0x7f0b50197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.900+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 >> 172.21.15.45:0/1313078360 conn(0x7f0b500fc280 msgr2=0x7f0b50105380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.901+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 shutdown_connections 2023-10-17T01:31:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:11.901+0000 7f0b55bff700 1 -- 172.21.15.45:0/1313078360 wait complete. 2023-10-17T01:31:11.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-17T01:31:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.125+0000 7f39dc1ff700 1 -- 172.21.15.45:0/3309574721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 msgr2=0x7f39d4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.125+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/3309574721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d4102390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009990 tx=0x7f39c4009ca0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 -- 172.21.15.45:0/3309574721 shutdown_connections 2023-10-17T01:31:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/3309574721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/3309574721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 0x7f39d4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/3309574721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d4102390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 -- 172.21.15.45:0/3309574721 >> 172.21.15.45:0/3309574721 conn(0x7f39d40fc290 msgr2=0x7f39d40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 -- 172.21.15.45:0/3309574721 shutdown_connections 2023-10-17T01:31:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.126+0000 7f39dc1ff700 1 -- 172.21.15.45:0/3309574721 wait complete. 2023-10-17T01:31:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.127+0000 7f39dc1ff700 1 Processor -- start 2023-10-17T01:31:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 -- start start 2023-10-17T01:31:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 0x7f39d4198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d41a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41a0820 con 0x7f39d41028d0 2023-10-17T01:31:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41a0990 con 0x7f39d4100d40 2023-10-17T01:31:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39dc1ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41a0b50 con 0x7f39d4101f30 2023-10-17T01:31:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39da79c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d41a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39d979a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39d9f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 0x7f39d4198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39da79c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d41a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45046/0 (socket says 172.21.15.45:45046) 2023-10-17T01:31:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39d979a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d41991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:34972/0 (socket says 172.21.15.45:34972) 2023-10-17T01:31:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.128+0000 7f39da79c700 1 -- 172.21.15.45:0/644367608 learned_addr learned my addr 172.21.15.45:0/644367608 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39da79c700 1 -- 172.21.15.45:0/644367608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 msgr2=0x7f39d41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39da79c700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39da79c700 1 -- 172.21.15.45:0/644367608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 msgr2=0x7f39d4198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39da79c700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 0x7f39d4198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39da79c700 1 -- 172.21.15.45:0/644367608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc009c40 con 0x7f39d41028d0 2023-10-17T01:31:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39d9f9b700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 0x7f39d4198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-17T01:31:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39da79c700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d41a0250 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c370 tx=0x7f39cc00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc017070 con 0x7f39d41028d0 2023-10-17T01:31:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c40095f0 con 0x7f39d41028d0 2023-10-17T01:31:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39cc00f040 con 0x7f39d41028d0 2023-10-17T01:31:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00e050 con 0x7f39d41028d0 2023-10-17T01:31:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.129+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d41a1150 con 0x7f39d41028d0 2023-10-17T01:31:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.130+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d410c5a0 con 0x7f39d41028d0 2023-10-17T01:31:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.132+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7f39cc00cb60 con 0x7f39d41028d0 2023-10-17T01:31:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.134+0000 7f39caffd700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f39c00a0b20 0x7f39c00a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.135+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f39cc0cbb50 con 0x7f39d41028d0 2023-10-17T01:31:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.135+0000 7f39d9f9b700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f39c00a0b20 0x7f39c00a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.135+0000 7f39d9f9b700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f39c00a0b20 0x7f39c00a2fe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f39d4199b80 tx=0x7f39d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.139+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7f39cc0954b0 con 0x7f39d41028d0 2023-10-17T01:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.397+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f39d41a1700 con 0x7f39d41028d0 2023-10-17T01:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.406+0000 7f39caffd700 1 -- 172.21.15.45:0/644367608 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f39cc098ad0 con 0x7f39d41028d0 2023-10-17T01:31:12.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f39c00a0b20 msgr2=0x7f39c00a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f39c00a0b20 0x7f39c00a2fe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f39d4199b80 tx=0x7f39d0009260 comp rx=0 tx=0).stop 2023-10-17T01:31:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 msgr2=0x7f39d41a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d41a0250 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c370 tx=0x7f39cc00c680 comp rx=0 tx=0).stop 2023-10-17T01:31:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 shutdown_connections 2023-10-17T01:31:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7f39c00a0b20 0x7f39c00a2fe0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d41028d0 0x7f39d41a0250 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39d4100d40 0x7f39d4198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 --2- 172.21.15.45:0/644367608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d4101f30 0x7f39d41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 >> 172.21.15.45:0/644367608 conn(0x7f39d40fc290 msgr2=0x7f39d40fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:12.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 shutdown_connections 2023-10-17T01:31:12.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.410+0000 7f39dc1ff700 1 -- 172.21.15.45:0/644367608 wait complete. 2023-10-17T01:31:12.414 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-17T01:31:12.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-17T01:31:12.657 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c[171726]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.658 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c[171726]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.658 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mds[171750]: -- 172.21.15.181:0/2981638450 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566e24cf1e0 con 0x5566e24a4800 2023-10-17T01:31:12.658 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c[171726]: starting mds.c at 2023-10-17T01:31:12.658 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f[171386]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.659 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f[171386]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.659 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mds[171410]: -- 172.21.15.181:0/2823507795 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621593011e0 con 0x5621592d6800 2023-10-17T01:31:12.659 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f[171386]: starting mds.f at 2023-10-17T01:31:12.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mon[134520]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-17T01:31:12.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/194791702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:31:12.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1313078360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:31:12.660 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i[172069]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.661 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i[172069]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.661 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mds[172093]: -- 172.21.15.181:0/1330947041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c9d8f0b1e0 con 0x55c9d8ee1000 2023-10-17T01:31:12.661 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i[172069]: starting mds.i at 2023-10-17T01:31:12.661 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l[171044]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.661 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l[171044]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.662 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-mds[171068]: -- 172.21.15.181:0/2286305585 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56270c6b71e0 con 0x56270c68c800 2023-10-17T01:31:12.662 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:31:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l[171044]: starting mds.l at 2023-10-17T01:31:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.660+0000 7fa99e5ff700 1 -- 172.21.15.45:0/92846009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9980737b0 msgr2=0x7fa998073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.660+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/92846009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9980737b0 0x7fa998073c30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009990 tx=0x7fa98c009ca0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 -- 172.21.15.45:0/92846009 shutdown_connections 2023-10-17T01:31:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/92846009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa998108c00 0x7fa99810b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/92846009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa998074d90 0x7fa9980731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/92846009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9980737b0 0x7fa998073c30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 -- 172.21.15.45:0/92846009 >> 172.21.15.45:0/92846009 conn(0x7fa9980fc290 msgr2=0x7fa9980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 -- 172.21.15.45:0/92846009 shutdown_connections 2023-10-17T01:31:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.661+0000 7fa99e5ff700 1 -- 172.21.15.45:0/92846009 wait complete. 2023-10-17T01:31:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 Processor -- start 2023-10-17T01:31:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 -- start start 2023-10-17T01:31:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 0x7fa998198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa998074d90 0x7fa9981991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa998108c00 0x7fa9981a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0810 con 0x7fa998074d90 2023-10-17T01:31:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0980 con 0x7fa998108c00 2023-10-17T01:31:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.662+0000 7fa99e5ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0b40 con 0x7fa9980737b0 2023-10-17T01:31:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 0x7fa998198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa998074d90 0x7fa9981991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 0x7fa998198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.45:35034/0 (socket says 172.21.15.45:35034) 2023-10-17T01:31:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 -- 172.21.15.45:0/1550695303 learned_addr learned my addr 172.21.15.45:0/1550695303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa99cb9c700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa998108c00 0x7fa9981a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 -- 172.21.15.45:0/1550695303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa998108c00 msgr2=0x7fa9981a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa998108c00 0x7fa9981a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 -- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa998074d90 msgr2=0x7fa9981991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa998074d90 0x7fa9981991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 -- 172.21.15.45:0/1550695303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0095f0 con 0x7fa9980737b0 2023-10-17T01:31:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa997fff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 0x7fa998198ca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa98800e9a0 tx=0x7fa98800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988009790 con 0x7fa9980737b0 2023-10-17T01:31:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.663+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa988010430 con 0x7fa9980737b0 2023-10-17T01:31:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.664+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9981a0e20 con 0x7fa9980737b0 2023-10-17T01:31:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.664+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988018680 con 0x7fa9980737b0 2023-10-17T01:31:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.664+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a1250 con 0x7fa9980737b0 2023-10-17T01:31:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.665+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9981072b0 con 0x7fa9980737b0 2023-10-17T01:31:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.667+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7fa9880105a0 con 0x7fa9980737b0 2023-10-17T01:31:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.668+0000 7fa9957fa700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9800a0ba0 0x7fa9800a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.668+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8612+0+0 (secure 0 0 0) 0x7fa988014070 con 0x7fa9980737b0 2023-10-17T01:31:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.668+0000 7fa9977fe700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9800a0ba0 0x7fa9800a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.669+0000 7fa9977fe700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9800a0ba0 0x7fa9800a3060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00b2e0 tx=0x7fa98c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.670+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7fa988096060 con 0x7fa9980737b0 2023-10-17T01:31:12.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-mon[98474]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-17T01:31:12.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/194791702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:31:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1313078360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:31:12.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a[98450]: 2023-10-17T01:31:12.398+0000 7ff95f484700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-17T01:31:12.802 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a[149125]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.803 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a[149125]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.803 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-mds[149149]: -- 172.21.15.45:0/3073359601 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56009c6951e0 con 0x56009c66c800 2023-10-17T01:31:12.803 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a[149125]: starting mds.a at 2023-10-17T01:31:12.803 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g[148731]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.804 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g[148731]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.804 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-mds[148755]: -- 172.21.15.45:0/436283962 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55826831b1e0 con 0x5582682f2800 2023-10-17T01:31:12.804 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:31:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g[148731]: starting mds.g at 2023-10-17T01:31:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-mon[134136]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-17T01:31:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/194791702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-17T01:31:12.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1313078360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-17T01:31:12.829 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e[170332]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.829 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e[170332]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.829 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e[170332]: starting mds.e at 2023-10-17T01:31:12.830 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.830 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.830 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-mds[171037]: -- 172.21.15.60:0/1043548330 <== mon.2 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556b5b0591e0 con 0x556b5b02ec00 2023-10-17T01:31:12.830 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: starting mds.b at 2023-10-17T01:31:12.831 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h[170672]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.831 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h[170672]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.831 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-mds[170696]: -- 172.21.15.60:0/90694844 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5600315f91e0 con 0x5600315ce800 2023-10-17T01:31:12.831 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h[170672]: starting mds.h at 2023-10-17T01:31:12.832 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k[169992]: ignoring --setuser ceph since I am not root 2023-10-17T01:31:12.832 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k[169992]: ignoring --setgroup ceph since I am not root 2023-10-17T01:31:12.832 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-mds[170016]: -- 172.21.15.60:0/2641705035 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a3b1931e0 con 0x562a3b168800 2023-10-17T01:31:12.832 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:31:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k[169992]: starting mds.k at 2023-10-17T01:31:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:12.929+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa99802cea0 con 0x7fa9980737b0 2023-10-17T01:31:13.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.416+0000 7fa9957fa700 1 -- 172.21.15.45:0/1550695303 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa988099680 con 0x7fa9980737b0 2023-10-17T01:31:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.419+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9800a0ba0 msgr2=0x7fa9800a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.419+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9800a0ba0 0x7fa9800a3060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00b2e0 tx=0x7fa98c005fb0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.419+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 msgr2=0x7fa998198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.419+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 0x7fa998198ca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa98800e9a0 tx=0x7fa98800ecb0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 shutdown_connections 2023-10-17T01:31:13.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7fa9800a0ba0 0x7fa9800a3060 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa998108c00 0x7fa9981a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980737b0 0x7fa998198ca0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 --2- 172.21.15.45:0/1550695303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa998074d90 0x7fa9981991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 >> 172.21.15.45:0/1550695303 conn(0x7fa9980fc290 msgr2=0x7fa9980fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:13.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 shutdown_connections 2023-10-17T01:31:13.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.420+0000 7fa99e5ff700 1 -- 172.21.15.45:0/1550695303 wait complete. 2023-10-17T01:31:13.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-17T01:31:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 -- 172.21.15.45:0/2642634117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c108c00 msgr2=0x7efc2c10b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 --2- 172.21.15.45:0/2642634117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c108c00 0x7efc2c10b090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efc2400b0a0 tx=0x7efc2400b3b0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 -- 172.21.15.45:0/2642634117 shutdown_connections 2023-10-17T01:31:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 --2- 172.21.15.45:0/2642634117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c108c00 0x7efc2c10b090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 --2- 172.21.15.45:0/2642634117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c074d90 0x7efc2c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 --2- 172.21.15.45:0/2642634117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc2c0737b0 0x7efc2c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.654+0000 7efc337ff700 1 -- 172.21.15.45:0/2642634117 >> 172.21.15.45:0/2642634117 conn(0x7efc2c0fc290 msgr2=0x7efc2c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.655+0000 7efc337ff700 1 -- 172.21.15.45:0/2642634117 shutdown_connections 2023-10-17T01:31:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.655+0000 7efc337ff700 1 -- 172.21.15.45:0/2642634117 wait complete. 2023-10-17T01:31:13.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/644367608' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-17T01:31:13.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-17T01:31:13.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-17T01:31:13.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: osdmap e86: 12 total, 12 up, 12 in 2023-10-17T01:31:13.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/644367608' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-17T01:31:13.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-17T01:31:13.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/1550695303' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:31:13.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:13 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:31:13.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 Processor -- start 2023-10-17T01:31:13.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 -- start start 2023-10-17T01:31:13.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc2c0737b0 0x7efc2c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:13.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 0x7efc2c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:13.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c108c00 0x7efc2c1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:13.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1a0930 con 0x7efc2c074d90 2023-10-17T01:31:13.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1a0a70 con 0x7efc2c108c00 2023-10-17T01:31:13.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc337ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1a0c00 con 0x7efc2c0737b0 2023-10-17T01:31:13.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc30d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 0x7efc2c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:13.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc30d9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 0x7efc2c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45116/0 (socket says 172.21.15.45:45116) 2023-10-17T01:31:13.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc30d9a700 1 -- 172.21.15.45:0/2903630406 learned_addr learned my addr 172.21.15.45:0/2903630406 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:13.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc3159b700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc2c0737b0 0x7efc2c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:13.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.656+0000 7efc31d9c700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c108c00 0x7efc2c1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:13.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc30d9a700 1 -- 172.21.15.45:0/2903630406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc2c0737b0 msgr2=0x7efc2c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:13.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc30d9a700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc2c0737b0 0x7efc2c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc30d9a700 1 -- 172.21.15.45:0/2903630406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c108c00 msgr2=0x7efc2c1a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:13.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc30d9a700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c108c00 0x7efc2c1a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:13.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc30d9a700 1 -- 172.21.15.45:0/2903630406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24009d00 con 0x7efc2c074d90 2023-10-17T01:31:13.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc31d9c700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c108c00 0x7efc2c1a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-17T01:31:13.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc30d9a700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 0x7efc2c1992f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e970 tx=0x7efc1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:13.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c009790 con 0x7efc2c074d90 2023-10-17T01:31:13.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc1c00eec0 con 0x7efc2c074d90 2023-10-17T01:31:13.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c003cf0 con 0x7efc2c074d90 2023-10-17T01:31:13.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c1a0ee0 con 0x7efc2c074d90 2023-10-17T01:31:13.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.657+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c1a1310 con 0x7efc2c074d90 2023-10-17T01:31:13.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.659+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c1072b0 con 0x7efc2c074d90 2023-10-17T01:31:13.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.661+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7efc1c00ccd0 con 0x7efc2c074d90 2023-10-17T01:31:13.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.662+0000 7efc227fc700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc180a0ba0 0x7efc180a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:13.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.662+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8612+0+0 (secure 0 0 0) 0x7efc1c014070 con 0x7efc2c074d90 2023-10-17T01:31:13.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.662+0000 7efc3159b700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc180a0ba0 0x7efc180a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:13.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.663+0000 7efc3159b700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc180a0ba0 0x7efc180a3060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efc28009c60 tx=0x7efc28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:13.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.666+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7efc1c095d10 con 0x7efc2c074d90 2023-10-17T01:31:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/644367608' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-17T01:31:13.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-17T01:31:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-17T01:31:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: osdmap e86: 12 total, 12 up, 12 in 2023-10-17T01:31:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/644367608' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-17T01:31:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-17T01:31:13.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/1550695303' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:31:13.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:13 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:31:13.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/644367608' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-17T01:31:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-17T01:31:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-17T01:31:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: osdmap e86: 12 total, 12 up, 12 in 2023-10-17T01:31:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/644367608' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-17T01:31:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-17T01:31:13.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/1550695303' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:31:13.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:13 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-17T01:31:13.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:13.880+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efc2c02cc30 con 0x7efc2c074d90 2023-10-17T01:31:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.416+0000 7efc227fc700 1 -- 172.21.15.45:0/2903630406 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7efc1c099330 con 0x7efc2c074d90 2023-10-17T01:31:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc180a0ba0 msgr2=0x7efc180a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc180a0ba0 0x7efc180a3060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efc28009c60 tx=0x7efc28009260 comp rx=0 tx=0).stop 2023-10-17T01:31:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 msgr2=0x7efc2c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 0x7efc2c1992f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e970 tx=0x7efc1c00ec80 comp rx=0 tx=0).stop 2023-10-17T01:31:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 shutdown_connections 2023-10-17T01:31:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7efc180a0ba0 0x7efc180a3060 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c108c00 0x7efc2c1a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc2c0737b0 0x7efc2c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 --2- 172.21.15.45:0/2903630406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc2c074d90 0x7efc2c1992f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.421+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 >> 172.21.15.45:0/2903630406 conn(0x7efc2c0fc290 msgr2=0x7efc2c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.422+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 shutdown_connections 2023-10-17T01:31:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.422+0000 7efc337ff700 1 -- 172.21.15.45:0/2903630406 wait complete. 2023-10-17T01:31:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-17T01:31:14.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-17T01:31:14.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-10-17T01:31:14.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-17T01:31:14.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-17T01:31:14.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: Cluster is now healthy 2023-10-17T01:31:14.658 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6834/4119756644,v1:172.21.15.60:6835/4119756644] up:boot 2023-10-17T01:31:14.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6837/4068078907,v1:172.21.15.181:6839/4068078907] up:boot 2023-10-17T01:31:14.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6832/2683274465,v1:172.21.15.181:6835/2683274465] up:boot 2023-10-17T01:31:14.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6833/2585269694,v1:172.21.15.181:6834/2585269694] up:boot 2023-10-17T01:31:14.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6832/4293204402,v1:172.21.15.60:6833/4293204402] up:boot 2023-10-17T01:31:14.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6836/3874232977,v1:172.21.15.60:6837/3874232977] up:boot 2023-10-17T01:31:14.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.60:6838/3300557860,v1:172.21.15.60:6839/3300557860] up:boot 2023-10-17T01:31:14.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.181:6836/953754297,v1:172.21.15.181:6838/953754297] up:boot 2023-10-17T01:31:14.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.45:6838/2152494253,v1:172.21.15.45:6839/2152494253] up:boot 2023-10-17T01:31:14.660 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: mds.? [v2:172.21.15.45:6840/4074869607,v1:172.21.15.45:6841/4074869607] up:boot 2023-10-17T01:31:14.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-17T01:31:14.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: fsmap 12 up:standby 2023-10-17T01:31:14.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-17T01:31:14.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-17T01:31:14.661 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-17T01:31:14.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-17T01:31:14.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-17T01:31:14.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-17T01:31:14.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-17T01:31:14.662 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-17T01:31:14.663 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-17T01:31:14.663 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-17T01:31:14.663 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:14 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2903630406' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-17T01:31:14.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 -- 172.21.15.45:0/2620455446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048101f50 msgr2=0x7ff0481023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:14.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2620455446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048101f50 0x7ff0481023b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0099c0 tx=0x7ff03c009cd0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 -- 172.21.15.45:0/2620455446 shutdown_connections 2023-10-17T01:31:14.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2620455446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0481028f0 0x7ff0481091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2620455446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff048100d60 0x7ff048101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2620455446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048101f50 0x7ff0481023b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 -- 172.21.15.45:0/2620455446 >> 172.21.15.45:0/2620455446 conn(0x7ff0480fc2d0 msgr2=0x7ff0480fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:14.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 -- 172.21.15.45:0/2620455446 shutdown_connections 2023-10-17T01:31:14.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.657+0000 7ff04cdff700 1 -- 172.21.15.45:0/2620455446 wait complete. 2023-10-17T01:31:14.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.658+0000 7ff04cdff700 1 Processor -- start 2023-10-17T01:31:14.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.658+0000 7ff04cdff700 1 -- start start 2023-10-17T01:31:14.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04cdff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 0x7ff048198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:14.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04cdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff048101f50 0x7ff0481991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04cdff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0481028f0 0x7ff0481a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04cdff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0481a07f0 con 0x7ff048100d60 2023-10-17T01:31:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04cdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0481a0960 con 0x7ff048101f50 2023-10-17T01:31:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04cdff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0481a0b20 con 0x7ff0481028f0 2023-10-17T01:31:14.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 0x7ff048198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 0x7ff048198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45124/0 (socket says 172.21.15.45:45124) 2023-10-17T01:31:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff04659c700 1 -- 172.21.15.45:0/2864500388 learned_addr learned my addr 172.21.15.45:0/2864500388 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-17T01:31:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff046d9d700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0481028f0 0x7ff0481a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:14.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.659+0000 7ff045d9b700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff048101f50 0x7ff0481991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04659c700 1 -- 172.21.15.45:0/2864500388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0481028f0 msgr2=0x7ff0481a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04659c700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0481028f0 0x7ff0481a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04659c700 1 -- 172.21.15.45:0/2864500388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff048101f50 msgr2=0x7ff0481991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04659c700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff048101f50 0x7ff0481991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:14.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04659c700 1 -- 172.21.15.45:0/2864500388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0095f0 con 0x7ff048100d60 2023-10-17T01:31:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04659c700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 0x7ff048198c80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d980 tx=0x7ff03800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038004020 con 0x7ff048100d60 2023-10-17T01:31:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03800f690 con 0x7ff048100d60 2023-10-17T01:31:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038003a60 con 0x7ff048100d60 2023-10-17T01:31:14.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0481a0e00 con 0x7ff048100d60 2023-10-17T01:31:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.660+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0481a1220 con 0x7ff048100d60 2023-10-17T01:31:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.661+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04810c5c0 con 0x7ff048100d60 2023-10-17T01:31:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.663+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 135532+0+0 (secure 0 0 0) 0x7ff03800fb00 con 0x7ff048100d60 2023-10-17T01:31:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.665+0000 7ff0377fe700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff0300a0ba0 0x7ff0300a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-17T01:31:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.665+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8058+0+0 (secure 0 0 0) 0x7ff0380cbf20 con 0x7ff048100d60 2023-10-17T01:31:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.665+0000 7ff045d9b700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff0300a0ba0 0x7ff0300a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-17T01:31:14.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.666+0000 7ff045d9b700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff0300a0ba0 0x7ff0300a3060 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff03c000c00 tx=0x7ff03c009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-17T01:31:14.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.667+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180373 (secure 0 0 0) 0x7ff038095a30 con 0x7ff048100d60 2023-10-17T01:31:14.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-10-17T01:31:14.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-17T01:31:14.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-17T01:31:14.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: Cluster is now healthy 2023-10-17T01:31:14.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6834/4119756644,v1:172.21.15.60:6835/4119756644] up:boot 2023-10-17T01:31:14.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6837/4068078907,v1:172.21.15.181:6839/4068078907] up:boot 2023-10-17T01:31:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6832/2683274465,v1:172.21.15.181:6835/2683274465] up:boot 2023-10-17T01:31:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6833/2585269694,v1:172.21.15.181:6834/2585269694] up:boot 2023-10-17T01:31:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6832/4293204402,v1:172.21.15.60:6833/4293204402] up:boot 2023-10-17T01:31:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6836/3874232977,v1:172.21.15.60:6837/3874232977] up:boot 2023-10-17T01:31:14.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.60:6838/3300557860,v1:172.21.15.60:6839/3300557860] up:boot 2023-10-17T01:31:14.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.181:6836/953754297,v1:172.21.15.181:6838/953754297] up:boot 2023-10-17T01:31:14.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.45:6838/2152494253,v1:172.21.15.45:6839/2152494253] up:boot 2023-10-17T01:31:14.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: mds.? [v2:172.21.15.45:6840/4074869607,v1:172.21.15.45:6841/4074869607] up:boot 2023-10-17T01:31:14.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-17T01:31:14.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: fsmap 12 up:standby 2023-10-17T01:31:14.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-17T01:31:14.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-17T01:31:14.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-17T01:31:14.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-17T01:31:14.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-17T01:31:14.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-17T01:31:14.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-17T01:31:14.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-17T01:31:14.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-17T01:31:14.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-17T01:31:14.807 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:14 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2903630406' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-17T01:31:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-10-17T01:31:14.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-17T01:31:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-17T01:31:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: Cluster is now healthy 2023-10-17T01:31:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6834/4119756644,v1:172.21.15.60:6835/4119756644] up:boot 2023-10-17T01:31:14.829 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6837/4068078907,v1:172.21.15.181:6839/4068078907] up:boot 2023-10-17T01:31:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6832/2683274465,v1:172.21.15.181:6835/2683274465] up:boot 2023-10-17T01:31:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6833/2585269694,v1:172.21.15.181:6834/2585269694] up:boot 2023-10-17T01:31:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6832/4293204402,v1:172.21.15.60:6833/4293204402] up:boot 2023-10-17T01:31:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6836/3874232977,v1:172.21.15.60:6837/3874232977] up:boot 2023-10-17T01:31:14.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.60:6838/3300557860,v1:172.21.15.60:6839/3300557860] up:boot 2023-10-17T01:31:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.181:6836/953754297,v1:172.21.15.181:6838/953754297] up:boot 2023-10-17T01:31:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.45:6838/2152494253,v1:172.21.15.45:6839/2152494253] up:boot 2023-10-17T01:31:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: mds.? [v2:172.21.15.45:6840/4074869607,v1:172.21.15.45:6841/4074869607] up:boot 2023-10-17T01:31:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-17T01:31:14.831 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: fsmap 12 up:standby 2023-10-17T01:31:14.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-17T01:31:14.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-17T01:31:14.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-17T01:31:14.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-17T01:31:14.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-17T01:31:14.832 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-17T01:31:14.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-17T01:31:14.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-17T01:31:14.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-17T01:31:14.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='mgr.14150 172.21.15.45:0/1646380698' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-17T01:31:14.833 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:14 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2903630406' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-17T01:31:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:14.880+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0481a1ac0 con 0x7ff048100d60 2023-10-17T01:31:15.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.424+0000 7ff0377fe700 1 -- 172.21.15.45:0/2864500388 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff038099050 con 0x7ff048100d60 2023-10-17T01:31:15.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff0300a0ba0 msgr2=0x7ff0300a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:15.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff0300a0ba0 0x7ff0300a3060 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff03c000c00 tx=0x7ff03c009f10 comp rx=0 tx=0).stop 2023-10-17T01:31:15.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 msgr2=0x7ff048198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-17T01:31:15.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 0x7ff048198c80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d980 tx=0x7ff03800dc90 comp rx=0 tx=0).stop 2023-10-17T01:31:15.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 shutdown_connections 2023-10-17T01:31:15.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:6800/869456926,v1:172.21.15.45:6801/869456926] conn(0x7ff0300a0ba0 0x7ff0300a3060 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:15.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0481028f0 0x7ff0481a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:15.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff048100d60 0x7ff048198c80 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d980 tx=0x7ff03800dc90 comp rx=0 tx=0).stop 2023-10-17T01:31:15.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 --2- 172.21.15.45:0/2864500388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff048101f50 0x7ff0481991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-17T01:31:15.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.428+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 >> 172.21.15.45:0/2864500388 conn(0x7ff0480fc2d0 msgr2=0x7ff0480fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-17T01:31:15.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.429+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 shutdown_connections 2023-10-17T01:31:15.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-17T01:31:15.429+0000 7ff04cdff700 1 -- 172.21.15.45:0/2864500388 wait complete. 2023-10-17T01:31:15.433 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-17T01:31:15.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:31:15.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:31:15.677 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:15 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2903630406' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-17T01:31:15.678 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:15 smithi060 ceph-mon[134136]: osdmap e87: 12 total, 12 up, 12 in 2023-10-17T01:31:15.678 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:15 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2864500388' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-17T01:31:15.703 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:31:15.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:15 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2903630406' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-17T01:31:15.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:15 smithi045 ceph-mon[98474]: osdmap e87: 12 total, 12 up, 12 in 2023-10-17T01:31:15.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:15 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2864500388' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-17T01:31:15.802 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:15 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2903630406' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-17T01:31:15.802 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:15 smithi181 ceph-mon[134520]: osdmap e87: 12 total, 12 up, 12 in 2023-10-17T01:31:15.802 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:15 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2864500388' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-17T01:31:15.837 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-17T01:31:15.838 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-17T01:31:15.902 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-17T01:31:15.919 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-17T01:31:15.936 INFO:tasks.cephadm:Stopping all daemons... 2023-10-17T01:31:15.936 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-17T01:31:15.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a 2023-10-17T01:31:16.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 systemd[1]: Stopping Ceph mon.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:16.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 ceph-mon[98474]: pgmap v818: 65 pgs: 65 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:31:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 ceph-mon[98474]: from='client.? 172.21.15.45:0/2864500388' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-17T01:31:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 ceph-mon[98474]: osdmap e88: 12 total, 12 up, 12 in 2023-10-17T01:31:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a[98450]: 2023-10-17T01:31:16.484+0000 7ff964c8f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:16.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a[98450]: 2023-10-17T01:31:16.484+0000 7ff964c8f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-17T01:31:16.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:16 smithi060 ceph-mon[134136]: pgmap v818: 65 pgs: 65 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:31:16.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:16 smithi060 ceph-mon[134136]: from='client.? 172.21.15.45:0/2864500388' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-17T01:31:16.830 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:16 smithi060 ceph-mon[134136]: osdmap e88: 12 total, 12 up, 12 in 2023-10-17T01:31:16.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:16 smithi181 ceph-mon[134520]: pgmap v818: 65 pgs: 65 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-17T01:31:16.908 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:16 smithi181 ceph-mon[134520]: from='client.? 172.21.15.45:0/2864500388' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-17T01:31:16.909 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:16 smithi181 ceph-mon[134520]: osdmap e88: 12 total, 12 up, 12 in 2023-10-17T01:31:17.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Oct 17 01:31:16 smithi045 bash[175334]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-a 2023-10-17T01:31:17.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.a.service' 2023-10-17T01:31:17.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:17.459 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-17T01:31:17.460 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-17T01:31:17.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.b 2023-10-17T01:31:17.828 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:17 smithi060 systemd[1]: Stopping Ceph mon.b for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:18.198 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-b[134112]: 2023-10-17T01:31:17.829+0000 7f1ae7387700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:18.198 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-b[134112]: 2023-10-17T01:31:17.829+0000 7f1ae7387700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-17T01:31:18.551 INFO:journalctl@ceph.mon.b.smithi060.stdout:Oct 17 01:31:18 smithi060 bash[177163]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-b 2023-10-17T01:31:18.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.b.service' 2023-10-17T01:31:18.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:18.611 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-17T01:31:18.611 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-17T01:31:18.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.c 2023-10-17T01:31:18.907 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:18 smithi181 systemd[1]: Stopping Ceph mon.c for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:19.360 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-c[134494]: 2023-10-17T01:31:19.051+0000 7fe50f19e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:19.360 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-c[134494]: 2023-10-17T01:31:19.051+0000 7fe50f19e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-17T01:31:19.657 INFO:journalctl@ceph.mon.c.smithi181.stdout:Oct 17 01:31:19 smithi181 bash[177728]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mon-c 2023-10-17T01:31:19.751 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mon.c.service' 2023-10-17T01:31:19.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:19.828 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-17T01:31:19.828 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-17T01:31:19.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x 2023-10-17T01:31:20.280 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:31:19 smithi045 systemd[1]: Stopping Ceph mgr.x for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:21.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:31:21 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:31:21.128+0000 7f30c26a9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:21.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:31:21 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x[98722]: 2023-10-17T01:31:21.128+0000 7f30c26a9700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-17T01:31:21.666 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Oct 17 01:31:21 smithi045 bash[175479]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-x 2023-10-17T01:31:21.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.x.service' 2023-10-17T01:31:22.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:22.074 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-17T01:31:22.075 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-17T01:31:22.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.y 2023-10-17T01:31:22.510 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:31:22 smithi060 systemd[1]: Stopping Ceph mgr.y for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:22.790 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:31:22 smithi060 bash[177323]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-y 2023-10-17T01:31:22.976 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Oct 17 01:31:22 smithi060 systemd[1]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-17T01:31:22.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.y.service' 2023-10-17T01:31:23.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:23.018 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-17T01:31:23.019 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-17T01:31:23.019 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.z 2023-10-17T01:31:23.407 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:31:23 smithi181 systemd[1]: Stopping Ceph mgr.z for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:23.724 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:31:23 smithi181 bash[177885]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mgr-z 2023-10-17T01:31:23.904 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Oct 17 01:31:23 smithi181 systemd[1]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-17T01:31:23.908 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mgr.z.service' 2023-10-17T01:31:23.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:23.945 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-17T01:31:23.946 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-17T01:31:23.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.0 2023-10-17T01:31:24.301 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:31:24 smithi045 systemd[1]: Stopping Ceph osd.0 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:24.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:31:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0[116817]: 2023-10-17T01:31:24.466+0000 7fcf91c0c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:24.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:31:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0[116817]: 2023-10-17T01:31:24.466+0000 7fcf91c0c700 -1 osd.0 88 *** Got signal Terminated *** 2023-10-17T01:31:24.802 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:31:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0[116817]: 2023-10-17T01:31:24.467+0000 7fcf91c0c700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:31:30.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:Oct 17 01:31:30 smithi045 bash[175629]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-0 2023-10-17T01:31:32.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.0.service' 2023-10-17T01:31:32.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:32.528 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-17T01:31:32.528 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-17T01:31:32.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.1 2023-10-17T01:31:32.828 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:31:32 smithi060 systemd[1]: Stopping Ceph osd.1 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:33.328 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:31:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1[139250]: 2023-10-17T01:31:32.885+0000 7fc905024700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:33.329 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:31:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1[139250]: 2023-10-17T01:31:32.885+0000 7fc905024700 -1 osd.1 88 *** Got signal Terminated *** 2023-10-17T01:31:33.329 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:31:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1[139250]: 2023-10-17T01:31:32.885+0000 7fc905024700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:31:39.078 INFO:journalctl@ceph.osd.1.smithi060.stdout:Oct 17 01:31:38 smithi060 bash[177469]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-1 2023-10-17T01:31:40.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.1.service' 2023-10-17T01:31:40.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:40.195 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-17T01:31:40.195 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-17T01:31:40.196 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.2 2023-10-17T01:31:40.548 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:31:40 smithi181 systemd[1]: Stopping Ceph osd.2 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:40.907 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:31:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2[140308]: 2023-10-17T01:31:40.546+0000 7f70ed4b1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:40.907 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:31:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2[140308]: 2023-10-17T01:31:40.546+0000 7f70ed4b1700 -1 osd.2 88 *** Got signal Terminated *** 2023-10-17T01:31:40.908 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:31:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2[140308]: 2023-10-17T01:31:40.546+0000 7f70ed4b1700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:31:46.657 INFO:journalctl@ceph.osd.2.smithi181.stdout:Oct 17 01:31:46 smithi181 bash[178032]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-2 2023-10-17T01:31:47.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.2.service' 2023-10-17T01:31:47.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:47.737 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-17T01:31:47.737 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-17T01:31:47.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.3 2023-10-17T01:31:48.256 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:47 smithi045 systemd[1]: Stopping Ceph osd.3 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:48.551 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:48 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:31:48.253+0000 7fd3350e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:48.551 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:48 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:31:48.253+0000 7fd3350e2700 -1 osd.3 88 *** Got signal Terminated *** 2023-10-17T01:31:48.552 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:48 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:31:48.253+0000 7fd3350e2700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:31:51.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:51.315+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:52.051 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:51 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:31:51.736+0000 7fd330ed7700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:28.169269+0000 front 2023-10-17T01:31:28.169297+0000 (oldest deadline 2023-10-17T01:31:51.669430+0000) 2023-10-17T01:31:52.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:52.356+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:53.051 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:52 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3[124823]: 2023-10-17T01:31:52.719+0000 7fd330ed7700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:28.169269+0000 front 2023-10-17T01:31:28.169297+0000 (oldest deadline 2023-10-17T01:31:51.669430+0000) 2023-10-17T01:31:53.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:52 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:52.797+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:53.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:53.355+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:54.028 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:53 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:53.776+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:54.647 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:54.379+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:54.801 INFO:journalctl@ceph.osd.3.smithi045.stdout:Oct 17 01:31:54 smithi045 bash[175910]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-3 2023-10-17T01:31:54.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:31:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:31:54.645+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:31:55.078 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:31:54 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:31:54.768+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:31:55.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:54 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:54.801+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:55.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:31:54 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:31:54.981+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:31:55.604 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:55.351+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:55.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:31:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:31:55.603+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:31:56.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:55 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:55.761+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:56.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:31:55 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:31:55.939+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:31:56.078 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:31:55 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:31:55.749+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:31:56.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.3.service' 2023-10-17T01:31:56.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:31:56.517 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-17T01:31:56.517 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-17T01:31:56.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.4 2023-10-17T01:31:56.618 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:56 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:56.335+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:56.828 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:31:56 smithi060 systemd[1]: Stopping Ceph osd.4 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:31:56.828 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:31:56 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:31:56.750+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:31:56.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:31:56 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:31:56.617+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:31:57.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:31:56 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:31:56.933+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:31:57.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:56 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:56.801+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:57.328 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:31:56 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:31:56.874+0000 7f458fce7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:31:57.328 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:31:56 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:31:56.874+0000 7f458fce7700 -1 osd.4 88 *** Got signal Terminated *** 2023-10-17T01:31:57.329 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:31:56 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:31:56.874+0000 7f458fce7700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:31:57.611 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:57 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:57.357+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:57.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:31:57 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:31:57.610+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:31:58.078 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:31:57 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:31:57.776+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:31:58.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:31:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:31:57.928+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:31:58.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:57 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:57.845+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:58.656 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:58 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:58.331+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:31:58.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:31:58 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:31:58.655+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:31:59.078 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:31:58 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:31:58.785+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:31:59.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:31:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:31:58.944+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:31:59.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:58.879+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:31:59.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:31:59 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:31:59.289+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:00.078 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:00 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:00.017+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:00.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:31:59 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:31:59.809+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:00.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:31:59 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:31:59.663+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:00.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:31:59 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:31:59.896+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:00.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:31:59 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:31:59.873+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:00.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:00 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:00.246+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:00.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:00 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:00.270+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:00.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:00 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:00.270+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:00.813 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:32:00 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:32:00.337+0000 7f45892d7700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.527635+0000 front 2023-10-17T01:31:36.527604+0000 (oldest deadline 2023-10-17T01:32:00.027794+0000) 2023-10-17T01:32:01.078 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:00 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:00.982+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:01.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:00 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:00.812+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:01.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:00 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:00.699+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:01.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:00 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:00.900+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:01.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:00 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:00.854+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:01.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:00 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:00.854+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:01.578 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:32:01 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4[147959]: 2023-10-17T01:32:01.323+0000 7f45892d7700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.527635+0000 front 2023-10-17T01:31:36.527604+0000 (oldest deadline 2023-10-17T01:32:00.027794+0000) 2023-10-17T01:32:01.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:01 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:01.230+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:01.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:01 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:01.298+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:01.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:01 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:01.298+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:02.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:01 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:01.703+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:02.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:01.942+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:02.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:01.942+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:02.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:01.824+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:02.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:01.824+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:02.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:01 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:01.937+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:02.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:01 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:01.832+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:02.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:02 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:02.235+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:02.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:02 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:02.267+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:02.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:02 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:02.267+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:02.906 INFO:journalctl@ceph.osd.4.smithi060.stdout:Oct 17 01:32:02 smithi060 bash[177729]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-4 2023-10-17T01:32:02.907 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:02 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:02.904+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:02.907 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:02 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:02.843+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:02.907 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:02 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:02.843+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:03.158 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:02 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:02.668+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:03.158 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:02 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:02.668+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:03.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:02.802+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:03.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:02.802+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:03.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:02.915+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:03.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:02 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:02.915+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:03.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:03.197+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:03.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:03.285+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:03.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:03.285+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:04.078 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:03 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:03.884+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:04.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:03 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:03.810+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:04.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:03 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:03.810+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:04.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.4.service' 2023-10-17T01:32:04.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:03.699+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:04.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:03 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:03.699+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:04.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:04.185 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-17T01:32:04.185 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-17T01:32:04.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.5 2023-10-17T01:32:04.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:03 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:03.877+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:04.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:03 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:03.877+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:04.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:03 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:03.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:04.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:03 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:03.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:04.535 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:04.157+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:04.536 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:04 smithi181 systemd[1]: Stopping Ceph osd.5 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:04.536 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:04.317+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:04.537 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:04.317+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:04.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:04.534+0000 7f1caa445700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:04.907 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:04.534+0000 7f1caa445700 -1 osd.5 88 *** Got signal Terminated *** 2023-10-17T01:32:04.908 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:04.534+0000 7f1caa445700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:04.908 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:04.747+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:04.908 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:04 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:04.747+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:05.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:04.825+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:05.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:04.825+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:05.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:04.882+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:05.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:04 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:04.882+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:05.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:04 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:04.875+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:05.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:04 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:04.850+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:05.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:04 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:04.850+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:05.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:05 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:05.188+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:05.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:05 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:05.337+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:05.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:05 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:05.337+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:06.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:05.798+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:06.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:05.798+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:06.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:05.895+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:06.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:05.895+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:06.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:05 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:05.754+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:06.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:05 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:05.754+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:06.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:05 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:05.914+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:06.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:05 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:05.879+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:06.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:05 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:05.879+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:06.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:06 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:06.203+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:06.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:06 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:06.335+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:06.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:06 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:06.335+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:07.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:06.903+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:07.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:06.903+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:07.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:06.751+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:07.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:06.751+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:07.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:06 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:06.751+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:07.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:06 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:06.790+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:07.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:06 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:06.790+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:07.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:06 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:06.905+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:07.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:06 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:06.905+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:07.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:06 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:06.961+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:07.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:06 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:06.961+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:07.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:07 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:07.191+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:07.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:07 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:07.191+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:07.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:07 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:07.385+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:07.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:07 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:07.385+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:08.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:07 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:07.911+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:08.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:07 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:07.911+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:08.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:07 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:07.784+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:08.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:07 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:07.784+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:08.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:07 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:07.784+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:08.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:07 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:07.750+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:08.157 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:07 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:07.750+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:08.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:07 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:07.867+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:08.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:07 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:07.867+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:08.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:07 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:07.913+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:08.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:07 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:07.913+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:08.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:08 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:08.177+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:08.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:08 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:08.177+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:08.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:08 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:08.373+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:08.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:08 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:08.373+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:09.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:08 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:08.894+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:09.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:08 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:08.894+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:09.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:08 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:08.756+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:09.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:08 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:08.756+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:09.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:08 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:08.756+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:09.156 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:08 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:08.771+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.301573+0000 front 2023-10-17T01:31:25.301641+0000 (oldest deadline 2023-10-17T01:31:54.102253+0000) 2023-10-17T01:32:09.156 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:08 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5[148199]: 2023-10-17T01:32:08.771+0000 7f1ca6a3b700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.403327+0000 front 2023-10-17T01:31:36.403352+0000 (oldest deadline 2023-10-17T01:32:02.303399+0000) 2023-10-17T01:32:09.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:08 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:08.907+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:09.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:08 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:08.907+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:09.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:08 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:08.893+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:09.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:08 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:08.893+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:09.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:08 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:08.893+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:09.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:09 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:09.154+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:09.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:09 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:09.154+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:09.408 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:09 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:09.384+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:09.408 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:09 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:09.384+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:10.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:09.760+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:10.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:09.760+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:10.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:09.760+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:10.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:09.900+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:10.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:09.900+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:10.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:09.900+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:10.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:09 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:09.933+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:10.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:09 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:09.933+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:10.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:09 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:09.881+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:10.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:09 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:09.881+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:10.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:09 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:09.881+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:10.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:10 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:10.196+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:10.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:10 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:10.196+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:10.658 INFO:journalctl@ceph.osd.5.smithi181.stdout:Oct 17 01:32:10 smithi181 bash[178292]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-5 2023-10-17T01:32:10.659 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:10 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:10.348+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:10.659 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:10 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:10.348+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:10.659 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:10 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:10.348+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:11.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:10.721+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:11.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:10.721+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:11.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:10.721+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:11.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:10.930+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:11.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:10.930+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:11.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:10 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:10.930+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:11.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:10 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:10.954+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:11.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:10 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:10.954+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:11.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:10 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:10.879+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:11.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:10 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:10.879+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:11.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:10 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:10.879+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:11.579 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:11.242+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:11.580 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:11.242+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:11.580 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:11.388+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:11.580 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:11.388+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:11.580 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:11 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:11.388+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:11.586 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.5.service' 2023-10-17T01:32:11.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:11.627 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-17T01:32:11.627 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-17T01:32:11.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.6 2023-10-17T01:32:11.971 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:11 smithi045 systemd[1]: Stopping Ceph osd.6 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:11.972 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:11 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:11.691+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:11.972 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:11 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:11.691+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:11.972 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:11 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:11.691+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:12.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:11 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:11.969+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:12.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:11 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:11.969+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:12.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:11 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:11.969+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:12.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:12.150+0000 7febf7142700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:12.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:12.150+0000 7febf7142700 -1 osd.6 88 *** Got signal Terminated *** 2023-10-17T01:32:12.303 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:12.150+0000 7febf7142700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:12.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:11.968+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:12.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:11.968+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:12.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:11.928+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:12.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:11.928+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:12.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:11 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:11.928+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:12.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:12.286+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:12.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:12.286+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:12.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:12.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:12.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:12.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:12.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:12 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:12.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:12.922 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:12.670+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:12.923 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:12.670+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:12.923 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:12.670+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:13.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:12.921+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:13.301 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:12.921+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:13.302 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:12 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:12.921+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:13.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:12.993+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:13.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:12.993+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:13.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:12.961+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:13.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:12.961+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:13.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:12 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:12.961+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:13.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:13 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:13.295+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:13.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:13 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:13.295+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:13.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:13 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:13.460+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:13.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:13 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:13.460+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:13.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:13 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:13.460+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:14.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:13 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:13.889+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:14.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:13 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:13.889+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:14.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:13 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:13.889+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:14.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:13 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:13.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:14.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:13 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:13.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:14.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:13 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:13.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:14.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:13.985+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:14.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:13.985+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:14.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:13.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:14.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:13.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:14.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:13.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:14.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:14 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:14.257+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:14.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:14 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:14.257+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:14.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:14 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:14.426+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:14.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:14 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:14.426+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:14.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:14 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:14.426+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:15.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:14.912+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:15.051 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:14.912+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:15.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:14.912+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:15.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:14.912+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:49.839595+0000 front 2023-10-17T01:31:49.839575+0000 (oldest deadline 2023-10-17T01:32:13.939607+0000) 2023-10-17T01:32:15.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:14.681+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:15.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:14.681+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:15.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:14 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:14.681+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:15.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:14.986+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:15.330 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:14.986+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:15.330 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:14.986+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:15.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:14.892+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:15.331 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:14.892+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:15.331 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:14.892+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:15.331 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:14 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:14.892+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:15.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:15 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:15.299+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:15.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:15 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:15.299+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:15.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:15 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:15.442+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:15.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:15 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:15.443+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:15.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:15 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:15.443+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:16.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:15.672+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:16.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:15.672+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:16.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:15.672+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:16.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:15.865+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:16.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:15.865+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:16.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:15.865+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:16.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:15 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:15.865+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:49.839595+0000 front 2023-10-17T01:31:49.839575+0000 (oldest deadline 2023-10-17T01:32:13.939607+0000) 2023-10-17T01:32:16.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:16.020+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:16.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:16.020+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:16.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:16.020+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:16.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:15 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:15.940+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:16.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:15 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:15.940+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:16.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:15 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:15.940+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:16.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:15 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:15.940+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:16.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:16 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:16.251+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:16.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:16 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:16.251+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:16.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:16 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:16.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:16.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:16 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:16.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:16.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:16 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:16.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:17.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:16.696+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:17.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:16.696+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:17.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:16.696+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:17.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:16.892+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.833497+0000 front 2023-10-17T01:31:27.833545+0000 (oldest deadline 2023-10-17T01:31:54.234356+0000) 2023-10-17T01:32:17.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:16.893+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.537549+0000 front 2023-10-17T01:31:37.537697+0000 (oldest deadline 2023-10-17T01:32:01.637545+0000) 2023-10-17T01:32:17.052 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:16.893+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.539292+0000 front 2023-10-17T01:31:44.539113+0000 (oldest deadline 2023-10-17T01:32:09.839016+0000) 2023-10-17T01:32:17.053 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:16 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6[132709]: 2023-10-17T01:32:16.893+0000 7febf0732700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:49.839595+0000 front 2023-10-17T01:31:49.839575+0000 (oldest deadline 2023-10-17T01:32:13.939607+0000) 2023-10-17T01:32:17.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:17.058+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:17.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:17.058+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:17.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:17.058+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:17.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:16.896+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:17.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:16.896+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:17.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:16.896+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:17.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:16 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:16.896+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:17.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:17 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:17.270+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:17.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:17 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:17.270+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:17.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:17 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:17.270+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:17.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:17 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:17.494+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:17.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:17 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:17.494+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:17.659 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:17 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:17.494+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:17.948 INFO:journalctl@ceph.osd.6.smithi045.stdout:Oct 17 01:32:17 smithi045 bash[176182]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-6 2023-10-17T01:32:17.948 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:17 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:17.671+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:17.949 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:17 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:17.671+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:17.949 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:17 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:17.671+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:18.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:18.029+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:18.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:18.029+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:18.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:18.029+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:18.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:17.931+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:18.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:17.931+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:18.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:17.931+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:18.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:17.931+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:18.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:18 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:18.262+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:18.657 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:18 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:18.262+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:18.658 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:18 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:18.262+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:18.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:18 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:18.479+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:18.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:18 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:18.479+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:18.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:18 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:18.479+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:19.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:18 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:18.652+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:19.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:18 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:18.652+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:19.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:18 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:18.652+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:19.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:18 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:18.652+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:19.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:18.998+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:19.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:18.998+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:19.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:18.998+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:19.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:18.923+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:19.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:18.923+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:19.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:18.923+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:19.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:18 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:18.923+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:19.511 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:19.212+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:19.512 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:19.212+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:19.512 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:19.212+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:19.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:19.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:19.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:19.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:19.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:19 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:19.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:19.973 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:19 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:19.660+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:19.973 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:19 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:19.660+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:19.974 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:19 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:19.660+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:19.974 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:19 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:19.660+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:19.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.6.service' 2023-10-17T01:32:20.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:20.088 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-17T01:32:20.088 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-17T01:32:20.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.7 2023-10-17T01:32:20.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:19.992+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:20.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:19.992+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:20.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:19.992+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:20.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 systemd[1]: Stopping Ceph osd.7 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:20.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:19.903+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:20.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:19.903+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:20.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:19.903+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:20.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:19 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:19.903+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:20.484 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:20 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:20.188+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:20.485 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:20 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:20.188+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:20.485 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:20 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:20.188+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:20.828 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:20.436+0000 7f22e4e24700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:20.828 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:20.436+0000 7f22e4e24700 -1 osd.7 88 *** Got signal Terminated *** 2023-10-17T01:32:20.829 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:20.436+0000 7f22e4e24700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:20.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:20 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:20.483+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:20.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:20 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:20.483+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:20.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:20 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:20.483+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:21.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:20 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:20.697+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:21.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:20 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:20.697+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:21.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:20 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:20.697+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:21.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:20 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:20.697+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:21.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:20.949+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:21.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:20.949+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:21.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:20.949+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:21.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:20.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:21.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:20.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:21.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:20.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:21.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:20 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:20.913+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:21.505 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:21 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:21.191+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:21.506 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:21 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:21.191+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:21.506 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:21 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:21.191+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:21.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:21 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:21.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:21.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:21 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:21.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:21.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:21 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:21.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:22.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:21 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:21.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:22.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:21 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:21.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:22.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:21 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:21.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:22.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:21 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:21.700+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:22.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:21.927+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:22.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:21.927+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:22.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:21.927+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:22.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:21.872+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:22.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:21.872+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:22.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:21.872+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:22.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:21.872+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:22.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:22 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:22.153+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:22.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:22 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:22.153+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:22.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:22 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:22.153+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:22.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:22 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:22.481+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:22.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:22 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:22.481+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:22.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:22 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:22.481+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:23.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:22 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:22.655+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:23.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:22 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:22.655+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:23.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:22 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:22.655+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:23.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:22 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:22.655+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:23.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:22.948+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:23.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:22.948+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:23.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:22.948+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:23.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:22.948+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:00.267951+0000 front 2023-10-17T01:32:00.268042+0000 (oldest deadline 2023-10-17T01:32:22.567998+0000) 2023-10-17T01:32:23.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:22.902+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:23.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:22.902+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:23.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:22.902+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:23.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:22 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:22.902+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:23.478 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:23 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:23.169+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:23.478 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:23 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:23.169+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:23.478 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:23 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:23.169+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:23.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:23 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:23.477+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:23.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:23 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:23.477+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:23.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:23 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:23.477+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:24.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:23.704+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:24.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:23.704+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:24.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:23.704+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:24.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:23.704+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:24.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:23 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:23.704+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:24.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:23.909+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:24.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:23.909+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:24.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:23.909+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:24.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:23.909+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:00.267951+0000 front 2023-10-17T01:32:00.268042+0000 (oldest deadline 2023-10-17T01:32:22.567998+0000) 2023-10-17T01:32:24.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:23.908+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:24.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:23.908+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:24.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:23.908+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:24.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:23.908+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:24.331 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:23 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:23.908+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:24.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:24 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:24.133+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:24.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:24 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:24.133+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:24.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:24 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:24.133+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:24.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:24 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:24.503+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:24.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:24 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:24.503+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:24.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:24 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:24.503+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:25.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:24.741+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:25.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:24.741+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:25.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:24.741+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:25.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:24.741+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:25.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:24 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:24.741+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:25.328 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:24.883+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.564129+0000 front 2023-10-17T01:31:34.564173+0000 (oldest deadline 2023-10-17T01:31:59.264302+0000) 2023-10-17T01:32:25.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:24.883+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:45.165658+0000 front 2023-10-17T01:31:45.165481+0000 (oldest deadline 2023-10-17T01:32:06.265614+0000) 2023-10-17T01:32:25.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:24.883+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.166804+0000 front 2023-10-17T01:31:52.166668+0000 (oldest deadline 2023-10-17T01:32:14.466546+0000) 2023-10-17T01:32:25.329 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7[155886]: 2023-10-17T01:32:24.883+0000 7f22e0c19700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:00.267951+0000 front 2023-10-17T01:32:00.268042+0000 (oldest deadline 2023-10-17T01:32:22.567998+0000) 2023-10-17T01:32:25.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:24.945+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:25.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:24.945+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:25.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:24.945+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:25.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:24.945+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:25.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:24 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:24.945+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:25.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:25.129+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:25.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:25.129+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:25.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:25.129+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:25.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:25.129+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:25.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:25.459+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:25.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:25.459+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:25.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:25.459+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:25.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:25 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:25.459+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:26.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:25.768+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:26.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:25.768+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:26.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:25.768+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:26.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:25.768+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:26.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:25 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:25.768+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:26.253 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:25.941+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:26.253 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:25.941+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:26.253 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:25.941+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:26.254 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:25.941+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:26.254 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:25.941+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:26.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:26.155+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:26.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:26.155+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:26.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:26.155+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:26.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:26.155+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:26.578 INFO:journalctl@ceph.osd.7.smithi060.stdout:Oct 17 01:32:26 smithi060 bash[178015]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-7 2023-10-17T01:32:26.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:26.472+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:26.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:26.472+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:26.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:26.472+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:26.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:26 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:26.472+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:27.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:26.782+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:27.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:26.782+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:27.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:26.782+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:27.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:26.782+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:27.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:26 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:26.782+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:27.253 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:26.929+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:27.254 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:26.929+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:27.254 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:26.929+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:27.254 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:26.929+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:27.254 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:26 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:26.929+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:27.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:27.142+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:27.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:27.142+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:27.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:27.142+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:27.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:27.142+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:27.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.7.service' 2023-10-17T01:32:27.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:27.762 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-17T01:32:27.762 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-17T01:32:27.763 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.8 2023-10-17T01:32:27.795 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:27.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:27.795 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:27.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:27.795 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:27.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:27.796 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:27 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:27.511+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:28.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:27.780+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:28.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:27.780+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:28.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:27.780+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:28.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:27.780+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:28.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:27 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:27.780+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:28.078 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:27 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:27.935+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:28.078 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:27 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:27.935+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:28.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:27 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:27.935+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:28.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:27 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:27.935+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:28.079 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:27 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:27.935+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:28.116 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:27 smithi181 systemd[1]: Stopping Ceph osd.8 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:28.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.115+0000 7fea6b0eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:28.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.115+0000 7fea6b0eb700 -1 osd.8 88 *** Got signal Terminated *** 2023-10-17T01:32:28.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.115+0000 7fea6b0eb700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:28.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.149+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:28.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.149+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:28.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.149+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:28.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:28.149+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:28.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:28.504+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:28.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:28.504+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:28.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:28.504+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:28.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:28 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:28.504+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:29.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:28.800+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:29.054 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:28.800+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:29.054 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:28.800+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:29.054 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:28.800+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:29.055 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:28 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:28.800+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:29.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:28 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:28.891+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:29.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:28 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:28.891+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:29.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:28 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:28.891+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:29.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:28 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:28.891+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:29.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:28 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:28.891+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:29.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:29.139+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:29.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:29.139+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:29.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:29.139+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:29.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:29.139+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:29.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:29.506+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:29.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:29.506+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:29.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:29.506+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:29.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:29.506+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:30.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:29.761+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:30.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:29.761+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:30.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:29.761+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:30.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:29.761+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:30.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:29 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:29.761+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:30.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:29 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:29.912+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:30.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:29 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:29.912+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:30.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:29 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:29.912+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:30.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:29 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:29.912+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:30.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:29 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:29.912+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:30.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:30.156+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:30.407 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:30.156+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:30.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:30.156+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:30.408 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:30.156+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:30.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:30.498+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:30.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:30.498+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:30.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:30.498+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:30.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:30 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:30.498+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:31.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:30.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:31.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:30.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:31.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:30.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:31.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:30.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:31.303 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:30 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:30.806+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:31.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:30 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:30.918+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:31.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:30 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:30.918+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:31.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:30 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:30.918+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:31.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:30 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:30.918+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:31.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:30 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:30.918+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:31.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:30 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:30.918+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:31.526 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:31.183+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:31.527 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:31.183+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:31.527 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:31.183+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:31.527 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:31.183+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:31.527 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:31.183+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.149573+0000 front 2023-10-17T01:32:09.149629+0000 (oldest deadline 2023-10-17T01:32:30.249782+0000) 2023-10-17T01:32:31.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:31.525+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:31.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:31.525+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:31.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:31.525+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:31.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:31 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:31.525+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:32.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:31.770+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:32.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:31.770+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:32.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:31.770+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:32.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:31.770+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:32.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:31.770+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:32.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:31 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:31.770+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:32.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:31 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:31.919+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:32.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:31 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:31.919+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:32.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:31 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:31.919+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:32.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:31 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:31.919+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:32.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:31 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:31.919+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:32.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:31 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:31.919+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:32.511 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:32.166+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:34.243165+0000 front 2023-10-17T01:31:34.243091+0000 (oldest deadline 2023-10-17T01:32:00.143067+0000) 2023-10-17T01:32:32.511 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:32.166+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:40.143473+0000 front 2023-10-17T01:31:40.143607+0000 (oldest deadline 2023-10-17T01:32:07.144341+0000) 2023-10-17T01:32:32.512 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:32.166+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.946086+0000 front 2023-10-17T01:31:52.946195+0000 (oldest deadline 2023-10-17T01:32:16.446196+0000) 2023-10-17T01:32:32.512 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:32.166+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.548622+0000 front 2023-10-17T01:32:01.548639+0000 (oldest deadline 2023-10-17T01:32:24.448612+0000) 2023-10-17T01:32:32.512 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8[156764]: 2023-10-17T01:32:32.166+0000 7fea63eda700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.149573+0000 front 2023-10-17T01:32:09.149629+0000 (oldest deadline 2023-10-17T01:32:30.249782+0000) 2023-10-17T01:32:32.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:32.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:32.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:32.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:32.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:32.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:32.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:32 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:32.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:33.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:32.734+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:33.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:32.734+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:33.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:32.734+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:33.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:32.734+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:33.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:32.734+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:33.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:32 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:32.734+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:33.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:32.943+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:33.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:32.943+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:33.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:32.943+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:33.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:32.943+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:33.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:32.943+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:33.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:32 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:32.943+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:33.741 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:33 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:33.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:33.741 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:33 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:33.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:33.742 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:33 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:33.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:33.742 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:33 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:33.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:34.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:33 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:33.771+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:34.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:33 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:33.771+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:34.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:33 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:33.771+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:34.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:33 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:33.771+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:34.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:33 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:33.771+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:34.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:33 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:33.771+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:34.157 INFO:journalctl@ceph.osd.8.smithi181.stdout:Oct 17 01:32:33 smithi181 bash[178561]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-8 2023-10-17T01:32:34.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:33.953+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:34.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:33.953+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:34.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:33.953+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:34.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:33.953+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:34.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:33.953+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:34.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:33 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:33.953+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:34.772 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:34 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:34.496+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:34.772 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:34 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:34.496+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:34.773 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:34 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:34.496+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:34.773 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:34 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:34.496+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:35.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.8.service' 2023-10-17T01:32:35.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:35.156 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-17T01:32:35.157 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-17T01:32:35.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.9 2023-10-17T01:32:35.225 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:34 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:34.813+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:35.226 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:34 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:34.813+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:35.226 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:34 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:34.813+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:35.226 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:34 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:34.813+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:35.226 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:34 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:34.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:35.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:34 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:34.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:35.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:34 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:34.957+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:35.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:34 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:34.957+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:35.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:34 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:34.957+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:35.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:34 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:34.957+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:35.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:34 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:34.957+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:35.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:34 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:34.957+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:35.551 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 systemd[1]: Stopping Ceph osd.9 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:35.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:35 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:35.453+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:35.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:35 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:35.453+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:35.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:35 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:35.453+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:35.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:35 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:35.453+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:35.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:35 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:35.453+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:36.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.680+0000 7f5b29494700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:36.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.680+0000 7f5b29494700 -1 osd.9 88 *** Got signal Terminated *** 2023-10-17T01:32:36.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.680+0000 7f5b29494700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:36.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.842+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:36.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.842+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:36.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.842+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:36.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.842+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:36.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.842+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:36.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:35 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:35.842+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:36.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:35 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:35.991+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:36.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:35 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:35.991+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:36.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:35 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:35.991+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:36.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:35 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:35.991+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:36.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:35 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:35.991+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:36.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:35 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:35.991+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:36.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:36.497+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:36.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:36.497+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:36.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:36.497+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:36.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:36.497+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:36.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:36 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:36.497+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:37.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:36.794+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:37.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:36.794+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:37.051 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:36.794+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:37.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:36.794+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:37.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:36.794+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:37.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:36 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:36.794+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:37.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:36 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:36.975+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:37.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:36 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:36.975+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:37.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:36 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:36.975+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:37.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:36 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:36.975+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:37.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:36 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:36.975+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:37.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:36 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:36.975+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:37.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:37.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:37.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:37.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:37.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:37.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:37.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:37.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:37.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:37.480+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:38.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:37 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:37.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:38.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:37 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:37.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:38.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:37 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:37.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:38.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:37 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:37.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:38.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:37 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:37.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:38.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:37 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:37.814+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:38.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:38.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:38.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:38.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:38.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:38.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:38.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:37 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:37.990+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:38.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:38.487+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:38.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:38.487+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:38.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:38.487+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:38.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:38.487+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:38.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:38 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:38.487+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:39.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:38 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:38.856+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:39.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:38 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:38.856+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:39.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:38 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:38.856+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:39.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:38 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:38.856+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:39.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:38 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:38.856+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:39.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:38 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:38.856+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:39.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:39.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:39.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:39.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:39.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:39.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:39.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:38 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:38.994+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:39.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:39.456+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:39.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:39.456+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:39.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:39.456+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:39.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:39.456+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:39.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:39 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:39.457+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:40.301 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:39 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:39.810+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:25.787842+0000 front 2023-10-17T01:31:25.787894+0000 (oldest deadline 2023-10-17T01:31:52.788927+0000) 2023-10-17T01:32:40.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:39 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:39.810+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:37.291918+0000 front 2023-10-17T01:31:37.291937+0000 (oldest deadline 2023-10-17T01:32:00.191962+0000) 2023-10-17T01:32:40.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:39 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:39.810+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:42.493226+0000 front 2023-10-17T01:31:42.493343+0000 (oldest deadline 2023-10-17T01:32:06.593224+0000) 2023-10-17T01:32:40.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:39 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:39.810+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:52.494536+0000 front 2023-10-17T01:31:52.494559+0000 (oldest deadline 2023-10-17T01:32:17.794933+0000) 2023-10-17T01:32:40.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:39 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:39.810+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:32:01.296170+0000 front 2023-10-17T01:32:01.296208+0000 (oldest deadline 2023-10-17T01:32:22.996115+0000) 2023-10-17T01:32:40.302 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:39 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9[140629]: 2023-10-17T01:32:39.810+0000 7f5b22a84700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:05.897552+0000 front 2023-10-17T01:32:05.897524+0000 (oldest deadline 2023-10-17T01:32:31.197301+0000) 2023-10-17T01:32:40.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:40.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:40.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:40.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:40.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:40.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:40.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:40 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:40.012+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:40.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:40.445+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:40.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:40.445+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:40.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:40.445+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:40.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:40.445+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:40.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:40 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:40.445+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:41.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:41.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:41.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:41.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:41.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:41.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:41.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:41 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:41.035+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:41.801 INFO:journalctl@ceph.osd.9.smithi045.stdout:Oct 17 01:32:41 smithi045 bash[176454]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-9 2023-10-17T01:32:41.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:41.408+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:41.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:41.408+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:41.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:41.408+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:41.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:41.408+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:41.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:41.408+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:41.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:41.408+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:42.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:42.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:42.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:42.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:42.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:42.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:42.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:42 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:42.075+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:42.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:42.433+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:42.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:42.433+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:42.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:42.433+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:42.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:42.433+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:42.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:42.433+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:42.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:42 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:42.433+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:43.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.9.service' 2023-10-17T01:32:43.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:43.432 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-17T01:32:43.432 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-17T01:32:43.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.10 2023-10-17T01:32:43.464 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:43.464 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:43.464 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:43.464 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:43.465 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:43.465 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:43.465 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.080+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:43.786 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 systemd[1]: Stopping Ceph osd.10 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:43.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:43.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:43.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:43.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:43.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:43.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:43.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:43.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:43.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:43.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:43.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:43 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:43.422+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:44.054 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.784+0000 7f1862e37700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:44.055 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.784+0000 7f1862e37700 -1 osd.10 88 *** Got signal Terminated *** 2023-10-17T01:32:44.055 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:43 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:43.784+0000 7f1862e37700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:44.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:44.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:44.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:44.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:44.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:44.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:44.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:44 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:44.054+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:44.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:44 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:44.431+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:44.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:44 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:44.431+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:44.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:44 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:44.431+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:44.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:44 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:44.431+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:44.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:44 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:44.431+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:44.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:44 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:44.431+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:45.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:45.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:45.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:45.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:45.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:45.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:45.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:45 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:45.031+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:45.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:45.398+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:45.657 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:45.398+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:45.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:45.398+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:45.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:45.398+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:45.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:45.398+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:45.658 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:45 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:45.398+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:46.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:46.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:46.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:46.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:46.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:46.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:46.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:46 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:46.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:46.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:46.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:46.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:46.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:46.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:46.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:46.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:46 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:46.430+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:47.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:47.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:47.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:47.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:47.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:47.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:47.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:47 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:47.067+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:47.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:47.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:47.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:47.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:47.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:47.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:47.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:47 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:47.450+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:48.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:24.722574+0000 front 2023-10-17T01:31:24.722471+0000 (oldest deadline 2023-10-17T01:31:54.723345+0000) 2023-10-17T01:32:48.328 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:35.224612+0000 front 2023-10-17T01:31:35.224644+0000 (oldest deadline 2023-10-17T01:32:02.225616+0000) 2023-10-17T01:32:48.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.526648+0000 front 2023-10-17T01:31:44.526686+0000 (oldest deadline 2023-10-17T01:32:08.626848+0000) 2023-10-17T01:32:48.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2023-10-17T01:31:48.627347+0000 front 2023-10-17T01:31:48.627333+0000 (oldest deadline 2023-10-17T01:32:14.428185+0000) 2023-10-17T01:32:48.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:56.728928+0000 front 2023-10-17T01:31:56.728902+0000 (oldest deadline 2023-10-17T01:32:23.129673+0000) 2023-10-17T01:32:48.329 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:04.831077+0000 front 2023-10-17T01:32:04.831014+0000 (oldest deadline 2023-10-17T01:32:30.731115+0000) 2023-10-17T01:32:48.330 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:48 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10[163905]: 2023-10-17T01:32:48.050+0000 7f185bc26700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.232517+0000 front 2023-10-17T01:32:14.232458+0000 (oldest deadline 2023-10-17T01:32:37.732191+0000) 2023-10-17T01:32:48.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:48.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:48.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:48.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:48.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:48.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:48.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:48 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:48.488+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:49.828 INFO:journalctl@ceph.osd.10.smithi060.stdout:Oct 17 01:32:49 smithi060 bash[178276]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-10 2023-10-17T01:32:49.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:49.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:49.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:49.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:49.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:49.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:49.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:49 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:49.520+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:50.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:50.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:50.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:50.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:50.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:50.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:50.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:50 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:50.507+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:50.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.10.service' 2023-10-17T01:32:51.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:51.056 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-17T01:32:51.056 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-17T01:32:51.056 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.11 2023-10-17T01:32:51.407 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 systemd[1]: Stopping Ceph osd.11 for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:51.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.410+0000 7f8a23abb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:51.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.410+0000 7f8a23abb700 -1 osd.11 88 *** Got signal Terminated *** 2023-10-17T01:32:51.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.410+0000 7f8a23abb700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-17T01:32:51.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:51.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:51.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:51.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:51.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:51.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:51.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:51 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:51.509+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:52.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:52.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:52.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:52.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:52.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:52.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:52.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:52 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:52.505+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:53.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:53.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:53.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:53.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:53.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:53.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:53.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:53 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:53.524+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:54.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:54.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:54.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:54.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:54.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:54.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:54.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:54 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:54.485+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:55.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2023-10-17T01:31:27.024075+0000 front 2023-10-17T01:31:27.023922+0000 (oldest deadline 2023-10-17T01:31:50.524111+0000) 2023-10-17T01:32:55.907 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6804 osd.1 since back 2023-10-17T01:31:36.826931+0000 front 2023-10-17T01:31:36.826890+0000 (oldest deadline 2023-10-17T01:31:59.726816+0000) 2023-10-17T01:32:55.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-10-17T01:31:44.229045+0000 front 2023-10-17T01:31:44.229027+0000 (oldest deadline 2023-10-17T01:32:10.029771+0000) 2023-10-17T01:32:55.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6812 osd.4 since back 2023-10-17T01:31:58.731813+0000 front 2023-10-17T01:31:58.731977+0000 (oldest deadline 2023-10-17T01:32:24.631811+0000) 2023-10-17T01:32:55.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-10-17T01:32:09.233490+0000 front 2023-10-17T01:32:09.233510+0000 (oldest deadline 2023-10-17T01:32:34.533614+0000) 2023-10-17T01:32:55.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2023-10-17T01:32:14.534290+0000 front 2023-10-17T01:32:14.534244+0000 (oldest deadline 2023-10-17T01:32:40.934678+0000) 2023-10-17T01:32:55.908 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.60:6820 osd.7 since back 2023-10-17T01:32:23.235782+0000 front 2023-10-17T01:32:23.235818+0000 (oldest deadline 2023-10-17T01:32:46.135634+0000) 2023-10-17T01:32:55.909 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:55 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11[165502]: 2023-10-17T01:32:55.532+0000 7f8a1c0a9700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-10-17T01:32:29.537071+0000 front 2023-10-17T01:32:29.537140+0000 (oldest deadline 2023-10-17T01:32:54.837043+0000) 2023-10-17T01:32:57.407 INFO:journalctl@ceph.osd.11.smithi181.stdout:Oct 17 01:32:56 smithi181 bash[178828]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-osd-11 2023-10-17T01:32:58.374 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@osd.11.service' 2023-10-17T01:32:58.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:32:58.414 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-17T01:32:58.415 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-17T01:32:58.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.a 2023-10-17T01:32:58.801 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:32:58 smithi045 systemd[1]: Stopping Ceph mds.a for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:32:59.301 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:32:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a[149125]: 2023-10-17T01:32:58.943+0000 7fbd0c8a0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:32:59.302 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:32:58 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a[149125]: 2023-10-17T01:32:58.943+0000 7fbd0c8a0700 -1 mds.a *** got signal Terminated *** 2023-10-17T01:33:01.051 INFO:journalctl@ceph.mds.a.smithi045.stdout:Oct 17 01:33:00 smithi045 bash[176726]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-a 2023-10-17T01:33:01.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.a.service' 2023-10-17T01:33:01.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:01.285 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-17T01:33:01.286 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-17T01:33:01.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.d 2023-10-17T01:33:01.551 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:33:01 smithi045 systemd[1]: Stopping Ceph mds.d for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:02.301 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:33:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-d[148119]: 2023-10-17T01:33:01.800+0000 7fb1f1b44700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:02.301 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:33:01 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-d[148119]: 2023-10-17T01:33:01.800+0000 7fb1f1b44700 -1 mds.d *** got signal Terminated *** 2023-10-17T01:33:04.551 INFO:journalctl@ceph.mds.d.smithi045.stdout:Oct 17 01:33:04 smithi045 bash[176872]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-d 2023-10-17T01:33:04.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.d.service' 2023-10-17T01:33:04.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:04.766 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-17T01:33:04.766 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-17T01:33:04.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.g 2023-10-17T01:33:05.051 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:33:04 smithi045 systemd[1]: Stopping Ceph mds.g for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:05.551 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:33:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g[148731]: 2023-10-17T01:33:05.275+0000 7f295f61e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:05.552 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:33:05 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g[148731]: 2023-10-17T01:33:05.275+0000 7f295f61e700 -1 mds.g *** got signal Terminated *** 2023-10-17T01:33:09.240 INFO:journalctl@ceph.mds.g.smithi045.stdout:Oct 17 01:33:08 smithi045 bash[177013]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-g 2023-10-17T01:33:09.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.g.service' 2023-10-17T01:33:09.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:09.356 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-17T01:33:09.356 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-17T01:33:09.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.j 2023-10-17T01:33:09.551 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:33:09 smithi045 systemd[1]: Stopping Ceph mds.j for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:10.301 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:33:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-j[147730]: 2023-10-17T01:33:09.874+0000 7ff99e077700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:10.301 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:33:09 smithi045 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-j[147730]: 2023-10-17T01:33:09.874+0000 7ff99e077700 -1 mds.j *** got signal Terminated *** 2023-10-17T01:33:12.828 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:33:12 smithi045 bash[177160]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-j 2023-10-17T01:33:13.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.j.service' 2023-10-17T01:33:13.100 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:33:13 smithi045 systemd[1]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.j.service: Succeeded. 2023-10-17T01:33:13.101 INFO:journalctl@ceph.mds.j.smithi045.stdout:Oct 17 01:33:13 smithi045 systemd[1]: Stopped Ceph mds.j for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:33:13.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:13.124 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-17T01:33:13.125 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-17T01:33:13.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.b 2023-10-17T01:33:13.491 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:33:13 smithi060 systemd[1]: Stopping Ceph mds.b for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:13.828 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:33:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: 2023-10-17T01:33:13.489+0000 7f2b6cc88700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:13.828 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:33:13 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b[171012]: 2023-10-17T01:33:13.489+0000 7f2b6cc88700 -1 mds.b *** got signal Terminated *** 2023-10-17T01:33:17.048 INFO:journalctl@ceph.mds.b.smithi060.stdout:Oct 17 01:33:16 smithi060 bash[178553]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-b 2023-10-17T01:33:17.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.b.service' 2023-10-17T01:33:17.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:17.093 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-17T01:33:17.093 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-17T01:33:17.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.e 2023-10-17T01:33:17.328 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:33:17 smithi060 systemd[1]: Stopping Ceph mds.e for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:17.828 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:33:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e[170332]: 2023-10-17T01:33:17.471+0000 7f8843a25700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:17.829 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:33:17 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e[170332]: 2023-10-17T01:33:17.471+0000 7f8843a25700 -1 mds.e *** got signal Terminated *** 2023-10-17T01:33:20.902 INFO:journalctl@ceph.mds.e.smithi060.stdout:Oct 17 01:33:20 smithi060 bash[178700]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-e 2023-10-17T01:33:20.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.e.service' 2023-10-17T01:33:20.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:20.944 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-17T01:33:20.944 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-17T01:33:20.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.h 2023-10-17T01:33:21.322 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:33:21 smithi060 systemd[1]: Stopping Ceph mds.h for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:21.578 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:33:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h[170672]: 2023-10-17T01:33:21.320+0000 7fb5894b6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:21.578 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:33:21 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h[170672]: 2023-10-17T01:33:21.321+0000 7fb5894b6700 -1 mds.h *** got signal Terminated *** 2023-10-17T01:33:25.096 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:33:24 smithi060 bash[178844]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-h 2023-10-17T01:33:25.096 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:33:25 smithi060 systemd[1]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.h.service: Succeeded. 2023-10-17T01:33:25.096 INFO:journalctl@ceph.mds.h.smithi060.stdout:Oct 17 01:33:25 smithi060 systemd[1]: Stopped Ceph mds.h for 2804d2ac-6c89-11ee-8db6-212e2dc638e7. 2023-10-17T01:33:25.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.h.service' 2023-10-17T01:33:25.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:25.132 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-17T01:33:25.133 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-17T01:33:25.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.k 2023-10-17T01:33:25.578 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:33:25 smithi060 systemd[1]: Stopping Ceph mds.k for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:26.078 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:33:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k[169992]: 2023-10-17T01:33:25.637+0000 7f8a35c5f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:26.078 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:33:25 smithi060 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k[169992]: 2023-10-17T01:33:25.637+0000 7f8a35c5f700 -1 mds.k *** got signal Terminated *** 2023-10-17T01:33:28.785 INFO:journalctl@ceph.mds.k.smithi060.stdout:Oct 17 01:33:28 smithi060 bash[178982]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-k 2023-10-17T01:33:28.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.k.service' 2023-10-17T01:33:29.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:29.029 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-17T01:33:29.029 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-17T01:33:29.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.c 2023-10-17T01:33:29.372 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:33:29 smithi181 systemd[1]: Stopping Ceph mds.c for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:29.657 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:33:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c[171726]: 2023-10-17T01:33:29.370+0000 7f7eada71700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:29.657 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:33:29 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c[171726]: 2023-10-17T01:33:29.370+0000 7f7eada71700 -1 mds.c *** got signal Terminated *** 2023-10-17T01:33:32.907 INFO:journalctl@ceph.mds.c.smithi181.stdout:Oct 17 01:33:32 smithi181 bash[179106]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-c 2023-10-17T01:33:32.946 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.c.service' 2023-10-17T01:33:32.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:32.986 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-17T01:33:32.986 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-17T01:33:32.987 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.f 2023-10-17T01:33:33.363 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:33:33 smithi181 systemd[1]: Stopping Ceph mds.f for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:33.657 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:33:33 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f[171386]: 2023-10-17T01:33:33.362+0000 7f9794ed8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:33.657 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:33:33 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f[171386]: 2023-10-17T01:33:33.362+0000 7f9794ed8700 -1 mds.f *** got signal Terminated *** 2023-10-17T01:33:36.832 INFO:journalctl@ceph.mds.f.smithi181.stdout:Oct 17 01:33:36 smithi181 bash[179247]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-f 2023-10-17T01:33:37.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.f.service' 2023-10-17T01:33:37.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:37.051 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-17T01:33:37.052 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-17T01:33:37.052 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.i 2023-10-17T01:33:37.407 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:33:37 smithi181 systemd[1]: Stopping Ceph mds.i for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:37.907 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:33:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i[172069]: 2023-10-17T01:33:37.445+0000 7f71d60ad700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:37.907 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:33:37 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i[172069]: 2023-10-17T01:33:37.445+0000 7f71d60ad700 -1 mds.i *** got signal Terminated *** 2023-10-17T01:33:40.794 INFO:journalctl@ceph.mds.i.smithi181.stdout:Oct 17 01:33:40 smithi181 bash[179385]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-i 2023-10-17T01:33:40.985 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.i.service' 2023-10-17T01:33:41.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:41.020 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-17T01:33:41.020 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-17T01:33:41.020 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.l 2023-10-17T01:33:41.403 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:33:41 smithi181 systemd[1]: Stopping Ceph mds.l for 2804d2ac-6c89-11ee-8db6-212e2dc638e7... 2023-10-17T01:33:41.657 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:33:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l[171044]: 2023-10-17T01:33:41.401+0000 7f41d9b86700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-17T01:33:41.657 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:33:41 smithi181 ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l[171044]: 2023-10-17T01:33:41.401+0000 7f41d9b86700 -1 mds.l *** got signal Terminated *** 2023-10-17T01:33:44.907 INFO:journalctl@ceph.mds.l.smithi181.stdout:Oct 17 01:33:44 smithi181 bash[179526]: ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7-mds-l 2023-10-17T01:33:44.987 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2804d2ac-6c89-11ee-8db6-212e2dc638e7@mds.l.service' 2023-10-17T01:33:45.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-17T01:33:45.021 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-17T01:33:45.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-17T01:33:48.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-17T01:33:50.764 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-17T01:33:52.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:33:52.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:33:52.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-17T01:33:52.885 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-17T01:33:52.887 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi045/crash 2023-10-17T01:33:52.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash -- . 2023-10-17T01:33:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-17T01:33:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2023-10-17T01:33:52.981 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi060/crash 2023-10-17T01:33:52.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash -- . 2023-10-17T01:33:53.027 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-17T01:33:53.027 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2023-10-17T01:33:53.029 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi181/crash 2023-10-17T01:33:53.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash -- . 2023-10-17T01:33:53.082 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-17T01:33:53.083 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2023-10-17T01:33:53.084 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-17T01:33:53.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-17T01:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:grep: /var/log/ceph/2804d2ac-6c89-11ee-8db6-212e2dc638e7/ceph.log: No such file or directory 2023-10-17T01:33:53.172 INFO:tasks.cephadm:Compressing logs... 2023-10-17T01:33:53.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-17T01:33:53.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-17T01:33:53.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-17T01:33:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-17T01:33:53.223 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-17T01:33:53.241 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-17T01:39:34.827 INFO:tasks.cephadm:Archiving logs... 2023-10-17T01:39:34.829 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi045/log 2023-10-17T01:39:34.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-17T01:40:05.173 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi060/log 2023-10-17T01:40:05.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-17T01:40:39.399 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi181/log 2023-10-17T01:40:39.400 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-17T01:41:15.012 INFO:tasks.cephadm:Removing cluster... 2023-10-17T01:41:15.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --force 2023-10-17T01:41:16.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --force 2023-10-17T01:41:18.007 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2804d2ac-6c89-11ee-8db6-212e2dc638e7 --force 2023-10-17T01:41:19.077 INFO:tasks.cephadm:Removing cephadm ... 2023-10-17T01:41:19.078 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-17T01:41:19.128 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-17T01:41:19.150 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-17T01:41:19.181 INFO:tasks.cephadm:Teardown complete 2023-10-17T01:41:19.182 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-17T01:41:19.193 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-17T01:41:19.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-17T01:41:19.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-17T01:41:19.201 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-17T01:41:19.340 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-17T01:41:19.351 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-17T01:41:19.352 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-17T01:41:19.357 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-17T01:41:19.360 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-17T01:41:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-10-17T01:41:19.377 INFO:teuthology.orchestra.run.smithi060.stderr:bash: ntpq: command not found 2023-10-17T01:41:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-10-17T01:41:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-17T01:41:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-10-17T01:41:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +684us[ +692us] +/- 84ms 2023-10-17T01:41:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -1338us[-1329us] +/- 69ms 2023-10-17T01:41:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +950us[ +959us] +/- 84ms 2023-10-17T01:41:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:41:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-17T01:41:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-10-17T01:41:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +678us[ +678us] +/- 84ms 2023-10-17T01:41:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -1350us[-1340us] +/- 69ms 2023-10-17T01:41:19.403 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +957us[ +957us] +/- 84ms 2023-10-17T01:41:19.404 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:41:19.407 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-17T01:41:19.407 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-17T01:41:19.407 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +682us[ +683us] +/- 84ms 2023-10-17T01:41:19.408 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -1336us[-1335us] +/- 69ms 2023-10-17T01:41:19.408 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +968us[ +968us] +/- 84ms 2023-10-17T01:41:19.408 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-17T01:41:19.409 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-17T01:41:19.420 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-17T01:41:19.421 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-17T01:41:19.431 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-17T01:41:19.625 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-17T01:41:19.774 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-17T01:41:19.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-17T01:41:19.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:41:19.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-17T01:41:20.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:41:20.009 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-17T01:41:20.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:41:20.044 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-17T01:41:20.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-17T01:41:20.088 INFO:teuthology.task.internal:Duration was 3145.990443 seconds 2023-10-17T01:41:20.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-17T01:41:20.101 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-17T01:41:20.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-17T01:41:20.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-17T01:41:20.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-17T01:41:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-17T01:41:20.208 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-17T01:41:20.224 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-17T01:41:20.434 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-17T01:41:20.434 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2023-10-17T01:41:20.434 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-17T01:41:20.495 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2023-10-17T01:41:20.495 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-17T01:41:20.526 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-10-17T01:41:20.526 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-17T01:41:20.552 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-17T01:41:20.552 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-17T01:41:20.557 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-17T01:41:20.568 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-17T01:41:20.655 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-17T01:41:20.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-17T01:41:20.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-17T01:41:20.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-17T01:41:20.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-17T01:41:20.834 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-17T01:41:20.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-17T01:41:20.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-17T01:41:20.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-17T01:41:20.967 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-17T01:41:20.978 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-17T01:41:20.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-17T01:41:20.985 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-17T01:41:21.010 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2023-10-17T01:41:21.038 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-10-17T01:41:21.048 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2023-10-17T01:41:21.085 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-17T01:41:21.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:41:21.127 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-17T01:41:21.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:41:21.144 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-17T01:41:21.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-17T01:41:21.162 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-17T01:41:21.174 INFO:teuthology.task.internal:Transferring archived files... 2023-10-17T01:41:21.174 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi045 2023-10-17T01:41:21.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-17T01:41:21.260 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi060 2023-10-17T01:41:21.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-17T01:41:21.301 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578/remote/smithi181 2023-10-17T01:41:21.302 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-17T01:41:21.340 INFO:teuthology.task.internal:Removing archive directory... 2023-10-17T01:41:21.341 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-17T01:41:21.346 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-17T01:41:21.348 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-17T01:41:21.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-17T01:41:21.404 INFO:teuthology.task.internal:Not uploading archives. 2023-10-17T01:41:21.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-17T01:41:21.416 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-17T01:41:21.416 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-17T01:41:21.430 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-17T01:41:21.433 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-17T01:41:21.448 INFO:teuthology.orchestra.run.smithi060.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 17 01:41 /home/ubuntu/cephtest 2023-10-17T01:41:21.449 INFO:teuthology.orchestra.run.smithi181.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 17 01:41 /home/ubuntu/cephtest 2023-10-17T01:41:21.469 INFO:teuthology.orchestra.run.smithi045.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 17 01:41 /home/ubuntu/cephtest 2023-10-17T01:41:21.473 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-17T01:41:21.484 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-17T01:41:21.508 INFO:teuthology.nuke:Checking targets against current locks 2023-10-17T01:41:21.536 DEBUG:teuthology.nuke:shortname: smithi045 2023-10-17T01:41:21.536 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-17T01:41:21.559 DEBUG:teuthology.nuke:shortname: smithi060 2023-10-17T01:41:21.559 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-17T01:41:21.563 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-17 00:40:02.310690', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-17T01:41:21.581 DEBUG:teuthology.nuke:shortname: smithi181 2023-10-17T01:41:21.582 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-17T01:41:21.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-17 00:40:02.311847', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-17T01:41:21.608 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-16_14:43:00-fs-wip-yuri4-testing-2023-10-11-0735-reef-distro-default-smithi/7429578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-17 00:40:02.312726', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-17T01:41:21.617 INFO:teuthology.orchestra.console.smithi045:Power off 2023-10-17T01:41:21.618 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-17T01:41:21.640 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2023-10-17T01:41:21.641 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:21.661 INFO:teuthology.orchestra.console.smithi060:Power off 2023-10-17T01:41:21.662 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-17T01:41:21.670 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-10-17T01:41:21.683 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2023-10-17T01:41:21.683 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:21.696 INFO:teuthology.orchestra.console.smithi181:Power off 2023-10-17T01:41:21.697 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-17T01:41:21.706 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-10-17T01:41:21.719 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-10-17T01:41:21.719 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:21.741 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-17T01:41:25.672 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:25.707 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:25.743 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:25.781 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-10-17T01:41:25.817 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-10-17T01:41:25.854 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-17T01:41:29.787 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:29.818 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:29.855 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:29.897 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2023-10-17T01:41:29.926 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2023-10-17T01:41:29.964 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-17T01:41:29.998 INFO:teuthology.orchestra.console.smithi045:Power off completed 2023-10-17T01:41:30.028 INFO:teuthology.orchestra.console.smithi060:Power off completed 2023-10-17T01:41:33.969 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-17T01:41:34.079 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-10-17T01:41:34.180 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-10-17T01:41:34.285 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 3145.990442752838 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-10-17T01:41:34.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-17T01:41:34.372 INFO:teuthology.run:FAIL